A few-shot font generation and copyright protection method based on a generative adversarial network

Abstract

The present application relates to the technical field of font generation, and particularly relates to a few-shot font generation and copyright protection method based on a generative adversarial network, which comprises the following steps: 1. optimizing the original self-attention mechanism by introducing a learnable spatial relationship embedding vector and a shape size correction factor based on a spatial perception attention mechanism; 2. extracting style features and component-level content features from a reference font and a content image by using a style encoder and a content encoder based on attention-enhanced style and content feature extraction; 3. fusing the extracted style features and content features by using a feature mixer, generating a target font image through a feature mapping and decoding process; 4. pre-training and optimizing a watermark decoding network; and 5. embedding and extracting high-quality watermark information on the basis of generating high-quality fonts, thereby realizing font generation and copyright protection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of font generation, in particular to a few-shot font generation and copyright protection method based on a generative adversarial network. BACKGROUND

[0002] In recent years, generative adversarial networks (GAN) have made significant progress in the field of image generation, and are widely used in image synthesis, style transfer, and image inpainting tasks. In the field of font generation, GAN technology has shown great potential by learning and imitating different font styles to achieve high-quality font image generation. However, existing font generation techniques usually require large-scale training data to achieve good results, and their performance in few-shot scenarios is significantly insufficient. In addition, traditional GAN models have difficulty effectively capturing the fine structural features of complex characters such as Chinese characters, resulting in difficulty in achieving structural accuracy and style consistency of generated fonts to meet the requirements of practical applications.

[0003] At the same time, with the widespread application of generative models, copyright protection issues have gradually emerged. Since the development of generative models usually requires a large amount of data, computing resources, and professional knowledge, these models themselves have become important intellectual property assets. Currently, digital watermarking technology has been widely used in the field of digital media copyright protection and is gradually being used to protect deep neural network model intellectual property. Existing neural network watermarking techniques are mainly divided into white-box and black-box categories. White-box watermarking requires access to model internal parameters, while black-box watermarking only needs to check the output of the model to a specific trigger signal, so black-box watermarking is more practical in actual applications. However, current watermarking techniques are mostly focused on discriminative neural networks, and watermarking protection for generative models, especially GANs, still faces challenges.

[0004] Existing methods have difficulty effectively extracting fine style features of fonts under few-shot conditions, lack flexible and effective style and content feature decoupling methods, resulting in insufficient model cross-language generalization performance. Secondly, traditional attention mechanisms mainly focus on feature similarity, ignoring the complex spatial relationships and shape size information between font components, making it difficult to achieve high-precision structure generation and style consistency control. In addition, existing font generation methods lack fine-grained supervision mechanisms at the component level, and it is difficult to effectively ensure the accuracy of generated font structures through overall image-level discrimination. Finally, current generative model copyright protection methods are not perfect, and there is limited research on black-box watermarking techniques for generative models, making it easy for model intellectual property to be infringed, which seriously limits the practical application value of generative models. Therefore, it is urgent to propose a technical solution that can effectively solve the problems of few-shot font generation quality, font component structure accurate supervision, and copyright protection. SUMMARY

[0005] Therefore, the present application aims to provide a few-shot font generation and copyright protection method based on a generative adversarial network to solve the problems of low font generation quality and low component structure precision of the generative adversarial network and imperfect copyright protection means.

[0006] To achieve the above purpose, the technical scheme of the present application is as follows:

[0007] A few-shot font generation and copyright protection method based on a generative adversarial network comprises:

[0008] Step one: based on the optimization of the spatial perception attention mechanism, the original self-attention mechanism is optimized by introducing a learnable spatial relationship embedding vector and a shape size correction factor;

[0009] Step two: style and content feature extraction with attention enhancement, based on the optimized spatial perception attention mechanism, style features and component-level content features are extracted from the reference font and content image using a style encoder and a content encoder respectively;

[0010] Step three: feature fusion and target font generation, the extracted style features and content features are fused using a feature mixer, and through the feature mapping and decoding process, a target font image is generated;

[0011] Step four: watermark decoding network pre-training and optimization, the encoder-decoder architecture is used to pre-train the watermark decoding network for embedding and extracting watermark information;

[0012] Step five: watermark embedding and font generation model copyright protection, the pre-trained watermark decoding network is integrated into the font generation model, and the generator is optimized through backpropagation so that the generated font image contains digital watermark.

[0013] Further, according to the step one based on the optimization of the spatial perception attention mechanism, the original self-attention mechanism is optimized by introducing a learnable spatial relationship embedding vector and a shape size correction factor, and the specific steps include:

[0014] By measuring the difference between the area difference, the perimeter difference and the aspect ratio difference of the components, the shape size correction factor is obtained; the optimized component distance is obtained by optimizing the component distance measurement through the shape size correction factor;

[0015] The spatial relationship weight is obtained by evaluating the component distance through a multi-layer perception, and the optimized attention weight is obtained by combining the spatial relationship weight with the original attention weight.

[0016] Further, according to the step of measuring the difference between the area difference, the perimeter difference and the aspect ratio difference of the components to obtain the shape size correction factor, the specific steps include:

[0017] obtaining the area, the perimeter and the aspect ratio of the any two font components, taking the absolute value of the difference between the areas of the any two font components as the first area difference of the any two font components, dividing the result by the maximum value of the areas of the any two font components, taking the absolute value of the difference between the perimeters of the any two font components as the first perimeter difference of the any two font components, dividing the result by the maximum value of the perimeters of the any two font components, and taking the absolute value of the difference between the aspect ratios of the any two font components as the first aspect ratio difference of the any two font components, dividing the result by the maximum value of the aspect ratios of the any two font components;

[0018] obtaining the learnable weight parameters of the first area difference, the first perimeter difference and the first aspect ratio difference of the font components, respectively, weighting and summing the first area difference, the first perimeter difference and the first aspect ratio difference of the font components according to the learnable weight parameters, and taking the result of the Sigmoid activation function mapping of the weighted sum and the learnable bias term as the shape size difference metric of the any two font components, and adding the result of the calculation to the constant 1 as the shape size correction factor.

[0019] Further, according to the optimization evaluation of the component distance metric by the shape size correction factor, an optimized component distance is obtained, and the specific steps include:

[0020] obtaining the center point coordinates of the any two font components by the component perception module, taking the Euclidean distance of the center point coordinates of the any two font components as the relative distance between the any two font components, dividing the result by the diagonal length of the glyph image, obtaining the shape size correction factor of the font components, and taking the product of the relative distance between the any two font components and the shape size correction factor as the optimized component distance between the any two font components.

[0021] Further, according to the spatial relationship evaluation of the component distance by the multi-layer perception machine, a spatial relationship weight is obtained, and the optimized attention weight is obtained by combining the spatial relationship weight with the original attention weight, and the specific steps include:

[0022] obtaining the spatial relationship embedding vector between the any two font components, mapping the spatial relationship embedding vector and the optimized component distance between the any two components by the multi-layer perception machine, and taking the result of the Sigmoid activation function mapping of the mapping result as the spatial relationship weight between the any two font components, and the calculation formula for obtaining the optimized attention weight by combining the spatial relationship weight with the original attention weight is:

[0023] ;

[0024] wherein, denotes the query vector matrix; denotes the key vector matrix; denotes the value vector matrix; denotes the scaling factor; denotes denotes the normalization function; denotes the transpose of the key vector matrix; denotes the dot product of matrices; denotes the spatial relation weight matrix.

[0025] Further, the multi-layer perception contains two hidden layers, wherein the input of the input layer is the spatial relation embedding vector and the component distance, the dimension number of the first hidden layer is 32, the dimension number of the second hidden layer is 16, the dimension number of the output layer is 1, the output value of the output layer is mapped through the Sigmoid activation function, and the mapped value is taken as the spatial relation weight.

[0026] Further, according to the feature fusion and target font generation in step three, the extracted style features and content features are fused by using a feature mixer, and the target font image is generated through a feature mapping and decoding process, including:

[0027] The font generator is generated and supervised by the component extraction part in the component perception module and the multi-component level discriminator; the style features and content features of the reference font and the content image are obtained, the style features and the content features are fused through the mixer, and the target glyph is generated by the generator.

[0028] Compared with the prior art, the present application has the following advantages:

[0029] The few-shot font generation and copyright protection method based on the generative adversarial network optimizes the self-attention mechanism, introduces learnable spatial relation embedding and shape size correction factor, thereby improving the capturing ability of the network model on the structure features of Chinese character fonts, and improving the quality and style generalization performance of font generation in the few-shot situation. BRIEF DESCRIPTION OF DRAWINGS

[0030] The accompanying drawings, which form a part of this application, are included to provide a further understanding of the application, illustrate the preferred embodiments of the application and assist in

[0031] Figure 1 A method flow chart of the font generation and copyright protection method based on the generative adversarial network according to the embodiments of the present application;

[0032] Figure 2 A model generalization ability experiment chart of the font generation and copyright protection method based on the generative adversarial network according to the embodiments of the present application;

[0033] Figure 3 A qualitative evaluation chart of the font generation and copyright protection method based on the generative adversarial network according to the embodiments of the present application;

[0034] Figure 4 A robustness evaluation chart of the font generation and copyright protection method based on the generative adversarial network according to the embodiments of the present application; DETAILED DESCRIPTION

[0035] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0036] In the description of the present application, it should be noted that the terms "upper", "lower", "inner", "back" and the like indicate the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the present application. In addition, the terms "first", "second" are only for the purpose of description, and cannot be understood as indicating or implying relative importance.

[0037] The present application will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.

[0038] Reference Figure 1 , a method flow chart of a few-shot font generation and copyright protection method based on a generative adversarial network provided by an embodiment of the present application, as Figure 1 shown, a few-shot font generation and copyright protection method based on a generative adversarial network can include:

[0039] Step S1, by introducing a learnable spatial relationship embedding vector and a shape size correction factor, the original self-attention mechanism is optimized.

[0040] In the few-shot font generation task based on , both the style encoder and the content encoder use the self-attention mechanism ( to capture the dependencies between different regions in the input image. The core of self-attention mechanism is the scaled dot-product attention ( ) formula, which determines the attention weights by computing the dot product between the query vector ( , the key vector ( ), and the value vector ( ). However, the standard scaled dot-product attention formula has a significant limitation when applied to Chinese character font generation: it mainly focuses on the similarity between feature vectors, ignoring the spatial positional relationship between features.

[0041] Specifically, the standard scaled dot-product attention formula only considers the dot product between the query vector and the key vector when computing the attention weights, which reflects the feature similarity of the two vectors. However, in Chinese character font generation, it is not enough to consider only the feature similarity. Chinese characters are composed of multiple components arranged in a certain spatial structure. For example, the "ming" character is composed of two components "ri" and "yue" arranged left and right, and the "zi" character is composed of two components "nian" and "zi" arranged up and down. The spatial relationship between these components is crucial to the structure of the font. If the attention mechanism cannot effectively capture these spatial relationships, it will result in the generated partial font structure being chaotic or the relative positions between components being incorrect.

[0042] Step S1.1, by measuring the difference between the area difference, perimeter difference, and aspect ratio difference between components, obtaining the shape size correction factor, and optimizing the component distance measurement by the shape size correction factor, obtaining the optimized component distance.

[0043] In Chinese character fonts, the shape and size of components are important factors affecting visual perception. Even if the center points of two components are very close, if their shapes or sizes differ greatly, they will still give people different visual impressions, and the spatial relationship between them should also be different. For example, a thin vertical line component and a thick square component, even if the center points coincide, their roles in the character shape and their interactions with other components are completely different. Therefore, it is necessary to further improve the distance measurement method to reflect the shape and size information of the components.

[0044] First, obtain the area, perimeter, and aspect ratio of any two font components. Divide the absolute value of the difference between the areas of the two font components by the maximum area of ​​the two font components to obtain the first area difference. Then, divide the absolute value of the difference between the perimeters of the two font components by the maximum perimeter of the two font components to obtain the first perimeter difference. Finally, divide the absolute value of the difference between the aspect ratios of the two font components by the maximum aspect ratio of the two font components to obtain the first aspect ratio difference. Learnable weight parameters for each of the first area difference, first perimeter difference, and first aspect ratio difference are then obtained. Finally, a weighted sum of these learnable weight parameters is applied to the first area difference, first perimeter difference, and first aspect ratio difference between the font components. The result of the weighted summation and the Sigmoid activation function mapping of the learnable bias term are used as the shape size difference measure of the two font components; the result of adding the shape size difference measure of the two font components to a constant 1 is used as the shape size correction factor.

[0045] In one implementation, assume the first The area of ​​each font component is Circumference is The aspect ratio is ;No. The area of ​​each font component is Circumference is The aspect ratio is The area difference weighting parameter is: The weighting parameter for the perimeter difference is The aspect ratio weighting parameter is Then the first The font component and the first The expression for calculating the shape-size correction factor between font components is:

[0046] ;

[0047] in, Indicates the first The font component and the first Shape size correction factor between font components; express Activation function; A weighted parameter representing the difference in perimeter between font components; Indicates the first The area of ​​each font component; Indicates the first The area of ​​each font component; Represents the maximum value function; A weighted parameter representing the difference in perimeter between font components; Indicates the first The perimeter of each font component; Indicates the first The perimeter of each font component; A weighted parameter representing the aspect ratio between font components; Indicates the first The aspect ratio of each font component; Indicates the first The aspect ratio of each font component; Represents a learnable bias term; This indicates absolute value calculation; Indicates the first The font component and the first The first area difference between the font components; Indicates the first The font component and the first The first perimeter difference between the font components; Indicates the first The font component and the first The first aspect ratio difference between the font components.

[0048] It should be noted that in Chinese fonts, the shape and size of components are crucial factors influencing visual perception. Even if two components are close in center, significant differences in shape or size will still evoke different visual feelings, and their spatial relationships should also be distinct. For example, a slender vertical line component and a short, thick square component, even if their center points coincide, will have drastically different roles in the character and interactions with other components. Therefore, distance measurement methods need improvement to reflect the shape and size information of components. The more similar the shapes and sizes of two components, the closer the correction factor is to 1, and the smaller the impact on distance; conversely, the greater the difference in shape and size, the larger the correction factor, thus increasing the perceived distance. Furthermore, the acquisition of the area, perimeter, and aspect ratio of the aforementioned font components is based on the bounding box information provided by the component perception module for each component. The minimum bounding rectangle of the component is calculated, and its area is taken as the component's area, its perimeter as the component's perimeter, and its aspect ratio as the component's aspect ratio.

[0049] After obtaining the shape-size correction factor, the component distance metric can be optimized and evaluated using the shape-size correction factor to obtain the optimized component distance, including:

[0050] The center point coordinates of any two font components are obtained by the component perception module, and the relative distance between the any two font components is obtained by dividing the Euclidean distance of the center point coordinates of the any two font components by the diagonal length of the glyph image. A shape size correction factor of the any two font components is obtained, and the optimized component distance between the any two font components is obtained by multiplying the relative distance between the any two font components and the shape size correction factor.

[0051] In an embodiment, assuming that the center point coordinates of the first font component are and the center point coordinates of the second font component are , the calculation expression of the optimized component distance is:

[0052] ;

[0053] wherein, represents the optimized component distance between the first font component and the second font component; represents the shape size correction factor between the first font component and the second font component; represents the center point coordinates of the first font component; represents the center point coordinates of the second font component; represents the diagonal length of the glyph image; represents the Euclidean distance between the first font component and the second font component.

[0054] After obtaining the optimized component distance, the spatial relationship of the component distance is evaluated by the multilayer perception machine to obtain the spatial relationship weight, and the optimized attention weight is obtained by combining the spatial relationship weight and the original attention weight. The specific steps include:

[0055] The spatial relationship embedding vector between any two font components is obtained, the spatial relationship embedding vector and the optimized component distance between the any two components are mapped by the multilayer perception machine, and the mapping result is mapped by the Sigmoid activation function. The result is used as the spatial relationship weight between the any two font components.

[0056] In an embodiment, the calculation expression of the spatial relationship weight between the first font component and the second font component is:

[0057] ;

[0058] wherein, represents the spatial relationship weight between the th font component and the th font component; represents the spatial relationship embedding vector between the component and the component , which is automatically learned through the back propagation process of the neural network without setting; represents the optimized component distance between the th font component and the th font component; represents the multi-layer perception; represents the activation function.

[0059] It should be noted that the multi-layer perception includes two hidden layers, wherein the input of the input layer is the spatial relationship embedding vector and the component distance, the dimension number of the first hidden layer is 32, the dimension number of the second hidden layer is 16, the dimension number of the output layer is 1, the output value of the output layer is mapped through the Sigmoid activation function, and the mapped value is taken as the spatial relationship weight.

[0060] After obtaining the spatial relationship weight, the optimized attention weight can be obtained by combining the spatial relationship weight and the original attention weight, and the calculation expression of the attention weight is:

[0061] ;

[0062] wherein, represents the query vector matrix; key vector matrix; represents the value vector matrix; represents the scaling factor; represents the normalization function; represents the transpose of the key vector matrix; represents the dot product of the matrix; represents the spatial relationship weight.

[0063] It should be noted that in order to integrate the learned spatial relationship embedding vector into the attention calculation, a small neural network (multi-layer perception, ) is further designed. This The spatial relationship embedding vector and the relative distance between components are received as inputs, and a scalar value is output as a spatial relationship weight. In this way, the spatial relationship embedding vector and the relative distance information are combined to jointly determine the attention weight. Finally, the calculated spatial relationship weight and the original attention weight (i.e. the calculation result of the scaled dot product attention) are multiplied element by element, so that the spatial relationship information is integrated into the final attention weight. In this way, the attention mechanism not only considers the similarity of feature vectors when calculating the weight, but also considers the spatial relationship between components. Therefore, the structural features of Chinese characters can be more accurately captured, and the quality of the generated font can be improved.

[0064] In step S2, based on the optimized spatial-aware attention mechanism, style features and content features are extracted from the reference font and the content image respectively using the style encoder and the content encoder.

[0065] First, the style encoder uses a convolutional neural network (CNN) to extract features. The network structure can include multiple convolutional layers, pooling layers and fully connected layers. The convolutional layer is used to extract low-level and high-level features, the pooling layer is used to reduce the dimension of the feature map, and the fully connected layer is used to output the final style feature vector. In addition, in order to enhance the expression ability of the style features, the attention mechanism optimized in step S1 is introduced into the network. The main task of the style encoder is to extract style features from the reference font image, and to learn the similarity of font styles in the embedding space through metric learning.

[0066] The content encoder also uses a convolutional neural network (CNN) to extract features. The network structure includes multiple convolutional layers and pooling layers, and the attention mechanism is applied after each convolutional layer to enhance the attention to important components. Finally, the content encoder outputs a content feature map for subsequent feature fusion. The content encoder is responsible for extracting component-level content features from the content image. Through the attention mechanism, the content encoder achieves fine-grained feature extraction at the component level. The content encoder converts the content image into a content feature map.

[0067] In step S3, the extracted style features and content features are fused using a feature mixer, and through a feature mapping and decoding process, a target font image is generated.

[0068] The font generator is supervised by the component extraction part in the component-aware module and the multi-component level discriminator; the style features and content features of the reference font and the content image are obtained, the style features and the content features are fused through the mixer, and the target glyph is generated by the generator. The discriminator guides the generator to generate realistic glyph images through adversarial learning. The discriminator not only distinguishes between generated images and real images, but also guides the generator through the integrated style encoder.

[0069] Step S4, pre-train the watermark decoding network using an encoder-decoder architecture for embedding and extracting watermark information.

[0070] A watermark decoding network is pre-trained. The main task of this network is to successfully embed watermark information into the generated image without significantly affecting the image quality, and to accurately extract these watermark information when needed. The watermark decoding network uses an encoder-decoder architecture and is implemented using the StegaStamp network.

[0071] Step S5, integrate the pre-trained watermark decoding network into the font generation model, optimize the generator through backpropagation to make the generated font image contain digital watermark, and protect the model copyright.

[0072] The pre-trained watermark decoding network is effectively integrated with the font generation model. Specifically, the encoder part of the watermark decoding network is added after the output layer of the font generator, and the generator parameters are jointly optimized through backpropagation to automatically and secretly embed predefined digital watermark information in the generated font image. This approach ensures that the generated font image has almost no difference from the unwatermarked image in terms of visual quality, but contains robust and hidden digital watermark information, thereby achieving effective copyright protection for the generated font model and preventing unauthorized misuse or infringement of the model's intellectual property rights.

[0073] To evaluate the method in the Chinese font generation task, and to more conveniently demonstrate the effectiveness of the proposed method, a dataset containing 423 fonts collected in CG-GAN is used. Referring to the experimental design in CG-GAN, 399 fonts are randomly selected as the training set (i.e. seen fonts), each of which contains 800 Chinese characters that can be decomposed into 385 components (i.e. seen characters). The one-time Chinese font generation capability is evaluated on two test sets: one is 399 seen fonts, each containing 150 unseen characters; the other is the remaining 24 unseen fonts, each containing 200 unseen characters. The invention also uses a Korean character test set consisting of 24 unseen fonts, each containing 200 Korean characters, to evaluate the generalization capability for unseen language characters.

[0074] The proposed model was compared with four state-of-the-art methods, including three few-shot Chinese font generation methods (LF-font, MX-Font, and CG-GAN) and one unsupervised image-to-image translation method (FUNIT). For a fair comparison, the proposed model uses SimSun as the source font, a common setting in font generation tasks. LF-Font suffers from lower visual quality during inference if only one reference sample is provided. Therefore, its performance was evaluated in both an eight-sample setting (its original setting) and a one-sample setting. All models were trained from scratch using their official code. In each iteration, a few-shot Chinese font was drawn from the dataset... and Small batches of samples. Select. and This makes them belong to the same font style. However, the character content is different. (Select) To make it have the same Different font styles However, the character content is the same. To calculate... ,use As an anchor point, As a positive sample As a negative sample, it was reused As a generator The input, by Input to discriminator And from the style head By extracting the characteristics, style features were obtained. Used to update the discriminator. The combat loss is through (fake samples) and (Based on real samples) Calculated for updating the generator The combat loss is through (Based on real samples) Calculated and reused Image as and The true value of the loss. R1 regularization, used in the discriminator, is applied. For stable training, the Adam optimizer was set to a learning rate of 0.002. and .

[0075] To verify whether the method proposed in this invention can generate high-quality results on unseen language characters, we conducted this experiment. The model was trained on Chinese character images, but during inference, the trained model was input with Korean (Hangul) characters, which the model had not seen before. Figure 2As shown, the model proposed by the present application is also effective for generating the character shape of Korean characters. The network can transfer the style of the Chinese reference character shape to the unseen Korean source character shape. This experiment shows that the model proposed by the present application has strong generalization ability and can well separate the content and style of the character shape image.

[0076] In the watermark protection experiment, the purpose is to apply the watermark embedding method proposed by the present application for protecting the intellectual property right of the generative adversarial network (GAN) to the combined font generation model, and to verify its effectiveness and robustness. Specifically, the present application will evaluate the watermark embedding effect of the method in the font generation task, and test its robustness and watermark bit accuracy under different post-processing operations. In order to better realize the watermark embedding and extraction of the generated font, multiple public image datasets are used for pre-training. These datasets include 200k CelebA (128x128 resolution), 70k FFHQ (256x256 resolution), 500k LSUN-bedroom (256x256 resolution), and 8k VGG flowers (128x128 resolution). The purpose of pre-training is to utilize the rich information and complex features of these large image datasets to enhance the generalization ability and robustness of the model when processing different image styles and contents. Since the font dataset is small, it cannot provide enough samples to effectively train a complex deep learning model, which to some extent limits the performance of the model. Public image datasets such as CelebA, FFHQ, LSUN-bedroom, and VGG flowers cover a wide range of image contents and styles, providing the model with rich training samples and promoting the model's learning of various visual features.

[0077] By pre-training on these public datasets, the model can learn rich visual features and complex structural information, so that it can still maintain good performance when facing relatively small and single font datasets. After pre-training, the model is fine-tuned for the font generation task to ensure that the generated font images have embedded invisible watermarks while maintaining high image quality. This pre-training and fine-tuning strategy not only improves the model's adaptability to different datasets, but also enhances the stability and robustness of watermark embedding. Specifically, pre-training enables the model to perform well in processing diverse images, while fine-tuning ensures the model's optimization effect on the Chinese and Korean font datasets used in this paper. Ultimately, this method achieves the goal of embedding and extracting watermarks in the font generation process, providing an effective means of copyright protection.

[0078] The model experiment results proposed in the present application are based on the performance of the contrast font generation model under different settings to evaluate the effectiveness and robustness of the watermark embedding method. To objectively evaluate the effect of watermark embedding, the model experiment proposed in the present application uses the following quantitative indicators: peak signal-to-noise ratio (PSNR), channel peak signal-to-noise ratio (CPSNR), mean square error per pixel (MSE), structural similarity index (SSIM), and watermark extraction accuracy (WEA) to train and evaluate the generation model under the few-shot and single-shot settings. Under each setting, experiments are conducted under the conditions of not embedding watermarks and embedding watermarks, and the quality of the generated images and the extraction accuracy of the watermarks are compared. The following table summarizes the quantitative evaluation results under different experimental settings. The experimental data includes PSNR, CPSNR, MSE, SSIM, and WEA indicators under the few-shot and single-shot settings.

[0079]

[0080] From the results in the above table, it can be seen that the embedding of watermarks has little effect on the quality of the generated images. The PSNR and CPSNR values decrease slightly, but remain at a high level, indicating that the generated images are similar in visual quality to the images without embedded watermarks. The slight increase in MSE value reflects the slight changes in the generated images at the pixel level, but overall it is still within an acceptable range. The slight decrease in SSIM value still remains high, indicating that the generated images are similar in structure to the real images. The WEA (Watermark Extraction Accuracy) indicator shows that the embedded watermarks can be reliably extracted, with a watermark extraction accuracy of 98.7% under the few-shot setting and 98.9% under the single-shot setting. This shows that the method proposed in the present application can effectively protect the intellectual property rights of the generated images under different training conditions.

[0081] To more intuitively evaluate the quality of the generated images and the effect of watermark embedding, qualitative analysis is continued. Qualitative evaluation is mainly through visual comparison of generated images and real images to confirm whether watermark embedding has a significant impact on image quality and the invisibility of watermark embedding. Font images generated by the generation model under the conditions of not embedding watermarks and embedding watermarks are qualitatively evaluated. Under each setting, the generator generates a set of font images from random noise, and the quality of the images and the concealment of the watermarks are evaluated by visual comparison. The experimental results are shown in Figure 3 .

[0082] The influence of different image processing methods on bit accuracy and peak signal-to-noise ratio can be seen. When using the processing layer for training, the watermark embedded in the generated image by the proposed GAN is very robust, and the robustness is improved compared with the method in the literature. In particular, the watermark bit accuracy in the present scheme remains above 75% in a relatively wide range of perturbations, and when the distortion introduced by the image is very high (low PSNR), the bit accuracy will drop below this value, making the image unusable. For example, when the generated image is JPEG compressed with a quality factor of 50, the bit accuracy is about 70%, which in practice means one re-compression. Figure 4 Some examples of processed font images are shown, as well as the bit accuracy of the retrieved watermark under very strong attacks in various cases. Figure 4 The watermark bit accuracy and PSNR values of the images generated using the proposed GAN embedding method in different processed images are shown in the results. After the images are processed in various ways (such as JPEG compression, noise addition, blurring, brightness adjustment, inversion, downsampling), the robustness of the watermark and the image quality are evaluated. The results show that the bit accuracy after JPEG compression is 65%, the bit accuracy after adding noise is 77%, the bit accuracy after blurring is 92%, the bit accuracy after increasing brightness is 78%, the bit accuracy after color inversion is 90%, and the bit accuracy after downsampling is 88%. These results show that the proposed GAN embedding method in the present invention maintains a bit accuracy of the watermark above 75% in a wide range of perturbations, and only when the image introduces very high distortion does the bit accuracy decrease significantly. For example, the bit accuracy after JPEG compression (quality factor 50) is about 70%. Overall, blurring and color inversion have the least impact on the watermark, while brightness adjustment and noise addition have a greater impact on the watermark. These results verify that the proposed GAN embedding method in the present invention can embed a watermark with strong robustness while ensuring the quality of the generated image, effectively improving the copyright protection capability of the image.

[0083] In summary, the method for generating and protecting the copyright of a few-sample font based on a generative adversarial network optimizes the self-attention mechanism, introduces a learnable spatial relationship embedding and a shape size correction factor, thereby improving the network model's ability to capture the structural features of Chinese characters and enhancing the quality and style generalization performance of font generation in a few-sample context. The introduction of the component perception module in the present invention realizes component-level supervision and fine optimization of the font generation process, and through component extraction and multi-component level discriminators, it ensures that the generated font images are accurate in component structure, reasonable in layout, and uniform in style, effectively avoiding the common structural errors and inconsistent styles in traditional GAN font generation methods.

[0084] The above merely provides the preferred embodiment of the present application, and is not used to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. A method for few-shot font generation and copyright protection based on generative adversarial networks, characterized in that: Step 1: Optimize the attention mechanism based on spatial awareness by introducing learnable spatial relationship embedding vectors and shape-size correction factors to optimize the original self-attention mechanism; Step 2: Attention-enhanced style and content feature extraction. Based on the optimized spatial awareness attention mechanism, style features and component-level content features are extracted from the reference font and content image using the style encoder and content encoder, respectively. Step 3: Feature fusion and target font generation. A feature mixer is used to fuse the extracted style features and content features. Through feature mapping and decoding, the target font image is generated. Step 4: Watermark decoding network pre-training and optimization. The watermark decoding network is pre-trained using an encoder-decoder architecture for embedding and extracting watermark information. Step 5: Watermark Embedding and Font Generation Model Copyright Protection. The pre-trained watermark decoding network is integrated into the font generation model, and the generator is optimized through backpropagation so that the generated font image contains a digital watermark. The optimization of the spatial awareness-based attention mechanism is achieved by introducing a learnable spatial relationship embedding vector and a shape-size correction factor to optimize the original self-attention mechanism. This includes: obtaining the shape-size correction factor by measuring the differences in area, perimeter, and aspect ratio between components. The component distance metric is optimized and evaluated by a shape-size correction factor to obtain the optimized component distance; the spatial relationship of the component distance is evaluated by a multilayer perceptron to obtain the spatial relationship weight; and the optimized attention weight is obtained by combining the spatial relationship weight with the original attention weight.

2. The method for few-shot font generation and copyright protection based on generative adversarial networks according to claim 1, characterized in that, Based on the aforementioned method of measuring the differences in area, perimeter, and aspect ratio between components to obtain a shape size correction factor, the specific steps include: Obtain the area, perimeter, and aspect ratio of any two font components. The absolute value of the difference between the areas of the two font components, divided by the maximum area of ​​the two font components, is used as the first area difference between the two font components. Similarly, the absolute value of the difference between the perimeters of the two font components, divided by the maximum perimeter of the two font components, is used as the first perimeter difference between the two font components. Finally, the absolute value of the difference between the aspect ratios of the two font components, divided by the maximum aspect ratio of the two font components, is used as the first aspect ratio difference between the two font components. Learnable weight parameters are obtained for the first area difference, first perimeter difference, and first aspect ratio difference of the font components respectively. The learnable weight parameters of the first area difference, first perimeter difference, and first aspect ratio difference of the font components are weighted and summed. The result of the weighted summation is then mapped to the learnable bias term using the Sigmoid activation function, and the result is used as the shape size difference measure of any two font components. The shape size difference measure of any two font components is added to a constant 1, and the result is used as the shape size correction factor.

3. The method for few-shot font generation and copyright protection based on generative adversarial networks according to claim 1, characterized in that, The optimized component distance is obtained by optimizing the component distance metric using the shape-size correction factor, and the specific steps include: The center point coordinates of any two font components are obtained through the component perception module. The Euclidean distance between the center point coordinates of the two font components is divided by the diagonal length of the character image to obtain the relative distance between the two font components. The shape size correction factor of the two font components is obtained. The product of the relative distance between the two font components and the shape size correction factor is obtained to obtain the optimized component distance between the two font components.

4. The method for few-shot font generation and copyright protection based on generative adversarial networks according to claim 1, characterized in that, The spatial relationship evaluation of component distances using a multilayer perceptron is described above, resulting in spatial relationship weights. These spatial relationship weights are then combined with the original attention weights to obtain optimized attention weights. The specific steps include: Obtain the spatial relationship embedding vector between any two font components. Map the spatial relationship embedding vector to the optimized component distance between the two components using a multilayer perceptron. Apply the mapping result to a Sigmoid activation function and use the result as the spatial relationship weight between the two font components. The optimized attention weight is obtained by combining the spatial relationship weight with the original attention weight. The formula for calculating the optimized attention weight is as follows: ； in, Represents the query vector matrix; Key vector matrix; Represents a value vector matrix; Indicates the scaling factor; express Normalization function; This represents the transpose of the key vector matrix; Represents the dot product of matrices; This represents the spatial relationship weight matrix.

5. The method for few-shot font generation and copyright protection based on generative adversarial networks according to claim 4, characterized in that, The multilayer perceptron contains two hidden layers. The input layer takes a spatial relationship embedding vector and the distance between components as inputs. The first hidden layer has a dimension of 32, the second hidden layer has a dimension of 16, and the output layer has a dimension of 1. The output values ​​are mapped using a sigmoid activation function, and the mapped values ​​are used as spatial relationship weights.

6. The method for few-shot font generation and copyright protection based on generative adversarial networks according to claim 1, characterized in that, Based on the feature fusion and target font generation in step three, a feature mixer is used to fuse the extracted style features and content features. Through feature mapping and decoding processes, a target font image is generated, including: The font generator is supervised by the component extraction part in the component awareness module and the multi-component level discriminator; the style features and content features of the reference font and content image are obtained, the style features and content features are fused by the mixer, and the target glyph is generated by the generator.

Images