A calligraphy image personalized generation method based on calligraphy aesthetics

Abstract

A calligraphy image personalized generation method based on calligraphy aesthetics, a calligraphy image personalized generation network including a generation unit and a discriminator is constructed. In the generation unit, the feature adaptive fusion module generates channel weights by splicing style features and global style features, modulates local spatial features channel by channel, continuously constrains the local stroke generation process with the overall aesthetic target, effectively strengthens the details related to the target attributes and suppresses the mismatched features; the aesthetic injection module linearly weights the input attributes and the target attributes through the attribute proportion parameter and injects them into the decoder multi-scale layer, so that large-scale features focus on character structure, medium and small-scale features focus on stroke form and detail expression, and hierarchical transmission of aesthetic attributes is realized; the discriminator performs adversarial discrimination and aesthetic feature extraction in the training stage. Through adaptive fusion and multi-scale attribute guidance, the calligraphy image personalized generation with controllability and consistency is realized.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of image processing and computer vision, and specifically relates to a method for personalized generation of calligraphy images based on calligraphic aesthetics. Background Technology

[0002] Personalized generation of calligraphy aesthetic images aims to generate calligraphy images that meet target requirements in terms of character structure, stroke form, and overall aesthetics, based on user-defined aesthetic attributes. Compared to general font generation tasks, personalized generation of calligraphy aesthetic images not only requires the generated results to maintain recognizability at the content level, but also requires them to demonstrate strong consistency in structural layout, style, and aesthetic expression.

[0003] However, existing generative models often separate global semantic modeling from local detail generation when handling calligraphy style generation tasks. This makes it difficult for overall style information to continuously influence the local writing process during generation. This leads to two problems: First, when global features only provide coarse-grained information in the initial generation stage, subsequent local feature generation can easily deviate from the user-defined aesthetic goals. This results in local writing details highly correlated with the target attributes not being effectively enhanced, while local responses inconsistent with the overall aesthetic trend may be retained. Second, calligraphy image generation networks typically maintain feature representations at multiple spatial scales simultaneously. If the influence of different scales on aesthetic expression is ignored during feature fusion, and multi-scale features are processed uniformly, it can easily lead to insufficient transmission of aesthetic attributes at different levels, resulting in an imbalance between the overall structure and local details in the generated result.

[0004] Personalized generation of calligraphy images involves not only preserving character content but also the coordinated expression of aesthetic attributes such as stroke thickness, turning force, structural tightness, composition, and overall spirit. Existing font generation or attribute editing methods typically rely on conditional vectors, style encoding, or simple feature concatenation for generation control. However, these approaches struggle to ensure the global aesthetic goal consistently functions across all stages of the generation network, easily leading to inconsistencies between local stroke details and the overall style. Furthermore, in the calligraphy image generation process, features at different scales correspond to the overall structure, spatial layout, and local stroke details, respectively. A unified fusion approach for multi-scale features can result in an imbalance in the expression of aesthetic attributes at different levels, consequently affecting the structural stability, aesthetic consistency, and naturalness of the generated image. Summary of the Invention

[0005] To address the aforementioned problems, the purpose of this invention is to provide a personalized calligraphy image generation method based on calligraphy aesthetics, in order to solve the problems of disconnect between global semantics and local generation processes and imbalance in attribute expression caused by unified processing of multi-scale features in the prior art.

[0006] To achieve the above objectives, the technical solution adopted by the present invention includes:

[0007] A method for constructing a personalized calligraphy image generation model based on calligraphy aesthetics includes the following steps: S1. Obtain the calligraphy image dataset, which includes N calligraphy images, each corresponding to an input attribute vector. Input attribute vector The process involves multiple dimensions, each corresponding to a different calligraphic aesthetic style; N calligraphic images are used, including one input image and N-1 style images. The input image provides character structure information, and the style images provide style reference information. The process involves obtaining a realistic image that matches the character structure information provided by the input image and the style reference information provided by the style images; and setting the input attribute vector. Target attribute vectors of equal dimensions and attribute percentage parameters ; S2, Construct a personalized calligraphy image generation network. The personalized calligraphy image generation network includes a generation unit and a discriminator. The generation unit includes a feature encoder, a style encoder, a feature adaptive fusion module, an aesthetic injection module, and a decoder. The feature encoder is used to extract features from calligraphy images to obtain global style features and local spatial features; the style encoder is used to extract features from style images to obtain style features. The input of the feature adaptive fusion module is connected to the output of the feature encoder and the output of the style encoder, respectively. It is used to receive the global style features and local spatial features output by the feature encoder and the style features output by the style encoder. The style features and global style features are concatenated together to generate channel weights to modulate the local spatial features channel by channel, and the resulting enhanced intermediate features are output to the decoder. The input of the aesthetic injection module is connected to both the output of the style encoder and the network input, and is used to receive style features and input attribute vectors. Target attribute vector and attribute percentage parameters , input attribute vector and target attribute vector By attribute percentage parameter The personalized attribute condition vector is obtained by performing linear weighting. And based on personalized attribute condition vectors Generate attribute-guided features from style features, and output the attribute-guided features to the decoder; The decoder receives the enhanced intermediate features output by the feature adaptive fusion module and the attribute-guided features output by the aesthetic injection module, performs stepwise decoding, and outputs the generated image. The discriminator includes a discrimination branch and an aesthetic feature extraction branch. The discrimination branch is used to distinguish between the generated image and the real image output by the decoder to assist adversarial training. The aesthetic feature extraction branch is used to extract the aesthetic feature vectors of the generated image and the real image. S3 combines calligraphy image datasets, real images, and target attribute vectors. and attribute percentage parameters The personalized calligraphy image generation network constructed using S2 is input and trained with an overall optimization objective function and an adversarial loss function to obtain trained generation units. These trained generation units are then used as the personalized calligraphy image generation model. The overall optimization objective function is as follows:

[0008] in, For pixel-level reconstruction constraints, For structure-aware constraints. For aesthetic consistency constraints, , and These represent the weights of the primitive-level reconstruction loss, the structure perception loss, and the aesthetic consistency loss, respectively.

[0009] Preferably, in S2, the feature adaptive fusion module concatenates style features and global style features to generate channel weights, and modulates local spatial features channel by channel, specifically including: The input image is fed into a feature encoder for feature extraction, resulting in feature map F. Global style features are then obtained from feature map F through spatial aggregation. Mapping the feature map F yields local spatial features. in, This represents a spatial aggregation operator used to preserve overall statistical characteristics; Represents a linear mapping function. Composed of convolution operators, used to characterize local structures; These represent the height, width, and number of channels of a local spatial feature, respectively. The style feature output by the style encoder is denoted as s. The global style feature g is concatenated with the style feature s to generate the channel weights. ;in, and It is a linear mapping function. This represents a non-linear activation operation. This is the Sigmoid activation function, used to constrain the stability of the weight distribution; The local spatial features are modulated channel by channel according to the channel weight 'a' to obtain the modulated local features. Where h, w, and c represent the height index, width index, and channel index of the enhanced intermediate features, respectively. This represents the weight value of the c-th channel; Modulated local features The enhanced intermediate features are obtained by back-injecting the feature integration mapping function into the backbone feature stream, using F as the backbone feature. ;in, This represents the feature integration mapping function, used to ensure the consistency of the modulated features with the backbone features in terms of dimension and distribution.

[0010] Preferably, the aesthetic injection module in S2 will input the attribute vector. and target attribute vector By attribute percentage parameter The personalized attribute condition vector is obtained by performing linear weighting. ; Personalized attribute condition vector The style features output by the style encoder are concatenated to form an aesthetic embedding vector; Obtain multiple sets of features at different spatial scales output by the intermediate layers of the decoder. ;in, Represents the total number of spatial scales. Indicates the first Characteristics of each spatial scale; Spatial aggregation operations are performed on features at each spatial scale to obtain a joint representation. ; Through learnable mapping functions Jointly expressed Mapped to a set of spatially scale-dependent modulation weights ;in, This represents the modulation weight at the s-th spatial scale; Based on the modulation weights, aesthetic attribute-driven recalibration of features at each scale is performed to obtain... ; where ⊙ represents element-wise multiplication; right Integration is achieved through a multi-scale fusion mapping function, outputting the multi-scale modulated feature representation. And use it as an attribute-guided feature; among which... This represents the multi-scale fusion mapping function.

[0011] Preferably, in S3, the personalized generation network for calligraphy images is trained, specifically including the following steps: S31, combine the input image, style image, real image, and target attribute vector from the calligraphy image dataset. and attribute percentage parameters Personalized generation network based on input calligraphy images; S32, input the input image into the feature encoder for feature extraction to obtain global style features and local spatial features; input the style image into the style encoder for feature extraction to obtain style features; input the real image as the supervisory signal of the discriminator into the discriminator; S33, the feature adaptive fusion module generates channel weights based on the splicing results of global style features and style features, and modulates local spatial features channel by channel to obtain enhanced intermediate features; S34, The aesthetic injection module is based on the input attribute vector. Target attribute vector and attribute percentage parameters Calculate the personalized attribute condition vector And based on personalized attribute condition vectors Style features are used to generate attribute-guided features; S35, the decoder progressively decodes the enhanced intermediate features and attribute-guided features, and outputs the generated image; S36. The loss value between the generated image and the real image is calculated using the overall optimization objective function and the adversarial loss function. The network parameters of the generator and the discriminator are updated according to the loss value through the backpropagation algorithm until the overall optimization objective function converges, and the trained generator is obtained.

[0012] Preferred attribute percentage parameters During training, values ​​are randomly sampled. In each iteration, a value is randomly sampled from the interval [0,1] as the attribute percentage parameter for the current iteration. .

[0013] Preferably, the pixel-level reconstruction constraints are: ;in, Represents a true calligraphy image. This represents the generated image output by the decoder. Denotes the L1 norm; the structure-aware constraint is: ;in, This represents the feature extraction network used to extract mid-to-high-level structural representations; aesthetic consistency constraint: ;in, The aesthetic feature extraction branch of the discriminator is used to generate the image. Extracted aesthetic feature vectors This represents the square of the L2 norm.

[0014] A method for personalized generation of calligraphy images based on calligraphic aesthetics includes the following steps: Step 1: Obtain the input image and the user-specified target attribute vector. Target attribute vector It is multi-dimensional, with each dimension corresponding to a different style of calligraphic aesthetics; Step 2: Combine the input image and the user-specified target attribute vector. Input the personalized calligraphy image generation model constructed by the model construction method according to any one of claims 1 to 6, and specify the attribute proportion parameter. ; Step 3: Output the generated image.

[0015] A computer-readable storage medium storing a computer program, which, when executed by a processor, implements the method for personalized generation of calligraphy images based on calligraphic aesthetics disclosed in this application.

[0016] A computer program product includes a computer program / instructions that, when executed by a processor, implement the method for personalized generation of calligraphy images based on calligraphic aesthetics disclosed in this application.

[0017] Compared with the prior art, the advantages of the present invention are: (1) The personalized generation method of calligraphy image based on calligraphy aesthetics of the present invention generates channel weights by combining global style features and style features through feature adaptive fusion module, and modulates local spatial features channel by channel, so that local features that are highly related to the target aesthetic attributes are strengthened and local details that are inconsistent with the overall style orientation are suppressed, thereby realizing that the overall aesthetic goal continuously constrains the local stroke generation process during the generation process, and solving the problem of disconnect between global semantics and local generation process.

[0018] (2) The personalized calligraphy image generation method based on calligraphy aesthetics of the present invention obtains multiple sets of feature representations of different spatial scales output by multiple intermediate layers of the decoder through the aesthetic injection module. After spatial aggregation of each scale feature, it is mapped to scale-related modulation weights. Based on these weights, each scale feature is recalibrated with aesthetic attribute drive, so that the large-scale features focus on the character structure and composition layout, and the medium and small-scale features focus on the stroke form and local detail expression, thus solving the problem of attribute expression imbalance caused by unified processing of multi-scale features.

[0019] (3) The personalized generation method of calligraphy image based on calligraphy aesthetics of the present invention sets the discriminator to a dual-branch structure including a discriminative branch and an aesthetic feature extraction branch. The discriminative branch is used for adversarial training, and the aesthetic feature extraction branch is directly used to extract the aesthetic feature vectors of the generated image and the real image to calculate the aesthetic consistency constraint. The extraction and constraint of aesthetic features are realized without adding extra network modules and training overhead. Attached Figure Description

[0020] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the following detailed description to explain the invention, but do not constitute a limitation thereof. In the drawings: Figure 1 This is a general framework diagram of the personalized calligraphy image generation model according to the present invention.

[0021] Figure 2 This is a schematic diagram of the specific network structure of the Feature Adaptive Fusion Module (AG-GLC module).

[0022] Figure 3 A schematic diagram of the specific network structure of the Aesthetic Injection Module (MSAM module).

[0023] Figure 4 This is a schematic diagram comparing the image generation effects of the "flat" aesthetic style according to an embodiment of the present invention.

[0024] Figure 5 This is a schematic diagram comparing the image generation effects of the "dejected" aesthetic style according to an embodiment of the present invention.

[0025] Figure 6 This is a schematic diagram comparing the image generation effects of the "stiff" aesthetic style according to an embodiment of the present invention.

[0026] Figure 7 This is a schematic diagram comparing the image generation effects of a "solemn" aesthetic style according to an embodiment of the present invention.

[0027] Figure 8 This is a schematic diagram comparing the image generation effects of the "majestic" aesthetic style according to an embodiment of the present invention.

[0028] Figure 9 This is a schematic diagram comparing the image generation effects of the "chaotic" aesthetic style according to an embodiment of the present invention. Detailed Implementation

[0029] The invention is not limited to the specific embodiments described below. All equivalent modifications made based on the technical solutions of this application fall within the protection scope of this invention. Unless otherwise specified, all components and devices in this invention utilize components and devices known in the prior art.

[0030] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. The following embodiments are for illustrative purposes only and should not be considered as limiting the scope of the invention.

[0031] Example 1 This embodiment provides a method for constructing a personalized calligraphy image generation model based on calligraphy aesthetics.

[0032] I. Data Preparation and Network Construction Obtain a dataset of calligraphy images. This dataset contains N calligraphy images, each corresponding to an input attribute vector. Input attribute vector It is multi-dimensional, with each dimension corresponding to a different calligraphic aesthetic style. In one specific implementation, the calligraphic aesthetic attributes total 37 dimensions, corresponding in order to: vigorous, elegant, firm, magnanimous, solemn, lively, archaic, mellow, flowing, light, dignified, delicate, refined, warm, rigorous, graceful, spacious, daring, rough, stubborn, rigid, weak, desolate, superficial, mixed, listless, aged, immature, unrestrained, even, slanted, gentle, worldly, pure and firm, harmonious, lofty, simple and quiet.

[0033] From N calligraphy images, one is selected as the input image to provide character structure information, and the remaining N-1 images are used as style images to provide style reference information. Simultaneously, a real image matching the character structure information provided by the input image and the style reference information provided by the style images is obtained; this real image serves as the supervision signal for the discriminator during the training phase. The input attribute vector is defined... Target attribute vectors of equal dimensions And the attribute proportion parameter P∈[0,1]P∈[0,1].

[0034] Construct a personalized calligraphy image generation network. (Refer to...) Figure 1 The network includes a generator unit and a discriminator. The generator unit includes a feature encoder, a style encoder, an adaptive feature fusion module (AG-GLC module), an aesthetic injection module (MSAM module), and a decoder. The discriminator includes a discrimination branch and an aesthetic feature extraction branch.

[0035] The feature encoder is used to extract features from calligraphy images, obtaining global style features and local spatial features. The style encoder is used to extract features from style images, obtaining style features.

[0036] The input terminals of the AG-GLC module are connected to the output terminals of the feature encoder and the style encoder, respectively, to receive the global style features and local spatial features output by the feature encoder, as well as the style features output by the style encoder. This module concatenates the style features and global style features to generate channel weights, performs channel-by-channel modulation on the local spatial features, and outputs the resulting enhanced intermediate features to the decoder.

[0037] The input terminals of the MSAM module are connected to the output terminal of the style encoder and the network input terminal, respectively, to receive style features and input attribute vectors. Target attribute vector And the attribute proportion parameter P. This module will take the attribute vector as input. and target attribute vector The personalized attribute condition vector is obtained by linearly weighting the attribute proportion parameter P. And based on personalized attribute condition vectors Generate attribute-guided features from style features, and output the attribute-guided features to the decoder.

[0038] The decoder receives the enhanced intermediate features output by the AG-GLC module and the attribute-guided features output by the MSAM module, performs step-by-step decoding, and outputs the generated image.

[0039] The discriminator's discriminant branch is used to distinguish between the generated image and the real image output by the decoder to assist in adversarial training, while the discriminator's aesthetic feature extraction branch is used to extract the aesthetic feature vectors of the generated image and the real image.

[0040] II. Feature Adaptive Fusion Processing Reference Figure 2 The specific processing procedure of the AG-GLC module is as follows.

[0041] The input image is fed into a feature encoder for feature extraction, resulting in a feature map F. Global style features are then obtained from feature map F through spatial aggregation.

[0042] in, This represents a spatial aggregation operator used to preserve overall statistical characteristics; This represents a linear mapping function. The vector g is considered an abstract description of the overall aesthetic state of the current image, reflecting high-level aesthetic characteristics such as structural tightness, visual balance, and consistency of aesthetic style.

[0043] The feature map F is processed by a mapping function. By performing mapping, local spatial features are obtained:

[0044] in, Composed of convolution operators, it is used to characterize local structures, reflecting strokes, edges, textures, and local morphological changes during the writing process. These represent the height, width, and number of channels of the feature map, respectively.

[0045] The style feature output by the style encoder is denoted as s. The global style feature g and the style feature s are concatenated to generate the channel weights.

[0046] in, and It is a linear mapping function. This represents a non-linear activation operation. is the Sigmoid activation function, configured to constrain the stability of the weight distribution. The value of each dimension in the channel weight a can reflect the correlation degree between the local details encoded by the corresponding channel and the target aesthetic attribute.

[0047] Per-channel modulation is performed on the local spatial features according to the channel weight a, to obtain modulated local features:

[0048] where h, w and c respectively denote the height direction index, width direction index and channel index of the feature map, denotes the weight value of the c-th channel. Local features highly correlated with the target aesthetic attribute are enhanced, while local details inconsistent with the overall style orientation are suppressed.

[0049] The modulated local features are injected back into the backbone feature flow through a feature integration mapping function, with F as the backbone feature, to obtain enhanced intermediate features:

[0050] where denotes the feature integration mapping function composed of convolutional layers, configured to ensure the consistency of dimensions and distribution between the modulated local features and the backbone features. The above formula adopts a residual structure, which can avoid excessive suppression of the local generation freedom by the global aesthetic constraint. Y is used as the enhanced intermediate feature and output to the decoder.

[0051] III. Aesthetic Attribute Guided Injection Processing With reference to Figure 3 , the specific processing procedure of the MSAM module is as follows.

[0052] The input attribute vector and the target attribute vector are linearly weighted by the attribute proportion parameter P, to obtain a personalized attribute condition vector:

[0053] When P=0, the generation condition completely retains the input attribute; when P=1, the generation condition is completely determined by the target attribute; when 0<P<1, the input attribute and the target attribute are linearly mixed in proportion. The personalized attribute condition vector is concatenated with the style feature output by the style encoder, to form an aesthetic embedding vector.

[0054] Multiple groups of feature representations of different spatial scales respectively output by a plurality of intermediate layers of the decoder are acquired:

[0055] where s denotes the total number of spatial scales, This represents the feature of the s-th spatial scale.

[0056] Spatial aggregation operations are performed on features at various spatial scales to jointly compute multi-scale information:

[0057] In this context, [ ] represents a vector concatenation operation.

[0058] Through learnable mapping functions The joint representation z is mapped to a set of spatially scale-dependent modulation weights:

[0059] in, This represents the modulation weight of the s-th spatial scale, used to characterize the importance of the scale feature under current aesthetic conditions.

[0060] Aesthetic attribute-driven recalibration of features at various scales based on modulation weights:

[0061] Here, ⊙ represents element-wise multiplication.

[0062] right Integration is achieved through a multi-scale fusion mapping function, and the output multi-scale modulated feature representation serves as attribute-guided features:

[0063] Here, Fm() represents the multi-scale fusion mapping function composed of convolutional layers. Through the above operations, the large-scale layers are responsible for modeling the character structure and composition, while the medium and small-scale layers further depict the aesthetic attributes such as stroke shape and local details.

[0064] IV. Model Training The input image, style image, real image, and target attribute vector from the calligraphy image dataset are combined. The personalized calligraphy image generation network is constructed by inputting the attribute proportion parameter P.

[0065] The input image is input to the feature encoder for feature extraction, resulting in global style features and local spatial features; the style image is input to the style encoder for feature extraction, resulting in style features; the real image is used as the supervisory signal for the discriminator.

[0066] The AG-GLC module generates channel weights based on the concatenation of global and style features, and modulates local spatial features channel by channel to obtain enhanced intermediate features. The MSAM module generates channel weights based on the input attribute vector. Target attribute vector Calculate the personalized attribute condition vector using the attribute proportion parameter P. And based on personalized attribute condition vectors The style features are used to generate attribute-guided features. The decoder progressively decodes the enhanced intermediate features and attribute-guided features to output the generated image.

[0067] The loss value between the generated image and the real image is calculated using the overall optimization objective function and the adversarial loss function. The network parameters of the generator unit and the discriminator are then updated using the backpropagation algorithm based on the loss value until the overall optimization objective function converges, thus obtaining the trained generator unit.

[0068] The overall optimization objective function is as follows:

[0069] in, For pixel-level reconstruction constraints, For structure-aware constraints. For aesthetic consistency constraints, , and These are the weights for pixel-level reconstruction loss, structure-aware loss, and aesthetic consistency loss, respectively, and all are positive numbers. In one specific embodiment, , , The weights mentioned above can be adjusted based on the training dataset, image resolution, and the structural preservation and aesthetic consistency of the generated images.

[0070] The specific details of each loss item are as follows: Pixel-level reconstruction constraints: ; in, Represents a true calligraphy image. This represents the generated image output by the decoder. This represents the L1 norm.

[0071] Structure-aware constraints: ; in, This represents a feature extraction network used to extract representations of mid-to-high-level structures.

[0072] Aesthetic consistency constraints: ; in, The aesthetic feature extraction branch pairs of the discriminator generate images. Extracted aesthetic feature vectors This represents the square of the L2 norm.

[0073] In one specific implementation, the attribute proportion parameter P is randomly sampled during the training process, and a value is randomly sampled from the interval [0,1] in each iteration as the attribute proportion parameter P for the current iteration.

[0074] After training, the trained generator units are used as a personalized calligraphy image generation model for inference.

[0075] The proposed model was compared with existing mainstream attribute editing generation models (FaceCLIP, MagicFace, AUEditNet, and EmoStyle) on a public calligraphy dataset. As shown in Table 1, the experimental results demonstrate that, under the same experimental conditions, the proposed method achieves superior performance on multiple evaluation metrics, including IS, FID, LPIPS, SSIM, and Pix-acc.

[0076] Table 1. Performance comparison of different methods on public calligraphy datasets

[0077] Further ablation experiments show that, after introducing the AG-GLC module, the model can explicitly enhance local features highly correlated with the target aesthetic attributes and suppress mismatched features. The introduction of the MSAM module successfully verifies the model's ability to continuously model changes in aesthetic attributes and its controllable generation capability. Experiments on the continuous adjustment of the attribute proportion parameter P show that, as P changes from 0 to 1, the aesthetic attributes of the generated image smoothly transition from the input style to the target style, verifying the controllability and interpretability of this invention in personalized calligraphy image generation. Example 2 This embodiment provides a method for personalized generation of calligraphy images based on calligraphy aesthetics, using the personalized generation model of calligraphy images trained in Embodiment 1.

[0078] I. Obtaining Input Obtain the input image and the user-specified target attribute vector. Target attribute vector It is multi-dimensional, with each dimension corresponding to a different calligraphic aesthetic style. Its dimensions are related to the input attribute vector used in the training phase. The dimensions are equal. The user also specifies the attribute percentage parameter P∈[0,1].

[0079] II. Feature Extraction Input the input image into the feature encoder of the calligraphy image personalized generation model for feature extraction, to obtain global style features and local spatial features. Acquire a style image, and input the style image into a style encoder for feature extraction, to obtain style features. It should be noted that in one embodiment, the style image may be provided by a user; in another embodiment, the style image may also be selected from a training dataset. The style features extracted by the style encoder are used for subsequent feature modulation and attribute guidance.

[0080] III. Adaptive Fusion and Attribute Injection The AG-GLC module generates channel weights according to the splicing result of the global style features and the style features, and performs channel-by-channel modulation on the local spatial features to obtain enhanced intermediate features.

[0081] The MSAM module acquires the input attribute vector of the input image , and calculates a personalized attribute condition vector according to the attribute proportion parameter P ; When P=0, the generation condition completely retains the input attribute vector ; when P=1, the generation condition is completely determined by the target attribute vector ; when 0<P<1, the input attribute vector and the target attribute vector are linearly mixed in proportion. The MSAM module performs feature fusion and recalibration at the multi-scale level according to the personalized attribute condition vector and the style features to obtain attribute-guided features.

[0082] IV. Decoding Generation The decoder performs step-by-step decoding on the enhanced intermediate features output by the AG-GLC module and the attribute-guided features output by the MSAM module, and outputs a generated image.

[0083] During the inference process, the attribute proportion parameter P can be flexibly specified by the user according to actual needs, so as to realize continuous transition control from the input style to the target style. When P approaches 0, the generation result tends to retain the original aesthetic style of the input image; when P approaches 1, the generation result approaches the target aesthetic style specified by the user; when P takes an intermediate value, the generation result presents a fusion effect of the two styles. The inference process does not involve loss function calculation and parameter update.

[0084] Figures 4 to 9The figures show a comparison of the generation effects of the method of this invention (CAIG model) with existing attribute editing generation models FaceCLIP, MagicFace, AUEditNet, and EmoStyle on six calligraphic aesthetic styles: "flat", "weak", "stiff", "solemn", "vigorous", and "mixed". In each figure, five colors are used at the top to represent the distribution of aesthetic style values: pure white for values ​​between 0-20, 15% darker color for 21-40, 25% darker color for 41-60, 35% darker color for 61-80, and 50% darker color for 81-100. The darker the color, the higher the value of the aesthetic style term.

[0085] like Figures 4 to 9 As shown, the results generated by FaceCLIP and EmoStyle are significantly lacking in overall aesthetic appeal and spatial density, resulting in poor structural stability and aesthetic consistency. While MagicFace and AUEditNet improve some aesthetic attributes, inconsistencies between local stroke details and the overall style persist. In contrast, the calligraphy images generated by the method of this invention (CAIG model) are closer to the target image in terms of overall composition, stroke form, and aesthetic details, effectively achieving a synergistic expression of the target aesthetic style in character structure and stroke form.

[0086] comprehensive Figures 4 to 9 The comparison results show that this invention establishes an adaptive mapping relationship between global aesthetic semantics and local writing details through the AG-GLC module, enabling the overall aesthetic goal to continuously constrain the local stroke generation process. Simultaneously, the MSAM module introduces calligraphic aesthetic attributes into a multi-scale feature layer, allowing large-scale features to focus on character structure and composition, while small- and medium-scale features focus on stroke form and local detail expression. These mechanisms enable the method of this invention to generate calligraphic images that meet the target aesthetic requirements in terms of overall style, structural details, and stylistic trends, regardless of the aesthetic style.

[0087] The preferred embodiments of this disclosure have been described in detail above with reference to the accompanying drawings. However, this disclosure is not limited to the specific details of the above embodiments. Within the scope of the technical concept of this disclosure, various simple modifications can be made to the technical solutions of this disclosure, and these simple modifications all fall within the protection scope of this disclosure.

[0088] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, this disclosure will not describe the various possible combinations separately.

[0089] Furthermore, the various implementation methods disclosed in this solution can be combined in any way, as long as they do not violate the spirit of this disclosure, they should also be regarded as the content invented by this disclosure.

Claims

1. A method for constructing a personalized calligraphy image generation model based on calligraphic aesthetics, characterized in that, Includes the following steps: S1, Obtain a calligraphy image dataset, which includes N calligraphy images, each corresponding to an input attribute vector. The input attribute vector It is multi-dimensional, with each dimension corresponding to a different calligraphic aesthetic style; the N calligraphic images include 1 input image and N-1 style images, the input image is used to provide character structure information, and the style images are used to provide style reference information; Obtain a realistic image that matches the glyph structure information provided by the input image and the style reference information provided by the style image; Set with the input attribute vector Target attribute vectors of equal dimensions and attribute percentage parameters ; S2, Construct a personalized calligraphy image generation network. The personalized calligraphy image generation network includes a generation unit and a discriminator. The generation unit includes a feature encoder, a style encoder, a feature adaptive fusion module, an aesthetic injection module, and a decoder. The feature encoder is used to extract features from calligraphy images to obtain global style features and local spatial features; The style encoder is used to extract features from the style image to obtain style features; The input of the feature adaptive fusion module is connected to the output of the feature encoder and the output of the style encoder, respectively. It is used to receive the global style features and local spatial features output by the feature encoder and the style features output by the style encoder. The style features and global style features are concatenated together to generate channel weights to modulate the local spatial features channel by channel, and the resulting enhanced intermediate features are output to the decoder. The input terminal of the aesthetic injection module is connected to both the output terminal of the style encoder and the network input terminal, and is used to receive the style features and input attribute vectors. Target attribute vector and attribute percentage parameters , input attribute vector and target attribute vector By attribute percentage parameter The personalized attribute condition vector is obtained by performing linear weighting. And based on the personalized attribute condition vector Generate attribute guidance features with the style features, and output the attribute guidance features to the decoder; The decoder is used to receive the enhanced intermediate features output by the feature adaptive fusion module and the attribute-guided features output by the aesthetic injection module, perform step-by-step decoding, and output the generated image. The discriminator includes a discrimination branch and an aesthetic feature extraction branch. The discrimination branch is used to distinguish between the generated image output by the decoder and the real image to assist adversarial training. The aesthetic feature extraction branch is used to extract aesthetic feature vectors from the generated image and the real image. S3, combine the calligraphy image dataset, real image, and target attribute vector. and attribute percentage parameters The calligraphy image personalization generation network constructed by input S2 is trained with the overall optimization objective function and the adversarial loss function to obtain the trained generation unit, and the trained generation unit is used as the calligraphy image personalization generation model. The overall optimization objective function is as follows: in, For pixel-level reconstruction constraints, For structure-aware constraints. For aesthetic consistency constraints, , and These represent the weights of the primitive-level reconstruction loss, the structure perception loss, and the aesthetic consistency loss, respectively.

2. The method for constructing a personalized calligraphy image generation model based on calligraphic aesthetics as described in claim 1, characterized in that, The feature adaptive fusion module in S2 generates channel weights by concatenating the style features and global style features, and modulates the local spatial features channel by channel, specifically including: The input image is fed into the feature encoder for feature extraction, resulting in feature map F. Global style features are then obtained from feature map F through spatial aggregation. Mapping the feature map F yields local spatial features. in, This represents a spatial aggregation operator used to preserve overall statistical characteristics; Represents a linear mapping function. Composed of convolution operators, used to characterize local structures; These represent the height, width, and number of channels of a local spatial feature, respectively. The style feature output by the style encoder is denoted as s. The global style feature g and the style feature s are concatenated to generate the channel weights. ; in, and It is a linear mapping function. This represents a non-linear activation operation. This is the Sigmoid activation function, used to constrain the stability of the weight distribution; The local spatial features are modulated channel by channel according to the channel weight 'a' to obtain the modulated local features. ; Where h, w, and c represent the height index, width index, and channel index of the enhanced intermediate features, respectively. This represents the weight value of the c-th channel; Modulated local features The enhanced intermediate features are obtained by back-injecting the feature integration mapping function into the backbone feature stream, using F as the backbone feature. ; in, This represents the feature integration mapping function, used to ensure the consistency of the modulated features with the backbone features in terms of dimension and distribution.

3. The method for constructing a personalized calligraphy image generation model based on calligraphic aesthetics as described in claim 1, characterized in that, The aesthetic injection module described in S2 will input attribute vectors. and target attribute vector By attribute percentage parameter The personalized attribute condition vector is obtained by performing linear weighting. ; The personalized attribute condition vector The style features output by the style encoder are concatenated to form an aesthetic embedding vector; Obtain multiple sets of features at different spatial scales output by the intermediate layers of the decoder. ;in, Represents the total number of spatial scales. Indicates the first Characteristics of each spatial scale; Spatial aggregation operations are performed on features at each spatial scale to obtain a joint representation. ; Through learnable mapping functions Jointly expressed Mapped to a set of spatially scale-dependent modulation weights ;in, This represents the modulation weight at the s-th spatial scale; Based on the modulation weights, aesthetic attribute-driven recalibration is performed on features at each scale to obtain... , ; where ⊙ represents element-wise multiplication; right Integration is achieved through a multi-scale fusion mapping function, outputting the multi-scale modulated feature representation. And use it as the attribute guiding feature; wherein This represents the multi-scale fusion mapping function.

4. The method for constructing a personalized calligraphy image generation model based on calligraphic aesthetics as described in any one of claims 1 to 3, characterized in that, In S3, the personalized calligraphy image generation network is trained, specifically including the following steps: S31, combine the input image and style image, the real image, and the target attribute vector from the calligraphy image dataset. and the attribute percentage parameter Input the personalized calligraphy image generation network; S32, input the input image into the feature encoder for feature extraction to obtain global style features and local spatial features; The style image is input into the style encoder for feature extraction to obtain style features; The real image is input into the discriminator as a supervisory signal; S33, the feature adaptive fusion module generates channel weights based on the splicing result of the global style features and the style features, and modulates the local spatial features channel by channel to obtain enhanced intermediate features; S34, the aesthetic injection module, based on the input attribute vector The target attribute vector and the attribute percentage parameter Calculate the personalized attribute condition vector And based on personalized attribute condition vectors Generate attribute-guided features with the style features; S35, the decoder decodes the enhanced intermediate features and the attribute-guided features step by step and outputs the generated image; S36, calculate the loss value between the generated image and the real image using the overall optimization objective function and the adversarial loss function, and update the network parameters of the generator and the discriminator according to the loss value through the backpropagation algorithm until the overall optimization objective function converges, thus obtaining the trained generator.

5. The method for constructing a personalized calligraphy image generation model based on calligraphy aesthetics as described in claim 4, characterized in that, The attribute percentage parameter During training, values ​​are randomly sampled. In each iteration, a value is randomly sampled from the interval [0,1] as the attribute percentage parameter for the current iteration. .

6. The method for constructing a personalized calligraphy image generation model based on calligraphy aesthetics as described in claim 5, characterized in that, The pixel-level reconstruction constraint is: ;in, Represents a true calligraphy image. This represents the generated image output by the decoder. Represents the L1 norm; The structure-aware constraint is: ;in, This represents a feature extraction network used to extract representations of mid-to-high-level structures. The aesthetic consistency constraint: ;in, The aesthetic feature extraction branch pairs of the discriminator generate images. Extracted aesthetic feature vectors This represents the square of the L2 norm.

7. A method for personalized generation of calligraphy images based on calligraphic aesthetics, characterized in that, Includes the following steps: Step 1: Obtain the input image and the user-specified target attribute vector. The target attribute vector It is multi-dimensional, with each dimension corresponding to a different style of calligraphic aesthetics; Step two, combine the input image with the user-specified target attribute vector. Input the personalized calligraphy image generation model constructed by the model construction method as described in any one of claims 1 to 6, and specify the attribute proportion parameter. ; Step 3: Output the generated image.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the personalized calligraphy image generation method based on calligraphy aesthetics as described in claim 7.

9. A computer program product, characterized in that, It includes a computer program / instruction, which, when executed by a processor, implements the personalized calligraphy image generation method based on calligraphy aesthetics as described in claim 7.

Images