A method and system for generating digital paintings in Jingchu style based on artificial intelligence

Abstract

The present invention discloses a method and system for generating digital paintings in the Jingchu style based on artificial intelligence, and the method includes Jingchu painting acquisition, image preprocessing, painting core area extraction, stylized painting generation and Jingchu style digital painting generation. The present invention relates to the field of digital painting generation technology, and specifically refers to a method and system for generating digital paintings in the Jingchu style based on artificial intelligence. This scheme adopts an improved lightweight U-shaped network to extract the core area of ​​the painting, and optimizes and improves the visual state module by calling the Mamba residual module, thereby improving the effect of core area segmentation and the data quality of subsequent painting synthesis; a convolutional neural network optimized by a style transfer network is used to generate stylized paintings, and a stylized loss function is optimized and designed by combining the content of the previous core area extraction, and then adversarial training is performed to improve the particularity of Jingchu style digital paintings and the quality of paintings.

Description

Technical Field

[0001] The present invention relates to the technical field of digital painting generation, and in particular to a method and system for generating digital paintings in a Jingchu style based on artificial intelligence. Background Art

[0002] The Jingchu style digital painting generation method based on artificial intelligence is a technology and corresponding software system that uses modern artificial intelligence technology to generate digital paintings with the Jingchu cultural style. The Jingchu style originated from the ancient Chinese Chu culture and has a unique artistic style. It usually includes landscapes, figures, animals and architecture, and uses specific colors, lines and compositions to reflect traditional cultural characteristics.

[0003] However, in the existing methods for generating digital paintings in the Jingchu style, there is a technical problem that the synthesis of paintings in the Jingchu style depends on the extraction of core elements of the style and the identification of regions, while the traditional methods of direct stylization transfer or segmentation with fixed algorithm rules may lose the key elements of the Jingchu style; in the existing methods for generating digital paintings in the Jingchu style, there is a technical problem that the generation of digital paintings in the specific painting style of the Jingchu style cannot effectively reflect the main characteristics of the Jingchu style by using simple and direct adversarial generative network training, and the traditional methods of digital painting synthesis lack the unique improvements and optimizations to adapt to the Jingchu style paintings. Summary of the invention

[0004] In view of the above situation, in order to overcome the defects of the prior art, the present invention provides a method and system for generating digital paintings in the style of Jingchu based on artificial intelligence. In view of the technical problem that the synthesis of paintings in the style of Jingchu depends on the extraction of the core elements of the style and the identification of the regions in the existing digital painting generation methods in the style of Jingchu, and the traditional methods perform direct stylization migration or segmentation based on fixed algorithm rules, which may lose the key elements of the style of Jingchu. This solution creatively adopts an improved lightweight U-shaped network to extract the core region of the painting, and optimizes and improves the visual state module by calling the Mamba residual module, thereby improving the effect of the core region segmentation and the subsequent painting. The data quality of Jingchu style painting synthesis is improved; in the existing Jingchu style digital painting generation methods, there are digital paintings that are generated in the specific painting style of Jingchu style. The simple and direct adversarial generative network training cannot effectively reflect the main characteristics of Jingchu style, and the traditional method of digital painting synthesis lacks the unique improvement and optimization technical problem that adapts to Jingchu style painting. This scheme creatively uses the convolutional neural network optimized by the style transfer network to generate stylized paintings. By combining the content extracted from the previous core area, the stylized loss function is optimized and designed, and then adversarial training is performed to improve the particularity of Jingchu style digital paintings and the quality of paintings.

[0005] The technical solution adopted by the present invention is as follows: The present invention provides a method for generating digital paintings in the Jingchu style based on artificial intelligence, and the method comprises the following steps:

[0006] Step S1: Collection of Jingchu paintings;

[0007] Step S2: image preprocessing;

[0008] Step S3: Extracting the core area of ​​the painting;

[0009] Step S4: stylized painting generation;

[0010] Step S5: Generate Jingchu style digital painting.

[0011] Further, in step S1, the Jingchu painting collection is used to collect the original image reference data set required to generate the Jingchu style digital painting, specifically, to obtain the original data of the Jingchu style digital painting from the Jingchu style artwork through data collection and basic data annotation;

[0012] The original data of Jingchu style digital paintings specifically include traditional painting data, mural data and decorative artwork data in Jingchu style;

[0013] The basic data annotation is specifically to manually extract and annotate the artistic style of the texture, color and theme of the original image.

[0014] Further, in step S2, the image preprocessing is used to optimize the original image, specifically, data enhancement and image preprocessing operations are performed on the image data in the original data of the Jingchu style digital painting to obtain optimized digital painting data;

[0015] The data enhancement specifically includes data set expansion and image attribute enhancement. The data set expansion specifically refers to an oversampling operation for balancing the labeled categories in the data set. The image data enhancement specifically refers to normalizing and unifying the contrast, color saturation and brightness of the image.

[0016] The image preprocessing specifically includes size normalization and denoising operations. The size normalization specifically refers to adjusting the size of the image to a uniform specification. The denoising specifically refers to removing noise from the image using a Gaussian filtering algorithm to optimize the basic quality of the image.

[0017] Further, in step S3, the painting core area extraction is used to detect and extract the core painting content of the optimized image, specifically, based on the optimized digital painting data, an improved lightweight U-shaped network is used to extract the painting core area to obtain the Jingchu style core painting area data;

[0018] The improved lightweight U-shaped network specifically includes a deep convolution extraction layer, an improved U-shaped encoding side, a bottleneck sub-block, a U-shaped decoding side and a feature mapping output layer;

[0019] The deep convolution extraction layer is used to preliminarily extract shallow features;

[0020] The improved U-type coding side is used to call the Mamba module in combination with the visual residual module to improve the U-type coding side, perform feature coding, and reduce the resolution of the feature image through a maximum pooling operation;

[0021] The bottleneck sub-block is used to model the spatial long-term dependency relationship within the image;

[0022] The U-type decoding side is used to perform feature decoding to restore image resolution;

[0023] The feature mapping output layer is used to construct a fully connected layer for feature mapping, and to generate a segmented image using a classifier and an activation function;

[0024] The step of using an improved lightweight U-shaped network to extract the core area of ​​the painting and obtain the core painting area data of the Jingchu style includes:

[0025] Step S31: construct a deep convolution extraction layer, specifically, by constructing a deep convolution layer, receiving the optimized digital painting data as an input data sample of the model, and performing shallow feature extraction on the data input to obtain shallow feature data, and the calculation formula is:

[0026] ;

[0027] In the formula, I is the input data sample of the model, F S is shallow feature data, R is the value range corresponding to the optimized digital painting data, where C is the number of channels of image data in the optimized digital painting data, H is the height of the image data, W is the width of the image data, D is the number of slices of the image data, and the value of 32 is used to represent a fixed number of feature filters. is a feature extraction operator;

[0028] Step S32: constructing an improved U-type coding side, specifically constructing a three-layer encoder sub-block, and calling the Mamba module in combination with the visual residual module to improve and optimize the U-type coding side to obtain deep feature data, including the following steps:

[0029] Step S321: Preliminary feature encoding, specifically constructing a layer of encoder sub-blocks, from the shallow feature data F S Extract the preliminary depth feature data;

[0030] Step S322: construct a visual state space module, specifically construct a dual-line branch for feature expansion and activation, wherein the first branch specifically uses a linear layer to expand the feature channel, and the second branch uses a SiLU activation function to activate, to obtain aggregated depth feature data, the calculation formula is:

[0031] ;

[0032] Where W OUT is the aggregate feature data, VSSM(·) is the visual state space module function, W1 is the first branch output feature, W2 is the second score output feature, is the Hadamard product operator, LayerNorm(·) is the layer normalization function, SSM(·) is the state-space module function, SiLU(·) is the SiLU activation function, DWConv(·) is the depthwise convolution operation, Linear(·) is the linear layer function, W in is the input feature received by the visual state space module;

[0033] Step S323: state space module enhancement, specifically calling the Mamba module, combining the visual residual module, and constructing a residual visual Mamba layer, the residual visual Mamba layer specifically includes a normalization layer, a compression layer and a visual state space module, the visual state space module specifically includes a linear layer, a deep convolution layer, a state module and a SiLU activation function, and by constructing the residual visual Mamba layer, the depth feature enhancement of the original state space module is performed, and the calculation formula is:

[0034] ;

[0035] In the formula, is the visual module output feature, VSSM(·) is the visual state space module function, LayerNorm(·) is the layer normalization function, is the preliminary depth feature data, s is the adjustment factor;

[0036] Step S324: Feature depth projection, which is used to construct a projection layer and perform deep feature conversion on the output features of the visual module. The calculation formula is:

[0037] ;

[0038] In the formula, is the deep output feature, Pro(·) is the projection layer function, LayerNorm(·) is the layer normalization function, is the output feature of the visual module;

[0039] Step S325: depth feature coding, specifically, using the visual state space module enhanced by the state space module as the second coding layer, and constructing the third coding layer, extracting the depth feature data from the depth output feature, and the calculation formula is:

[0040] ;

[0041] In the formula, is the depth feature output corresponding to the lth encoding layer, the specific value range of l is {1,2,3}, R is the value domain corresponding to the optimized digital painting data, H is the height of the image data, W is the width of the image data, D is the number of slices of the image data, and F S It is shallow feature data;

[0042] Step S33: constructing a bottleneck sub-block, specifically, constructing a bottleneck sub-block after the third-layer encoder sub-block, simulating the long-range spatial dependency, obtaining the bottleneck sub-block output feature, and connecting the bottleneck sub-block output feature to the U-type decoding side;

[0043] Step S34: constructing a U-shaped decoding side, specifically constructing a three-layer decoder to use addition operation to perform feature fusion, and then perform feature decoding to obtain decoding feature data. The calculation formula is:

[0044] ;

[0045] Where P OUT is the decoded feature data, ReLU(·) is the nonlinear activation function, DWConv(·) is the deep convolution operation, P in is the bottleneck sub-block output feature, is the deep feature output, is the decoding adjustment factor;

[0046] Step S35: constructing a feature mapping output layer, specifically by constructing a fully connected layer, performing feature mapping of the decoded feature data, and constructing a Softmax classifier for feature classification and detection, to obtain detection classification data information of the painting core area;

[0047] Step S36: core extraction model training, specifically, by constructing the deep convolution extraction layer, constructing the improved U-shaped encoding side, constructing the bottleneck sub-block, constructing the U-shaped decoding side and constructing the feature map output layer, the core extraction model training is performed to obtain the painting core area extraction model Model MU ;

[0048] Step S37: Extracting the core area of ​​the painting, specifically, using the painting core area extraction model Model according to the optimized digital painting data MU, extract the core area of ​​the painting and obtain the core painting area data of the Jingchu style.

[0049] Further, in step S4, the stylized painting generation is used to implement the painting style transfer of the neural network and generate a stylized digital painting, specifically, based on the Jingchu style core painting area data and the optimized digital painting data, a convolutional neural network optimized by the style transfer network is used to generate the stylized painting to obtain the Jingchu stylized painting data;

[0050] The convolutional neural network optimized by the style transfer network specifically includes a basic convolutional generation subnet, a style transfer optimization loss function group and an edge detection synthesis operator;

[0051] The basic convolutional generation subnetwork is used to construct a basic network for generating digital paintings;

[0052] The style transfer optimization loss function group is used to optimize the transfer of the painting style of Jingchu style;

[0053] The edge detection synthesis operator is used to optimize the edge effect between image elements in the digital painting synthesis process;

[0054] The step of using a convolutional neural network optimized by a style transfer network to generate stylized paintings and obtain Jingchu stylized painting data includes:

[0055] Step S41: construct a basic convolutional subnet, specifically, construct a basic convolutional subnet with the fully connected layer removed, including an input layer, a convolutional layer, a pooling layer, and an output layer, and optimize features by average pooling;

[0056] Step S42: constructing a style transfer optimization loss function group, specifically, taking the optimized digital painting data as the basic content image, taking the Jingchu style core painting area data as the style pattern image, constructing a style transfer optimization loss function group, and performing stylized painting model training optimization by constructing the style transfer optimization loss function group, including the following steps:

[0057] Step S421: construct a stylized content loss function, the calculation formula is:

[0058] ;

[0059] Where, L Sc is the stylized content loss function, N C is the total number of feature maps corresponding to the basic content image, M C is the product of the height and width of the corresponding basic content image, X C is the basic content image, i is the horizontal index of the feature map pixel, j is the vertical index of the feature map pixel, and FC (·) is the filter activation function, is to generate an image;

[0060] Step S422: construct a stylized feature space loss function, the calculation formula is:

[0061] ;

[0062] Where, L Ss is the stylized feature space loss function, L is the convolution layer index, N S is the total number of feature maps corresponding to the style image, w L is the stylized feature space weight of the Lth layer, i is the horizontal index of the feature map pixel, j is the vertical index of the feature map pixel, G(·) is the Grammar matrix, X S is the style image, is to generate an image;

[0063] Step S423: construct a stylized integration loss function, the calculation formula is:

[0064] ;

[0065] Where, L TOTAL is the stylized integration loss function, is the stylized content loss weight, L Sc is the stylized content loss function, is the stylized feature space loss weight, L Ss is the stylized feature space loss function;

[0066] Step S43: constructing an edge detection synthesis operator, specifically using a Sobel operator to perform edge detection, and optimizing drawing synthesis through edge detection;

[0067] Step S44: stylized painting model training, specifically, by constructing the basic convolutional generation subnet, constructing the style transfer optimization loss function group and constructing the edge detection synthesis operator, using the adversarial training method to train the stylized painting model to obtain the stylized painting model Model SP ;

[0068] Step S45: generating a stylized painting, specifically using the stylized painting model Model SP , Jingchu stylized painting data is generated.

[0069] Furthermore, in step S5, the Jingchu style digital painting is generated to generate a specific Jingchu style painting work, specifically by obtaining the Jingchu stylized painting data, creating a digital painting that conforms to the Jingchu style, and generating a Jingchu style digital painting work image.

[0070] The present invention provides a Jingchu style digital painting generation system based on artificial intelligence, comprising a Jingchu painting acquisition module, an image preprocessing module, a painting core area extraction module, a stylized painting generation module and a Jingchu style digital painting generation module;

[0071] The Jingchu painting collection module is used for collecting Jingchu paintings, obtaining Jingchu style digital painting original data through Jingchu painting collection, and sending the Jingchu style digital painting original data to the image preprocessing module;

[0072] The image preprocessing module is used for image preprocessing, and obtains optimized digital painting data through image preprocessing, and sends the optimized digital painting data to the painting core area extraction module and the stylized painting generation module;

[0073] The painting core area extraction module is used for painting core area extraction, and obtains Jingchu style core painting area data through painting core area extraction, and sends the Jingchu style core painting area data to the stylized painting generation module;

[0074] The stylized painting generation module is used for stylized painting generation, obtains Jingchu stylized painting data through stylized painting generation, and sends the Jingchu stylized painting data to the Jingchu style digital painting generation module;

[0075] The Jingchu style digital painting generation module is used for Jingchu style digital painting generation, and obtains a Jingchu style digital painting work image through Jingchu style digital painting generation.

[0076] The beneficial effects achieved by the present invention using the above scheme are as follows:

[0077] (1) In the existing Jingchu style digital painting generation methods, there is a technical problem that the synthesis of Jingchu style paintings depends on the extraction of style core elements and the identification of regions. The traditional methods perform direct stylization transfer or segmentation with fixed algorithm rules, which may lose the key Jingchu style elements. This scheme creatively adopts an improved lightweight U-shaped network to extract the core area of ​​the painting, and optimizes the visual state module by calling the Mamba residual module, which improves the effect of core area segmentation and the data quality of subsequent painting synthesis.

[0078] (2) In the existing methods for generating digital paintings in the Jingchu style, there are digital paintings that are generated in a specific Jingchu style. The simple and direct adversarial generative network training cannot effectively reflect the main characteristics of the Jingchu style, and the traditional method of digital painting synthesis lacks the technical problem of unique improvements and optimizations that adapt to the Jingchu style paintings. This scheme creatively uses a convolutional neural network optimized by a style transfer network to generate stylized paintings. By combining the content extracted from the previous core area, the stylized loss function is optimized and designed, and then adversarial training is performed to improve the specificity of Jingchu style digital paintings and the quality of paintings. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] Figure 1 A schematic diagram of a process of generating a Jingchu style digital painting based on artificial intelligence provided by the present invention;

[0080] Figure 2 A schematic diagram of a Jingchu style digital painting generation system based on artificial intelligence provided by the present invention;

[0081] Figure 3 A schematic diagram of the process of drawing the core area extraction in step S3;

[0082] Figure 4 A schematic diagram of a process for constructing an improved U-type coding side for step S32;

[0083] Figure 5 Schematic diagram of the process of stylized painting generation in step S4.

[0084] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention. DETAILED DESCRIPTION

[0085] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0086] In the description of the present invention, it should be understood that terms such as “upper”, “lower”, “front”, “back”, “left”, “right”, “top”, “bottom”, “inside” and “outside” indicating directions or positional relationships are based on the directions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore should not be understood as limiting the present invention.

[0087] Example 1, see Figure 1 The present invention provides a method for generating digital paintings in Jingchu style based on artificial intelligence, the method comprising the following steps:

[0088] Step S1: Collection of Jingchu paintings;

[0089] Step S2: image preprocessing;

[0090] Step S3: Extracting the core area of ​​the painting;

[0091] Step S4: stylized painting generation;

[0092] Step S5: Generate Jingchu style digital painting.

[0093] Example 2, see Figure 1 and Figure 2 In step S1, the Jingchu painting collection is used to collect the original image reference data set required to generate Jingchu style digital paintings, specifically, to obtain the original data of Jingchu style digital paintings from Jingchu style artworks through data collection and basic data annotation;

[0094] The original data of Jingchu style digital paintings specifically include traditional painting data, mural data and decorative artwork data in Jingchu style;

[0095] The basic data annotation is specifically to manually extract and annotate the artistic style of the texture, color and theme of the original image.

[0096] Example 3, see Figure 1 and Figure 2 This embodiment is based on the above embodiment. In step S2, the image preprocessing is used to optimize the original image, specifically, data enhancement and image preprocessing operations are performed on the image data in the original data of the Jingchu style digital painting to obtain optimized digital painting data;

[0097] The data enhancement specifically includes data set expansion and image attribute enhancement. The data set expansion specifically refers to an oversampling operation for balancing the labeled categories in the data set. The image data enhancement specifically refers to normalizing and unifying the contrast, color saturation and brightness of the image.

[0098] The image preprocessing specifically includes size normalization and denoising operations. The size normalization specifically refers to adjusting the size of the image to a uniform specification. The denoising specifically refers to removing noise from the image using a Gaussian filtering algorithm to optimize the basic quality of the image.

[0099] Example 4, see Figure 1 , Figure 2, Figure 3 and Figure 4 This embodiment is based on the above embodiment. In step S3, the painting core area extraction is used to detect and extract the core painting content of the optimized image. Specifically, based on the optimized digital painting data, an improved lightweight U-shaped network is used to extract the painting core area to obtain the Jingchu style core painting area data.

[0100] The improved lightweight U-shaped network specifically includes a deep convolution extraction layer, an improved U-shaped encoding side, a bottleneck sub-block, a U-shaped decoding side and a feature mapping output layer;

[0101] The deep convolution extraction layer is used to preliminarily extract shallow features;

[0102] The improved U-type coding side is used to call the Mamba module in combination with the visual residual module to improve the U-type coding side, perform feature coding, and reduce the resolution of the feature image through a maximum pooling operation;

[0103] The bottleneck sub-block is used to model the spatial long-term dependency relationship within the image;

[0104] The U-type decoding side is used to perform feature decoding to restore image resolution;

[0105] The feature mapping output layer is used to construct a fully connected layer for feature mapping, and to generate a segmented image using a classifier and an activation function;

[0106] The step of using an improved lightweight U-shaped network to extract the core area of ​​the painting and obtain the core painting area data of the Jingchu style includes:

[0107] Step S31: construct a deep convolution extraction layer, specifically, by constructing a deep convolution layer, receiving the optimized digital painting data as an input data sample of the model, and performing shallow feature extraction on the data input to obtain shallow feature data, and the calculation formula is:

[0108] ;

[0109] In the formula, I is the input data sample of the model, F S is shallow feature data, R is the value range corresponding to the optimized digital painting data, where C is the number of channels of image data in the optimized digital painting data, H is the height of the image data, W is the width of the image data, D is the number of slices of the image data, and the value of 32 is used to represent a fixed number of feature filters. is a feature extraction operator;

[0110] Step S32: constructing an improved U-type coding side, specifically constructing a three-layer encoder sub-block, and calling the Mamba module in combination with the visual residual module to improve and optimize the U-type coding side to obtain deep feature data, including the following steps:

[0111] Step S321: Preliminary feature encoding, specifically constructing a layer of encoder sub-blocks, from the shallow feature data F S Extract the preliminary depth feature data;

[0112] Step S322: construct a visual state space module, specifically construct a dual-line branch for feature expansion and activation, wherein the first branch specifically uses a linear layer to expand the feature channel, and the second branch uses a SiLU activation function to activate, to obtain aggregated depth feature data, the calculation formula is:

[0113] ;

[0114] Where W OUT is the aggregate feature data, VSSM(·) is the visual state space module function, W1 is the first branch output feature, W2 is the second score output feature, is the Hadamard product operator, LayerNorm(·) is the layer normalization function, SSM(·) is the state-space module function, SiLU(·) is the SiLU activation function, DWConv(·) is the depthwise convolution operation, Linear(·) is the linear layer function, W in is the input feature received by the visual state space module;

[0115] Step S323: state space module enhancement, specifically calling the Mamba module, combining the visual residual module, and constructing a residual visual Mamba layer, the residual visual Mamba layer specifically includes a normalization layer, a compression layer and a visual state space module, the visual state space module specifically includes a linear layer, a deep convolution layer, a state module and a SiLU activation function, and by constructing the residual visual Mamba layer, the depth feature enhancement of the original state space module is performed, and the calculation formula is:

[0116] ;

[0117] In the formula, is the visual module output feature, VSSM(·) is the visual state space module function, LayerNorm(·) is the layer normalization function, is the preliminary depth feature data, s is the adjustment factor;

[0118] Step S324: Feature depth projection, which is used to construct a projection layer and perform deep feature conversion on the output features of the visual module. The calculation formula is:

[0119] ;

[0120] In the formula, is the deep output feature, Pro(·) is the projection layer function, LayerNorm(·) is the layer normalization function, is the output feature of the visual module;

[0121] Step S325: depth feature coding, specifically, using the visual state space module enhanced by the state space module as the second coding layer, and constructing the third coding layer, extracting the depth feature data from the depth output feature, and the calculation formula is:

[0122] ;

[0123] In the formula, is the depth feature output corresponding to the lth encoding layer, the specific value range of l is {1,2,3}, R is the value domain corresponding to the optimized digital painting data, H is the height of the image data, W is the width of the image data, D is the number of slices of the image data, and F S It is shallow feature data;

[0124] Step S33: constructing a bottleneck sub-block, specifically, constructing a bottleneck sub-block after the third-layer encoder sub-block, simulating the long-range spatial dependency, obtaining the bottleneck sub-block output feature, and connecting the bottleneck sub-block output feature to the U-type decoding side;

[0125] Step S34: constructing a U-shaped decoding side, specifically constructing a three-layer decoder to use addition operation to perform feature fusion, and then perform feature decoding to obtain decoding feature data. The calculation formula is:

[0126] ;

[0127] Where P OUT is the decoded feature data, ReLU(·) is the nonlinear activation function, DWConv(·) is the deep convolution operation, P in is the bottleneck sub-block output feature, is the deep feature output, is the decoding adjustment factor;

[0128] Step S35: constructing a feature mapping output layer, specifically by constructing a fully connected layer, performing feature mapping of the decoded feature data, and constructing a Softmax classifier for feature classification and detection, to obtain detection classification data information of the painting core area;

[0129] Step S36: core extraction model training, specifically, by constructing the deep convolution extraction layer, constructing the improved U-shaped encoding side, constructing the bottleneck sub-block, constructing the U-shaped decoding side and constructing the feature map output layer, the core extraction model training is performed to obtain the painting core area extraction model Model MU ;

[0130] Step S37: Extracting the core area of ​​the painting, specifically, using the painting core area extraction model Model according to the optimized digital painting data MU , extract the core area of ​​the painting and obtain the core painting area data of the Jingchu style.

[0131] By performing the above operations, in view of the technical problem that in the existing Jingchu style digital painting generation method, the synthesis of Jingchu style paintings depends on the extraction of style core elements and the identification of regions, while the traditional method of direct stylization migration or segmentation with fixed algorithm rules may lose the key Jingchu style elements, this scheme creatively adopts an improved lightweight U-shaped network to extract the core area of ​​the painting, and optimizes and improves the visual state module by calling the Mamba residual module, thereby improving the effect of core area segmentation and the data quality of subsequent painting synthesis.

[0132] Example 5, see Figure 1 , Figure 2 and Figure 5 This embodiment is based on the above embodiment. In step S4, the stylized painting generation is used to implement the painting style transfer of the neural network and generate a stylized digital painting. Specifically, based on the Jingchu style core painting area data and the optimized digital painting data, a convolutional neural network optimized by a style transfer network is used to generate a stylized painting to obtain Jingchu stylized painting data.

[0133] The convolutional neural network optimized by the style transfer network specifically includes a basic convolutional generation subnet, a style transfer optimization loss function group and an edge detection synthesis operator;

[0134] The basic convolutional generation subnetwork is used to construct a basic network for generating digital paintings;

[0135] The style transfer optimization loss function group is used to optimize the transfer of the painting style of Jingchu style;

[0136] The edge detection synthesis operator is used to optimize the edge effect between image elements in the digital painting synthesis process;

[0137] The step of using a convolutional neural network optimized by a style transfer network to generate stylized paintings and obtain Jingchu stylized painting data includes:

[0138] Step S41: construct a basic convolutional subnet, specifically, construct a basic convolutional subnet with the fully connected layer removed, including an input layer, a convolutional layer, a pooling layer, and an output layer, and optimize features by average pooling;

[0139] The basic convolutional generation subnet specifically uses the VGG-19 model for image feature representation;

[0140] Step S42: constructing a style transfer optimization loss function group, specifically, taking the optimized digital painting data as the basic content image, taking the Jingchu style core painting area data as the style pattern image, constructing a style transfer optimization loss function group, and performing stylized painting model training optimization by constructing the style transfer optimization loss function group, including the following steps:

[0141] Step S421: construct a stylized content loss function, the calculation formula is:

[0142] ;

[0143] Where, L Sc is the stylized content loss function, N C is the total number of feature maps corresponding to the basic content image, M C is the product of the height and width of the corresponding basic content image, X C is the basic content image, i is the horizontal index of the feature map pixel, j is the vertical index of the feature map pixel, and F C (·) is the filter activation function, is to generate an image;

[0144] Step S422: construct a stylized feature space loss function, the calculation formula is:

[0145] ;

[0146] Where, L Ss is the stylized feature space loss function, L is the convolution layer index, N S is the total number of feature maps corresponding to the style image, w L is the stylized feature space weight of the Lth layer, i is the horizontal index of the feature map pixel, j is the vertical index of the feature map pixel, G(·) is the Grammar matrix, X S is the style image, is to generate an image;

[0147] Step S423: construct a stylized integration loss function, the calculation formula is:

[0148] ;

[0149] Where, LTOTAL is the stylized integration loss function, is the stylized content loss weight, L Sc is the stylized content loss function, is the stylized feature space loss weight, L Ss is the stylized feature space loss function;

[0150] Step S43: constructing an edge detection synthesis operator, specifically using a Sobel operator to perform edge detection, and optimizing drawing synthesis through edge detection;

[0151] Step S44: stylized painting model training, specifically, by constructing the basic convolutional generation subnet, constructing the style transfer optimization loss function group and constructing the edge detection synthesis operator, using the adversarial training method to train the stylized painting model to obtain the stylized painting model Model SP ;

[0152] Step S45: generating a stylized painting, specifically using the stylized painting model Model SP , Jingchu stylized painting data is generated.

[0153] By performing the above operations, in the existing Jingchu style digital painting generation methods, there is a problem that the digital painting works are generated for the specific painting style of Jingchu style. The simple and direct adversarial generative network training cannot effectively reflect the main characteristics of the Jingchu style, and the traditional method of digital painting synthesis lacks the unique improvement and optimization technical problem that adapts to the Jingchu style painting. This scheme creatively uses the convolutional neural network optimized by the style transfer network to generate stylized paintings. By combining the content extracted from the previous core area, the stylized loss function is optimized and designed, and then adversarial training is performed to improve the particularity of Jingchu style digital paintings and the quality of paintings.

[0154] Example 6, see Figure 1 and Figure 2 This embodiment is based on the above embodiment. In step S5, the Jingchu style digital painting is generated to generate a specific Jingchu style painting work. Specifically, by obtaining the Jingchu stylized painting data, a digital painting that conforms to the Jingchu style is created to generate a Jingchu style digital painting work.

[0155] Embodiment 7, see Figure 1 and Figure 2 , this embodiment is based on the above embodiment, and the present invention provides a Jingchu style digital painting generation system based on artificial intelligence, including a Jingchu painting acquisition module, an image preprocessing module, a painting core area extraction module, a stylized painting generation module and a Jingchu style digital painting generation module;

[0156] The Jingchu painting collection module is used for collecting Jingchu paintings, obtaining Jingchu style digital painting original data through Jingchu painting collection, and sending the Jingchu style digital painting original data to the image preprocessing module;

[0157] The image preprocessing module is used for image preprocessing, and obtains optimized digital painting data through image preprocessing, and sends the optimized digital painting data to the painting core area extraction module and the stylized painting generation module;

[0158] The painting core area extraction module is used for painting core area extraction, and obtains Jingchu style core painting area data through painting core area extraction, and sends the Jingchu style core painting area data to the stylized painting generation module;

[0159] The stylized painting generation module is used for stylized painting generation, obtains Jingchu stylized painting data through stylized painting generation, and sends the Jingchu stylized painting data to the Jingchu style digital painting generation module;

[0160] The Jingchu style digital painting generation module is used for Jingchu style digital painting generation, and obtains a Jingchu style digital painting work image through Jingchu style digital painting generation.

[0161] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0162] While the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that many changes, modifications, substitutions and variations can be made to the embodiments without departing from the principles and spirit of the invention.

[0163] The present invention and its embodiments are described above, and such description is not restrictive. The drawings show only one embodiment of the present invention, and the actual structure is not limited thereto. In short, if ordinary technicians in the field are inspired by it, without departing from the purpose of the invention, they can design a structure and embodiment similar to the technical solution without creativity, which should belong to the protection scope of the present invention.

Claims

1. A method for generating digital paintings in Jingchu style based on artificial intelligence, characterized by: The method comprises the following steps: Step S1: Collect Jingchu paintings to obtain original data of Jingchu style digital paintings; Step S2: image preprocessing to obtain optimized digital painting data; Step S3: Extraction of the core painting area, used to detect and extract the core painting content of the optimized image, specifically, based on the optimized digital painting data, an improved lightweight U-shaped network is used to extract the core painting area to obtain the core painting area data of the Jingchu style, including the following steps: Step S31: constructing a deep convolution extraction layer; Step S32: constructing an improved U-shaped encoding side; Step S33: constructing a bottleneck sub-block; Step S34: constructing a U-shaped decoding side; Step S35: constructing a feature map output layer; Step S36: core extraction model training; Step S37: drawing core area extraction; In step S32, the construction of the improved U-type coding side includes the following steps: Step S321: Preliminary feature encoding, specifically constructing a layer of encoder sub-blocks, from the shallow feature data F S Extract the preliminary depth feature data; Step S322: construct a visual state space module, specifically construct a dual-line branch for feature expansion and activation, wherein the first branch specifically uses a linear layer to expand the feature channel, and the second branch uses a SiLU activation function for activation to obtain aggregated depth feature data, and the calculation formula is: Where W OUT is the aggregate feature data, VSSM(·) is the visual state space module function, W1 is the first branch output feature, W2 is the second branch output feature, ⊙ is the Hadamard product operator, LayerNorm(·) is the layer normalization function, SSM(·) is the state space module function, SiLU(·) is the SiLU activation function, DWConv(·) is the depth convolution operation, Linear(·) is the linear layer function, W in is the input feature received by the visual state space module; Step S323: state space module enhancement, specifically calling the Mamba module, combining the visual residual module, and constructing a residual visual Mamba layer, the residual visual Mamba layer specifically includes a normalization layer, a compression layer and a visual state space module, the visual state space module specifically includes a linear layer, a deep convolution layer, a state module and a SiLU activation function, and by constructing the residual visual Mamba layer, the depth feature enhancement of the original state space module is performed, and the calculation formula is: In the formula, is the visual module output feature, VSSM(·) is the visual state space module function, LayerNorm(·) is the layer normalization function, is the preliminary depth feature data, s is the adjustment factor; Step S324: Feature depth projection, which is used to construct a projection layer and perform deep feature conversion on the output features of the visual module. The calculation formula is: In the formula, is the deep output feature, Pro(·) is the projection layer function, LayerNorm(·) is the layer normalization function, is the output feature of the visual module; Step S325: depth feature coding, specifically, using the visual state space module enhanced by the state space module as the second coding layer, and constructing the third coding layer, extracting the depth feature data from the depth output feature, and the calculation formula is: In the formula, is the depth feature output corresponding to the lth encoding layer, the specific value range of l is {1,2,3}, R is the value domain corresponding to the optimized digital painting data, H is the height of the image data, W is the width of the image data, D is the number of slices of the image data, and F S It is shallow feature data; Step S4: stylized painting generation, for implementing the painting style transfer of the neural network and generating stylized digital paintings, specifically, based on the Jingchu style core painting area data and the optimized digital painting data, using the convolutional neural network optimized by the style transfer network to generate stylized paintings, and obtaining Jingchu stylized painting data, specifically including the following steps: Step S41: construct a basic convolutional generation subnet; Step S42: construct a style transfer optimization loss function group; Step S43: construct an edge detection synthesis operator; Step S44: stylized painting model training; Step S45: stylized painting generation; Step S5: Generate a Jingchu style digital painting to obtain a Jingchu style digital painting work.

2. The method for generating digital paintings in Jingchu style based on artificial intelligence according to claim 1, characterized in that: In step S1, the Jingchu painting collection is used to collect the original image reference data set required to generate Jingchu style digital paintings, specifically, to obtain the original data of Jingchu style digital paintings from Jingchu style artworks through data collection and basic data annotation.

3. The method for generating digital paintings in Jingchu style based on artificial intelligence according to claim 2 is characterized by: In step S3, the improved lightweight U-shaped network specifically includes a deep convolution extraction layer, an improved U-shaped encoding side, a bottleneck sub-block, a U-shaped decoding side and a feature mapping output layer; The deep convolution extraction layer is used to preliminarily extract shallow features; The improved U-type coding side is used to call the Mamba module in combination with the visual residual module to improve the U-type coding side, perform feature coding, and reduce the resolution of the feature image through a maximum pooling operation; The bottleneck sub-block is used to model the spatial long-term dependency relationship within the image; The U-type decoding side is used to perform feature decoding to restore image resolution; The feature mapping output layer is used to construct a fully connected layer for feature mapping, and uses a classifier and an activation function to generate a segmented image.

4. The method for generating digital paintings in Jingchu style based on artificial intelligence according to claim 3 is characterized by: In step S3, the step of using the improved lightweight U-shaped network to extract the core area of ​​the painting to obtain the core painting area data of the Jingchu style includes: Step S31: construct a deep convolution extraction layer, specifically, by constructing a deep convolution layer, receiving the optimized digital painting data as an input data sample of the model, and performing shallow feature extraction on the data input to obtain shallow feature data, and the calculation formula is: I∈R C×H×W×D →F S ∈R 32×H×W×D ; In the formula, I is the input data sample of the model, F S is shallow feature data, R is the value domain corresponding to the optimized digital painting data, wherein C is the number of channels of image data in the optimized digital painting data, H is the height of the image data, W is the width of the image data, D is the number of slices of the image data, a value of 32 is used to represent a fixed number of feature filters, and → is a feature extraction operator; Step S32: constructing an improved U-type coding side, specifically constructing a three-layer encoder sub-block, and calling the Mamba module in combination with the visual residual module to improve and optimize the U-type coding side to obtain deep feature data; Step S33: constructing a bottleneck sub-block, specifically, constructing a bottleneck sub-block after the third-layer encoder sub-block, simulating the long-range spatial dependency, obtaining the bottleneck sub-block output feature, and connecting the bottleneck sub-block output feature to the U-type decoding side; Step S34: constructing a U-shaped decoding side, specifically constructing a three-layer decoder to use addition operation to perform feature fusion, and then perform feature decoding to obtain decoding feature data. The calculation formula is: Where P OUT is the decoded feature data, ReLU(·) is the nonlinear activation function, DWConv(·) is the deep convolution operation, P in is the bottleneck sub-block output feature, is the deep feature output, s′ is the decoding adjustment factor; Step S35: constructing a feature mapping output layer, specifically by constructing a fully connected layer, performing feature mapping of the decoded feature data, and constructing a Softmax classifier for feature classification and detection, to obtain detection classification data information of the painting core area; Step S36: core extraction model training, specifically, by constructing the deep convolution extraction layer, constructing the improved U-shaped encoding side, constructing the bottleneck sub-block, constructing the U-shaped decoding side and constructing the feature map output layer, the core extraction model training is performed to obtain the painting core area extraction model Model MU ; Step S37: Extracting the core area of ​​the painting, specifically, using the painting core area extraction model Model according to the optimized digital painting data MU , extract the core area of ​​the painting and obtain the core painting area data of the Jingchu style.

5. The method for generating digital paintings in Jingchu style based on artificial intelligence according to claim 4 is characterized by: The convolutional neural network optimized by the style transfer network in step S4 specifically includes a basic convolutional generation subnet, a style transfer optimization loss function group and an edge detection synthesis operator; The basic convolutional generation subnetwork is used to construct a basic network for generating digital paintings; The style transfer optimization loss function group is used to optimize the transfer of the painting style of Jingchu style; The edge detection synthesis operator is used to optimize the edge effect between image elements in the digital painting synthesis process; The step of using a convolutional neural network optimized by a style transfer network to generate stylized paintings and obtain Jingchu stylized painting data includes: Step S41: construct a basic convolutional subnet, specifically, construct a basic convolutional subnet with the fully connected layer removed, including an input layer, a convolutional layer, a pooling layer, and an output layer, and optimize features by average pooling; Step S42: constructing a style transfer optimization loss function group, specifically, taking the optimized digital painting data as the basic content image, taking the Jingchu style core painting area data as the style pattern image, constructing a style transfer optimization loss function group, and performing stylized painting model training optimization by constructing the style transfer optimization loss function group, including the following steps: Step S421: construct a stylized content loss function, the calculation formula is: Where, L Sc is the stylized content loss function, N C is the total number of feature maps corresponding to the basic content image, M C is the product of the height and width of the corresponding basic content image, X C is the basic content image, i is the horizontal index of the feature map pixel, j is the vertical index of the feature map pixel, and F C (·) is the filter activation function, is to generate an image; Step S422: construct a stylized feature space loss function, the calculation formula is: Where, L Ss is the stylized feature space loss function, L is the convolution layer index, N S is the total number of feature maps corresponding to the style image, w L is the stylized feature space weight of the Lth layer, i is the horizontal index of the feature map pixel, j is the vertical index of the feature map pixel, G(·) is the Grammar matrix, X S is the style image, is to generate an image; Step S423: construct a stylized integration loss function, the calculation formula is: L TOTAL =α·L Sc +β·L Ss ; Where, L TOTAL is the stylized integration loss function, α is the stylized content loss weight, L Sc is the stylized content loss function, β is the stylized feature space loss weight, L Ss is the stylized feature space loss function; Step S43: constructing an edge detection synthesis operator, specifically using a Sobel operator to perform edge detection, and optimizing drawing synthesis through edge detection; Step S44: stylized painting model training, specifically, by constructing the basic convolutional generation subnet, constructing the style transfer optimization loss function group and constructing the edge detection synthesis operator, using the adversarial training method to train the stylized painting model to obtain the stylized painting model Model SP ; Step S45: generating a stylized painting, specifically using the stylized painting model Model SP , Jingchu stylized painting data is generated.

6. The method for generating digital paintings in Jingchu style based on artificial intelligence according to claim 5 is characterized by: In step S5, the Jingchu style digital painting is generated to generate a specific Jingchu style painting work, specifically by obtaining the Jingchu stylized painting data, creating a digital painting that conforms to the Jingchu style, and generating a Jingchu style digital painting work image.

7. A Jingchu style digital painting generation system based on artificial intelligence, used to implement a Jingchu style digital painting generation method based on artificial intelligence as described in any one of claims 1 to 6, characterized in that: It includes Jingchu painting acquisition module, image preprocessing module, painting core area extraction module, stylized painting generation module and Jingchu style digital painting generation module.

8. The artificial intelligence-based Jingchu style digital painting generation system according to claim 7 is characterized by: The Jingchu painting collection module is used for collecting Jingchu paintings, obtaining Jingchu style digital painting original data through Jingchu painting collection, and sending the Jingchu style digital painting original data to the image preprocessing module; The image preprocessing module is used for image preprocessing, and obtains optimized digital painting data through image preprocessing, and sends the optimized digital painting data to the painting core area extraction module and the stylized painting generation module; The painting core area extraction module is used for painting core area extraction, and obtains Jingchu style core painting area data through painting core area extraction, and sends the Jingchu style core painting area data to the stylized painting generation module; The stylized painting generation module is used for stylized painting generation, obtains Jingchu stylized painting data through stylized painting generation, and sends the Jingchu stylized painting data to the Jingchu style digital painting generation module; The Jingchu style digital painting generation module is used for Jingchu style digital painting generation, and obtains a Jingchu style digital painting work image through Jingchu style digital painting generation.

Images