Picture generation method based on artificial intelligence

Through feature extraction and fusion technology based on CNN model, the problems of low efficiency and high cost of traditional drawing technology are solved, and fast and high-quality image generation is achieved, adapting to personalized customization needs, and improving the robustness and generalization of the model are improved. ability.

CN119090986APending Publication Date: 2024-12-06SHANGHAI JINGSHUO NETWORK TECH CO LTD
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202411153018.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-21
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

Traditional drawing technology relies on manual operations, is inefficient and costly, and is difficult to meet the needs of personalized customization, especially in complex tasks and large-scale image data processing, which is difficult to quickly generate high-quality renderings.

Method used

A feature extraction method based on the CNN model is adopted to extract and fuse the user's required style diagrams and content diagrams, and the computer's rapid processing capabilities are used to generate renderings, maintain the main information of the content image and naturally integrate the features of the style image.

Benefits of technology

Reduces manual intervention, improves processing efficiency and accuracy, and can quickly process large amounts of image data. The model can still maintain good performance in styles or content that have not been seen before. By combining the loss function, it generates high-quality and visually good images. Improves the robustness and generalization capabilities of the model.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119090986A_ABST
    Figure CN119090986A_ABST
Patent Text Reader

Abstract

The invention discloses a picture generation method based on artificial intelligence, and the method comprises the steps: S1, setting a data set, and carrying out the standardization processing; s2, performing data enhancement on the picture data; s3, constructing a convolutional neural network (CNN) model; s4, training a CNN model; s5, migrating a preset style picture to the feature graph to generate an effect graph; and S6, evaluating the quality of the effect picture, modifying model parameters according to an evaluation result, and performing parameter optimization on the CNN model. Based on the feature extraction capability of a CNN model, feature extraction fusion is performed on a style image and a content image required by a user, an effect image is automatically generated, the effect image is generated with high quality and high efficiency by using the rapid processing capability of a computer, and the features of the style image can be naturally fused while the main information of the content image is maintained, so that the user experience is improved. Therefore, the requirements of users are met.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of picture generation, and more specifically, particularly relates to a picture generation method based on artificial intelligence. Background Art

[0002] With the continuous improvement of computing resources and the development of optimization algorithms, modern image generation technology is moving towards higher scalability and real-time performance. Traditional drawing relies heavily on manual operation. This dependence is not only reflected in painting skills and artistic creation, but also runs through multiple links in the entire drawing process. It not only relies on the artist's imagination and creativity, but also requires the selection of appropriate techniques and the proficiency in their application skills. Therefore, it requires a lot of manpower and high skills;

[0003] However, with the development of science and technology, digital drawing technology has gradually emerged and has been widely used. With the help of tools such as computers and drawing software, digital drawing technology has realized the automation and digitization of the drawing process, greatly improving drawing efficiency and work quality. This means that even when dealing with large-scale data sets or complex tasks, it can maintain a faster generation speed and better generation quality. This is especially important for scenarios such as real-time video processing, online interactive applications, and large-scale image data set generation.

[0004] In modern society, the demand for personalized customization is increasing. Whether it is advertising design, product design or media content creation, it needs to be customized according to the preferences and needs of the target audience in order to attract and retain users. Manual drawing takes a long time to complete complex tasks and requires rich experience and professional technology, resulting in high costs and insufficient efficiency to adapt to a large number of tasks. Summary of the invention

[0005] The purpose of the present invention is to solve the shortcomings of the above-mentioned prior art and provide an artificial intelligence-based picture generation method. Through the feature extraction capability based on the CNN model, the style map required by the user is integrated with the content map for feature extraction and automatic generation of the effect map. The fast processing capability of the computer is used to generate the effect map with high quality and efficiency. While maintaining the main information of the content image, the characteristics of the style image can be naturally integrated to meet the needs of users.

[0006] To achieve the above object, the present invention provides the following technical solution: a method for generating an image based on artificial intelligence, comprising the following steps:

[0007] S1. Collect a large amount of image data, set up a data set, and standardize the image data in the data set;

[0008] S2, perform data enhancement on the image data, including scaling, cropping and normalization operations on the image;

[0009] S3. Construct a convolutional neural network (CNN) model, extract high-level features from the original image based on the CNN model, and capture the key information of the image;

[0010] S4, training the CNN model, using the preprocessed data set to train the CNN model, and adjusting the CNN model parameters to optimize the quality and diversity of the generated images;

[0011] S5. Based on the trained CNN model, the preset style image is migrated to the feature map, the style and content are integrated, and the effect map is generated;

[0012] S6. Evaluate the quality of the renderings, modify the model parameters based on the evaluation results, and optimize the parameters of the CNN model.

[0013] Optionally, in step S1, the data set includes: a training set, a validation set and a test set;

[0014] The training set, validation set, and test set are allocated in a ratio of 7:2:1 and are independent of each other;

[0015] The training set is used to train the CNN model and annotate the feature data of the images in the training set;

[0016] The validation set is used to evaluate the performance of the model during the training process, and to adjust the hyperparameters and structure of the model according to the results of the validation set;

[0017] The test set is used to evaluate the final performance and generalization ability of the model;

[0018] The normalization process includes adjusting the image pixel size, unifying the color mode and the channel order.

[0019] Optionally, in step S2, the data enhancement includes: rotation, scaling, cropping, flipping, color conversion, normalization and image filtering;

[0020] The rotation, scaling, cropping, flipping and color transformation generate a large number of new image samples that are similar to and different from the original images to increase the training set;

[0021] The specific process of normalization is as follows:

[0022] 1) Determine the target range of data normalization to be [-1,1];

[0023] 2) Calculate the minimum and maximum values ​​of a data set

[0024] 3) According to the selected normalization range [-1,1], the data is processed using the normalization formula;

[0025] 4) The original data is converted by a normalization formula to obtain data within a specified range;

[0026] The normalization algorithm formula is:

[0027]

[0028] Among them, X is the new data;

[0029] Y is a value of the original data;

[0030] min is the minimum value in the data set;

[0031] max is the maximum value in the data set.

[0032] Optionally, in step S3, the construction of the convolutional neural network (CNN) model includes: an input layer, a convolutional layer, an activation function, a pooling layer, and a fully connected layer;

[0033] Input layer: receives image data and passes it to the convolutional layer for processing;

[0034] Convolutional layer: extracts local features of image data, calculates the weights of local features, and generates feature maps;

[0035] Activation function: performs nonlinear transformation on the output of the convolutional layer to provide nonlinear feature representation;

[0036] Pooling layer: compresses and reduces the dimension of feature maps, reduces the number of parameters of feature maps and prevents overfitting;

[0037] Fully connected layer: globally integrates the features extracted by the convolutional layer and the pooling layer to generate a feature vector of fixed length;

[0038] The convolution layer has a set of convolution kernels built in, and the convolution kernels are configured with weight parameters;

[0039] The process of extracting features by the convolutional layer is as follows:

[0040] 1) The convolution kernel slides the input image from left to right and from top to bottom according to a certain step size;

[0041] 2) After the convolution kernel moves a step size, the image pixel value within the range of the convolution kernel is obtained, and then each weight parameter of the convolution kernel is multiplied by the image pixel value at the corresponding position to obtain the feature value of the range;

[0042] 3) Multiple different convolution kernels slide to obtain a set of eigenvalues, and the feature map is obtained by combining the eigenvalues.

[0043] Optionally, in step S4, the specific process of training the CNN model is:

[0044] S41. Set hyperparameters, loss functions, and optimizers, including learning rate, batch size, number of iterations, regularization parameters, and network structure parameters;

[0045] S42, input the training set image, and obtain the output image through the CNN model;

[0046] S43, calculating the loss value through the loss function;

[0047] S44. According to the loss value, reversely adjust the parameters of each layer in the CNN model;

[0048] S45, repeating steps S42 to S44 for the purpose of reducing the loss value;

[0049] S46, inserting validation set image data at intervals to detect the output capability of the CNN model and determine whether the CNN model is overfitting;

[0050] S47, input the test set to detect the CNN model training results;

[0051] The meaning of the loss value is:

[0052] The greater the difference between the output image and the predicted image, the higher the loss value;

[0053] The smaller the difference between the output image and the predicted image, the smaller the loss value.

[0054] Optionally, in step S43, the loss function is:

[0055] L custom =λ1·L MSE +λ2·L SSIM +λ3·L Gragient

[0056] Among them, λ1, λ2, and λ3 are weights for balancing the contributions of different loss terms, and need to be adjusted according to the training results and data sets;

[0057] L MSE is the mean square error loss;

[0058]

[0059] Where: N is the total number of pixels;

[0060] y i is the pixel value of the output image at pixel i;

[0061] is the pixel value of the output image at pixel i;

[0062] L SSIM is the structural similarity loss;

[0063]

[0064] in, To calculate the output image y and the output image The function of the SSIM value between

[0065] The L SSIM The closer the value is to 1, the more similar the images are;

[0066] L Gragient is the gradient loss;

[0067]

[0068] in, and Represent the gradient operations in the horizontal and vertical directions respectively.

[0069] Optionally, the specific process of step S5 is:

[0070] S51, inputting the preset style image and content image into the CNN model respectively;

[0071] S52,CNN model extracts feature maps of style image and content image through convolution layer and pooling layer respectively;

[0072] S53, calculating the correlation between the feature graphs of the style image by using a Gram matrix algorithm to represent the style;

[0073] S54, performing feature reorganization on the feature map to fuse the feature maps of the style and content images;

[0074] S55, inputting the melted feature map into the automatic encoder to generate an effect map;

[0075] S56. Post-processing the generated image, including adjusting color, contrast, and sharpening.

[0076] Optionally, the specific process of step S6 is:

[0077] S61. Quality evaluation of renderings: Visual observation of the extent to which style features are transferred to content images and systematic evaluation;

[0078] S62. Determine the optimization direction based on the analysis results, including fusion features, optimization algorithm and increasing model complexity;

[0079] S63, iterative optimization: retrain the model based on the adjusted parameters and structure;

[0080] S64. Convergence judgment: Observe the loss curve and the changing trend of the evaluation indicators during the training process to determine whether the model has converged to the optimal state.

[0081] Optionally, in step S61, the system evaluation includes: style similarity evaluation, content preservation evaluation and image quality evaluation;

[0082] The style similarity evaluation and content preservation evaluation are both performed by setting quantitative indicators to evaluate the similarity between the generated effect image and the style image, including the following steps:

[0083] 1) Extract feature maps of output image and style image;

[0084] 2) Calculate the Gram matrix of the feature map and obtain the global statistical information of the feature map;

[0085] 3) Calculate the similarity between the style image and the generated effect image;

[0086] The image quality assessment: assesses the overall quality of the generated image, including clarity, noise level, and color reproduction, and provides feedback on the image quality.

[0087] Optionally, the similarity calculation formula is:

[0088] The feature vectors of the two images are A=[A1,A2,A3,...A n ,] and B=[B1,B2,B3,...B n ,]

[0089] 1) Calculate the dot product:

[0090]

[0091] 3) Calculate the module length:

[0092]

[0093] 3) Calculate similarity:

[0094]

[0095] Where: θ is the angle between the two vectors.

[0096] The technical effects and advantages of the present invention are as follows: compared with the traditional manually drawn renderings, first, the CNN model can automatically extract hierarchical features from the input image, reducing the need for manual intervention and improving processing efficiency and accuracy; second, through parallel processing and optimization algorithms, a large amount of image data can be processed faster; third, a fully trained CNN model can learn the general feature representation in the image, not just the features for a specific image or style, so that the model can still maintain good performance when processing unseen styles or content; fourth, features of different scales are extracted through convolutional layers of different depths to improve the quality of the fused image;

[0097] Secondly, by combining gradient loss, MSE loss and SSIM loss, the edge information, overall error and structural characteristics of the image can be comprehensively considered to train a model with better performance. In the image generation task, this combined loss function helps to generate images with higher quality and better visual effects. The combined use of different loss functions can reduce the model's sensitivity to specific types of errors, thereby improving the model's robustness and generalization ability.

[0098] Further features and advantages of the present invention will become apparent from the following detailed description of exemplary embodiments of the present invention with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0099] Figure 1 It is a flow chart of the picture generation method provided by the present invention;

[0100] Figure 2 It is a step diagram of training CNN model provided by the present invention;

[0101] Figure 3 It is a picture generation operation flow chart provided by the present invention. DETAILED DESCRIPTION

[0102] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention is further described in detail below in conjunction with specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0103] like Figures 1 to 3 As shown, an artificial intelligence-based image generation method provided by an embodiment of the present invention includes the following steps:

[0104] S1. Collect a large amount of image data, set up a data set, and standardize the image data in the data set;

[0105] S2, perform data enhancement on the image data, including scaling, cropping and normalization operations on the image;

[0106] S3. Construct a convolutional neural network (CNN) model, extract high-level features from the original image based on the CNN model, and capture the key information of the image;

[0107] S4, training the CNN model, using the preprocessed data set to train the CNN model, and adjusting the CNN model parameters to optimize the quality and diversity of the generated images;

[0108] S5. Based on the trained CNN model, the preset style image is migrated to the feature map, the style and content are integrated, and the effect map is generated;

[0109] S6. Evaluate the quality of the renderings, modify the model parameters based on the evaluation results, and optimize the parameters of the CNN model.

[0110] Furthermore, based on the powerful feature extraction capability of the convolutional neural network (CNN) model, each convolutional layer can learn different features, from low-level edges and textures to high-level shapes and patterns, by performing hierarchical feature extraction on the input image. This hierarchical feature extraction mechanism enables CNN to adaptively learn and understand image content;

[0111] Furthermore, according to the above method, a large number of images need to be collected as data sets to train the constructed CNN model, so that the CNN model can generate the required renderings more accurately. Then, based on the powerful extraction capability of the CNN model, the style map and the content map are input into the CNN model, and the feature maps of the two are extracted and fused using the CNN model. Then, the fused feature maps are input into the autoencoder to generate renderings, thereby realizing automatic generation of images.

[0112] Furthermore, the CNN model is installed in a server, and a control module is provided in the server, including a processor and related software algorithms for calculating and extracting image features and generating effect diagrams;

[0113] When the feedback module is used to train the CNN model, it receives system evaluation to optimize the model parameters;

[0114] The human-computer interaction module is used to communicate with the user, receive image data input by the user, and output effect diagrams.

[0115] Specifically, in step S1, the data set includes: a training set, a validation set and a test set;

[0116] The training set, validation set, and test set are allocated in a ratio of 7:2:1 and are independent of each other;

[0117] The training set is used to train the CNN model and annotate the feature data of the images in the training set;

[0118] The validation set is used to evaluate the performance of the model during the training process, and to adjust the hyperparameters and structure of the model according to the results of the validation set;

[0119] The test set is used to evaluate the final performance and generalization ability of the model;

[0120] The normalization process includes adjusting the image pixel size, unifying the color mode and the channel order.

[0121] Furthermore, by training and evaluating the model on the training set, validation set, and test set, we can ensure that the model not only performs well on the training data, but also makes accurate predictions on new, unseen data; using the validation set can help us detect whether the model is overfitting in a timely manner and take measures to solve it; the test set provides an independent evaluation to determine how the model may perform in actual applications, which helps to make a realistic and objective expectation of the model's performance.

[0122] Specifically, in step S2, the data enhancement includes: rotation, scaling, cropping, flipping, color conversion, normalization and image filtering;

[0123] The rotation, scaling, cropping, flipping and color transformation generate a large number of new image samples that are similar to and different from the original images to increase the training set;

[0124] Furthermore, by rotating, scaling, cropping, flipping, color transformation and other operations on the original images in the dataset, the number of images in the dataset is further increased, thereby providing a large amount of original training data for the CNN model. In addition, rotating the image around its center by a certain angle can help the CNN model learn to recognize objects in different directions; scaling the size of the image proportionally helps the model adapt to objects of different sizes; cropping a part of the image can simulate different perspectives and focal points, allowing the model to handle partial occlusions; mirroring the image along the horizontal or vertical axis allows the model to learn the same object from different angles; changing the color balance of the image, such as adjusting brightness, contrast, saturation, etc., helps the model adapt to different lighting conditions and color deviations; by reasonably applying operations such as rotation, scaling, cropping, flipping and color transformation, the performance and robustness of the model can be improved;

[0125] Furthermore, when making the above adjustments to the original image, the transformations should be diverse enough to cover situations that may be encountered in practical applications; avoid overuse of certain transformations, which may lead to skewed data sets; and the transformed image needs to remain clear to prevent distortion or other quality issues.

[0126] Specifically, the specific process of the normalization is:

[0127] 1) Determine the target range of data normalization to be [-1,1];

[0128] 2) Calculate the minimum and maximum values ​​of a data set

[0129] 3) According to the selected normalization range [-1,1], the data is processed using the normalization formula;

[0130] 4) The original data is converted by a normalization formula to obtain data within a specified range;

[0131] The normalization algorithm formula is:

[0132]

[0133] Among them, X is the new data;

[0134] Y is a value of the original data;

[0135] min is the minimum value in the data set;

[0136] max is the maximum value in the data set.

[0137] Furthermore, due to the differences in the value ranges of different features, some features will dominate the model training process, while the influence of other features will be weakened. Through normalization, the value ranges of all features can be scaled to the same scale, thereby eliminating this scale difference and enabling the model to consider all features more fairly.

[0138] During the training process, unifying the value range of features can make the amplitude and direction of parameter updates more consistent, thereby accelerating the model training process; and normalization can significantly improve the performance of the model. By eliminating scale differences, the model can more accurately capture the relationship between features, thereby improving the accuracy of prediction or classification.

[0139] In addition, normalization can reduce the model's sensitivity to the absolute value of the input data, allowing the model to maintain good performance on unseen data, improving the model's generalization ability and thus reducing the degree of overfitting of the model to the training data.

[0140] Specifically, in step S3, the convolutional neural network (CNN) model is constructed including: an input layer, a convolutional layer, an activation function, a pooling layer, and a fully connected layer;

[0141] Input layer: receives image data and passes it to the convolutional layer for processing;

[0142] Convolutional layer: extracts local features of image data, calculates the weights of local features, and generates feature maps;

[0143] Activation function: performs nonlinear transformation on the output of the convolutional layer to provide nonlinear feature representation;

[0144] Pooling layer: compresses and reduces the dimension of feature maps, reduces the number of parameters of feature maps and prevents overfitting;

[0145] Fully connected layer: globally integrates the features extracted by the convolutional layer and the pooling layer to generate a feature vector of fixed length;

[0146] The convolution layer has a set of convolution kernels built in, and the convolution kernels are configured with weight parameters;

[0147] Specifically, the process of extracting features by the convolutional layer is as follows:

[0148] 1) The convolution kernel slides the input image from left to right and from top to bottom according to a certain step size;

[0149] 2) After the convolution kernel moves a step size, the image pixel value within the range of the convolution kernel is obtained, and then each weight parameter of the convolution kernel is multiplied by the image pixel value at the corresponding position to obtain the feature value of the range;

[0150] 3) Multiple different convolution kernels slide to obtain a set of eigenvalues, and the feature map is obtained by combining the eigenvalues.

[0151] Furthermore, convolution kernels are the core components of the CNN model, which are responsible for performing sliding window operations on the input image to extract useful features, including edges, textures, shapes, etc.

[0152] Furthermore, the parameters of the convolution kernel include size, stride, padding, weight, and bias;

[0153] The size of the convolution kernel determines the size of the local area considered during the convolution operation, such as 1x1, 3x3, 5x5, etc. The size of the convolution kernel is selected as an odd number to maintain the central symmetry of the feature map;

[0154] The step size of the convolution kernel determines the distance the convolution kernel slides on the input data;

[0155] The padding is to add extra pixel values ​​at the edge of the input data to control the size of the output feature map, thereby ensuring that the size of the output feature map is the same as the input data or is reduced as expected;

[0156] The weights of the convolution kernels are coefficients in the convolution operation, which are weighted summed with the local area of ​​the input data to produce an output feature map, and the weights are learned through network training;

[0157] The bias is an optional parameter used on the output of the convolution operation. By adjusting it during the network training process, the model's ability to extract features is improved.

[0158] Specifically, in step S4, the specific process of training the CNN model is:

[0159] S41. Set hyperparameters, loss functions, and optimizers, including learning rate, batch size, number of iterations, regularization parameters, and network structure parameters;

[0160] S42, input the training set image, and obtain the output image through the CNN model;

[0161] S43, calculating the loss value through the loss function;

[0162] S44. According to the loss value, reversely adjust the parameters of each layer in the CNN model;

[0163] S45, repeating steps S42 to S44 for the purpose of reducing the loss value;

[0164] S46, inserting validation set image data at intervals to detect the output capability of the CNN model and determine whether the CNN model is overfitting;

[0165] S47, input the test set to detect the CNN model training results;

[0166] Furthermore, the learning rate is used to determine the magnitude of weight updates during the CNN model learning process, affecting the convergence speed and stability;

[0167] The batch size determines the number of samples selected for a single update, affecting memory usage and the smoothness of the learning curve;

[0168] The network structure parameters determine the number of convolutional layers, the size of the convolution kernel, the number of neurons in the fully connected layer, etc., which affect the network capacity and complexity.

[0169] The meaning of the loss value is:

[0170] The greater the difference between the output image and the predicted image, the higher the loss value;

[0171] The smaller the difference between the output image and the predicted image, the smaller the loss value.

[0172] Specifically, in step S43, the loss function is:

[0173] L custom =λ1·L MSE +λ2·L SSIM +λ3·L Gragient

[0174] Among them, λ1, λ2, and λ3 are weights for balancing the contributions of different loss terms, and need to be adjusted according to the training results and data sets;

[0175] L MSE is the mean square error loss;

[0176]

[0177] Where: N is the total number of pixels;

[0178] y i is the pixel value of the output image at pixel i;

[0179] is the pixel value of the output image at pixel i;

[0180] L SSIM is the structural similarity loss;

[0181]

[0182] in, To calculate the output image y and the output image The function of the SSIM value between

[0183] The L SSIM The closer the value is to 1, the more similar the images are;

[0184] L Gragient is the gradient loss;

[0185]

[0186] in, and Represent the gradient operations in the horizontal and vertical directions respectively.

[0187] Furthermore, by optimizing the loss function, the mean square error, structural similarity loss and gradient loss are added when calculating the loss value;

[0188] Gradient loss focuses on the rate of change of pixel values ​​in an image, i.e. edge information. In image generation tasks, increasing gradient loss can help the model better generate clear edges, thereby improving the visual effect and detail quality of the image. By penalizing discontinuities between adjacent pixels, gradient loss helps ensure that the generated image is visually smoother and more natural.

[0189] The MSE loss function is simple to calculate and can directly measure the mean square error between the predicted value and the actual value, making the optimization process relatively easy and helping the model converge quickly;

[0190] The SSIM loss function combines three factors: brightness similarity, contrast similarity, and structural similarity, which is more in line with the human eye's perception of image quality. Therefore, in the image generation task, the use of the SSIM loss function can generate images that are more in line with human visual perception. In addition, the SSIM loss function pays special attention to the structural information of the image, which helps the model maintain the structural features of the original image when generating images, such as texture and shape.

[0191] By combining gradient loss, MSE loss and SSIM loss, the edge information, overall error and structural characteristics of the image can be comprehensively considered to train a model with better performance. In the image generation task, this combined loss function helps to generate images with higher quality and better visual effects. The combined use of different loss functions can reduce the model's sensitivity to specific types of errors, thereby improving the model's robustness and generalization ability.

[0192] Specifically, the specific process of step S5 is:

[0193] S51, inputting the preset style image and content image into the CNN model respectively;

[0194] S52,CNN model extracts feature maps of style image and content image through convolution layer and pooling layer respectively;

[0195] S53, calculating the correlation between the feature graphs of the style image by using a Gram matrix algorithm to represent the style;

[0196] S54, performing feature reorganization on the feature map to fuse the feature maps of the style and content images;

[0197] S55, inputting the melted feature map into the automatic encoder to generate an effect map;

[0198] S56. Post-processing the generated image, including adjusting color, contrast, and sharpening.

[0199] Furthermore, based on the style transfer process of the trained CNN model, by inputting different style images and content images, users can flexibly generate new images with various artistic styles. The entire process is highly automated, from feature extraction to feature fusion, to image generation and post-processing, all completed automatically by computers without human intervention, greatly improving processing efficiency and being able to generate high-quality style transfer images in a short time, meeting the needs of fast iteration and real-time processing;

[0200] Furthermore, in the feature recombination process, the key features of the content image, such as shape and layout, are effectively retained, ensuring that the generated image is visually consistent with the original content image. The style features are integrated into the feature map of the content image, so that the generated image not only retains the structural information of the original content image, but also has a new artistic style.

[0201] Furthermore, users can control the style intensity, clarity and other attributes of the generated image to meet different needs by adjusting the parameters of the CNN model, autoencoder and the parameters in the post-processing steps such as color, contrast, sharpening intensity, etc.; by adjusting the weight ratio of the style image and the content image in the feature fusion process, the degree of style transfer can be controlled so that the generated image achieves an ideal balance between style and content.

[0202] Specifically, the specific process of step S6 is:

[0203] S61. Quality evaluation of renderings: Visual observation of the extent to which style features are transferred to content images and systematic evaluation;

[0204] S62. Determine the optimization direction based on the analysis results, including fusion features, optimization algorithm and increasing model complexity;

[0205] S63, iterative optimization: retrain the model based on the adjusted parameters and structure;

[0206] S64. Convergence judgment: Observe the loss curve and the changing trend of the evaluation indicators during the training process to determine whether the model has converged to the optimal state.

[0207] Furthermore, by including whether the style elements such as color, texture, and brushstrokes in the image are consistent with the target style, and observing whether these elements merge naturally with the content image, we can evaluate whether the generated image successfully transfers the features of the style image to the content image. In addition to subjective visual observation, objective system evaluation methods can also be used, such as calculating the weighted sum of style loss and content loss to quantitatively evaluate the closeness of the generated image to the expected target.

[0208] Furthermore, based on the evaluation results, we can first adjust the fusion weights of feature maps at different layers, or adopt a more complex fusion mechanism to better balance the relationship between style transfer and content preservation. Second, we can try different optimizers, learning rate adjustment strategies, or regularization techniques to speed up the training process and improve model performance.

[0209] If the current model cannot fully capture the characteristics of the style image or cannot generate high-quality images, the complexity of the model can be increased by increasing the number of convolutional layers, using a more complex network structure, or introducing an attention mechanism.

[0210] Specifically, in step S61, the system evaluation includes: style similarity evaluation, content preservation evaluation and image quality evaluation;

[0211] The style similarity evaluation and content preservation evaluation are both performed by setting quantitative indicators to evaluate the similarity between the generated effect image and the style image, including the following steps:

[0212] 1) Extract feature maps of output image and style image;

[0213] 2) Calculate the Gram matrix of the feature map and obtain the global statistical information of the feature map;

[0214] 3) Calculate the similarity between the style image and the generated effect image;

[0215] The image quality assessment: assesses the overall quality of the generated image, including clarity, noise level, and color reproduction, and provides feedback on the image quality.

[0216] Specifically, the similarity calculation formula is:

[0217] The feature vectors of the two images are A=[A1,A2,A3,...A n ,] and B=[B1,B2,B3,...B n ,]

[0218] 1) Calculate the dot product:

[0219]

[0220] 4) Calculate the module length:

[0221]

[0222] 3) Calculate similarity:

[0223]

[0224] Where: θ is the angle between the two vectors.

[0225] Furthermore, in image processing, since style transfer often only focuses on the transfer of artistic style and does not require absolute parameters such as brightness and contrast to be completely consistent, even if the style image and the generated effect image have differences in brightness and contrast, as long as they are similar in directional features such as color distribution and texture pattern, cosine similarity can still give a higher similarity score, which is particularly important for style transfer tasks;

[0226] Furthermore, in style transfer, different feature dimensions correspond to different elements (such as color, brushstroke, texture, etc.), and cosine similarity can well capture the relative importance between different elements in the feature vector, thereby more accurately evaluating the similarity of the overall style;

[0227] The value range of cosine similarity is between -1 and 1, where 1 means that the two vectors are exactly the same, -1 means that the two vectors are completely opposite, and 0 means that the two vectors are orthogonal (i.e., neither similar nor opposite). This standardized processing method makes cosine similarity more comparable and interpretable, which facilitates comparison and analysis between different images.

[0228] In summary, compared with the traditional manually drawn renderings, the present invention has the following advantages: first, the CNN model can automatically extract hierarchical features from the input image, reducing the need for manual intervention and improving processing efficiency and accuracy; second, through parallel processing and optimization algorithms, a large amount of image data can be processed faster; third, the fully trained CNN model can learn the general feature representation in the image, not just the features for a specific image or style, so that the model can still maintain good performance when processing unseen styles or content; fourth, features of different scales are extracted through convolutional layers of different depths to improve the quality of the fused image;

[0229] Secondly, by combining gradient loss, MSE loss and SSIM loss, the edge information, overall error and structural characteristics of the image can be comprehensively considered to train a model with better performance. In the image generation task, this combined loss function helps to generate images with higher quality and better visual effects. The combined use of different loss functions can reduce the model's sensitivity to specific types of errors, thereby improving the model's robustness and generalization ability.

[0230] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for generating pictures based on artificial intelligence, characterized in that: The steps include: S1. Collect a large amount of image data, set up a data set, and standardize the image data in the data set; S2, perform data enhancement on the image data, including scaling, cropping and normalization operations on the image; S3. Construct a convolutional neural network (CNN) model, extract high-level features from the original image based on the CNN model, and capture the key information of the image; S4, training the CNN model, using the preprocessed data set to train the CNN model, and adjusting the CNN model parameters to optimize the quality and diversity of the generated images; S5. Based on the trained CNN model, the preset style image is migrated to the feature map, the style and content are integrated, and the effect map is generated; S6. Evaluate the quality of the renderings, modify the model parameters based on the evaluation results, and optimize the parameters of the CNN model.

2. The method for generating an image based on artificial intelligence according to claim 1, characterized in that: In step S1, the data set includes: a training set, a validation set and a test set; The training set, validation set, and test set are allocated in a ratio of 7:2:1 and are independent of each other; The training set is used to train the CNN model and annotate the feature data of the images in the training set; The validation set is used to evaluate the performance of the model during the training process, and to adjust the hyperparameters and structure of the model according to the results of the validation set; The test set is used to evaluate the final performance and generalization ability of the model; The normalization process includes adjusting the image pixel size, unifying the color mode and the channel order.

3. The method for generating an image based on artificial intelligence according to claim 1, characterized in that: In step S2, the data enhancement includes: rotation, scaling, cropping, flipping, color conversion, normalization and image filtering; The rotation, scaling, cropping, flipping and color transformation generate a large number of new image samples that are similar to and different from the original images to increase the training set; The specific process of normalization is as follows: 1) Determine the target range of data normalization to be [-1,1]; 2) Calculate the minimum and maximum values ​​of a data set 3) According to the selected normalization range [-1,1], the data is processed using the normalization formula; 4) The original data is converted by a normalization formula to obtain data within a specified range; The normalization algorithm formula is: Among them, X is the new data; Y is a value of the original data; min is the minimum value in the data set; max is the maximum value in the data set.

4. The method for generating an image based on artificial intelligence according to claim 1, characterized in that: In step S3, the convolutional neural network (CNN) model is constructed, including: an input layer, a convolutional layer, an activation function, a pooling layer, and a fully connected layer; Input layer: receives image data and passes it to the convolutional layer for processing; Convolutional layer: extracts local features of image data, calculates the weights of local features, and generates feature maps; Activation function: performs nonlinear transformation on the output of the convolutional layer to provide nonlinear feature representation; Pooling layer: compresses and reduces the dimension of feature maps, reduces the number of parameters of feature maps and prevents overfitting; Fully connected layer: globally integrates the features extracted by the convolutional layer and the pooling layer to generate a feature vector of fixed length; The convolution layer has a set of convolution kernels built in, and the convolution kernels are configured with weight parameters; The process of extracting features by the convolutional layer is as follows: 1) The convolution kernel slides the input image from left to right and from top to bottom according to a certain step size; 2) After the convolution kernel moves a step size, the image pixel value within the range of the convolution kernel is obtained, and then each weight parameter of the convolution kernel is multiplied by the image pixel value at the corresponding position to obtain the feature value of the range; 3) Multiple different convolution kernels slide to obtain a set of eigenvalues, and the feature map is obtained by combining the eigenvalues.

5. The method for generating pictures based on artificial intelligence according to claim 1, characterized in that: In step S4, the specific process of training the CNN model is as follows: S41. Set hyperparameters, loss functions, and optimizers, including learning rate, batch size, number of iterations, regularization parameters, and network structure parameters; S42, input the training set image, and obtain the output image through the CNN model; S43, calculating the loss value through the loss function; S44. According to the loss value, reversely adjust the parameters of each layer in the CNN model; S45, repeating steps S42 to S44 for the purpose of reducing the loss value; S46, inserting validation set image data at intervals to detect the output capability of the CNN model and determine whether the CNN model is overfitting; S47, input the test set to detect the CNN model training results; The meaning of the loss value is: The greater the difference between the output image and the predicted image, the higher the loss value; The smaller the difference between the output image and the predicted image, the smaller the loss value.

6. The method for generating an image based on artificial intelligence according to claim 5, characterized in that: In step S43, the loss function is: L custom =λ1·L MSE +λ2·L SSIM +λ3·L Gragient Among them, λ1, λ2, and λ3 are weights for balancing the contributions of different loss terms, and need to be adjusted according to the training results and data sets; L MSE is the mean square error loss; Where: N is the total number of pixels; y i is the pixel value of the output image at pixel i; is the pixel value of the output image at pixel i; L SSIM is the structural similarity loss; in, To calculate the output image y and the output image The function of the SSIM value between The L SSIM The closer the value is to 1, the more similar the images are; L Gragient is the gradient loss; in, and Represent the gradient operations in the horizontal and vertical directions respectively.

7. The method for generating an image based on artificial intelligence according to claim 6, characterized in that: The specific process of step S5 is: S51, inputting the preset style image and content image into the CNN model respectively; S52,CNN model extracts feature maps of style image and content image through convolution layer and pooling layer respectively; S53, calculating the correlation between the feature graphs of the style image by using a Gram matrix algorithm to represent the style; S54, performing feature reorganization on the feature map to fuse the feature maps of the style and content images; S55, inputting the melted feature map into the automatic encoder to generate an effect map; S56. Post-processing the generated image, including adjusting color, contrast, and sharpening.

8. The method for generating pictures based on artificial intelligence according to claim 1, characterized in that: The specific process of step S6 is: S61. Quality evaluation of renderings: Visual observation of the extent to which style features are transferred to content images and systematic evaluation; S62. Determine the optimization direction based on the analysis results, including fusion features, optimization algorithm and increasing model complexity; S63, iterative optimization: retrain the model based on the adjusted parameters and structure; S64. Convergence judgment: Observe the loss curve and the changing trend of the evaluation indicators during the training process to determine whether the model has converged to the optimal state.

9. The method for generating an image based on artificial intelligence according to claim 8, characterized in that: In step S61, the system evaluation includes: style similarity evaluation, content preservation evaluation and image quality evaluation; The style similarity evaluation and content preservation evaluation are both performed by setting quantitative indicators to evaluate the similarity between the generated effect image and the style image, including the following steps: 1) Extract feature maps of output image and style image; 2) Calculate the Gram matrix of the feature map and obtain the global statistical information of the feature map; 3) Calculate the similarity between the style image and the generated effect image; The image quality assessment: assesses the overall quality of the generated image, including clarity, noise level, and color reproduction, and provides feedback on the image quality.

10. The method for generating an image based on artificial intelligence according to claim 9, characterized in that: The similarity calculation formula is: The feature vectors of the two images are A=[A1,A2,A3,...A n ,] and B=[B1,B2,B3,...B n ,] 1) Calculate the dot product: 2) Calculate the module length: 3) Calculate similarity: Where: θ is the angle between the two vectors.

Citation Information

Cited By

  • Substation defect identification method and device, medium and equipment

    CN119274145A

  • Game image generation system and method based on artificial intelligence enabling

    CN121366215A