An AI-based image style design method and system
By employing an AI-based image style design method and utilizing multimodal large language models and diffusion model training techniques, the problem of strong designer dependence in academic publication cover design has been solved. This approach enables efficient and diverse image style generation, improving image quality and design accuracy.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BEIJING YINGKE CAIXIN TECHNOLOGY CO LTD
- Filing Date
- 2025-04-21
- Publication Date
- 2026-06-02
AI Technical Summary
Cover design in the academic publishing industry mainly relies on manual creation by designers, which is time-consuming and labor-intensive. It is also limited by the designer's personal knowledge and aesthetic level, making it difficult to ensure that the design works accurately and comprehensively convey the connotation of academic works, and it is difficult to efficiently explore and realize diverse design styles.
An AI-based image style design method is adopted. By determining the initial and supplementary fields of the prompt words in the input text, a high-dimensional vector is generated. The model is trained using a multimodal large language model, a diffusion model, and a masking algorithm. Combined with a preset image algorithm, the visual image is processed to ensure that the image fits the text content and improves the image quality.
It improves the fit between visual images and text content, enhances the precision and diversity of image style design, reduces reliance on designers' professional skills, and improves image quality and the ability to present complex information.
Smart Images
Figure CN120411306B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image design technology, and more specifically, to an image style design method and system based on artificial intelligence. Background Technology
[0002] With the rapid development of technology, AI technology is being applied in many fields. Among them, AI image generation technology has been widely used in many industries such as game development, advertising design, and film and television production, helping designers to quickly produce high-quality and diverse images and improve creative efficiency.
[0003] Although AI image generation technology has achieved some degree of popularity, its application in the academic publishing industry remains insufficient. Cover design in academic publishing primarily relies on manual creation by designers. This process is not only time-consuming and labor-intensive but also limited by the designer's individual knowledge and aesthetic abilities, making it difficult to ensure that the design accurately and comprehensively conveys the content of the academic work. Furthermore, due to a lack of targeted tools, designers lack effective technical support during the creative process, hindering their ability to efficiently explore and realize diverse design styles.
[0004] Therefore, it is necessary to design an image style design method and system based on artificial intelligence to solve the problems existing in the current technology. Summary of the Invention
[0005] In view of this, the present invention proposes an image style design method and system based on artificial intelligence, aiming to solve the problem that cover design in the academic publishing industry mainly relies on manual creation by designers, which is not only time-consuming and laborious, but also limited by the designer's personal knowledge and aesthetic level, making it difficult to ensure that the design works accurately and comprehensively convey the connotation of academic works, and difficult to efficiently explore and realize diverse design styles.
[0006] In one aspect, this invention proposes an image style design method based on artificial intelligence, comprising:
[0007] Determine the input text, extract the initial field of prompt words from the input text, generate a high-dimensional vector based on the initial field of prompt words, sample the high-dimensional vector in the probability distribution space, determine the supplementary field of prompt words based on the predicted word occurrence probability, and combine the initial field of prompt words and the supplementary field of prompt words to determine the preliminary text;
[0008] The initial text is processed to determine the target text, and a repetition analysis is performed on the target text to determine the order of each keyword in the target text.
[0009] A multimodal large language model is set up, and the multimodal large language model is trained using a diffusion model and a masking algorithm. A target multimodal large language model is determined, and a visual image of the target text is determined based on the arrangement order and the target multimodal large language model.
[0010] The visual image is processed using a preset image algorithm to determine the target visual image.
[0011] Furthermore, when extracting the initial field of prompt words from the input text, the process includes:
[0012] Based on NLP analysis of the input text, word segmentation, stop word removal, and stemming are performed on the input text to determine the actual text;
[0013] A syntactic analysis model is used to analyze the relationships between words in the actual text and extract all noun phrases. Noun phrases that conform to the syntactic structure are used as the initial field of prompt words for the input text.
[0014] Furthermore, when generating a high-dimensional vector based on the initial field of the prompt word, the process includes:
[0015] The BERT model is used to transform the initial field of the prompt word into a word embedding vector, and self-attention is calculated based on each attention head of each layer of the encoder. The high-dimensional vector is determined based on the self-attention calculation result.
[0016] The self-attention calculation performs a linear transformation on the word embedding vector to determine the value vector, query vector, and key vector. It then determines the dot product of the query vector and the key vector, normalizes the dot product, performs residual connection, and inputs it into a feedforward neural network for feature transformation.
[0017] Furthermore, when sampling in the probability distribution space based on the high-dimensional vector and determining the prompt word supplementary field based on the predicted word occurrence probability, the process includes:
[0018] The high-dimensional vector is input into a fully connected layer, and the output dimension of the fully connected layer is equal to the size of the vocabulary.
[0019] The output of the fully connected layer is converted into a probability distribution by applying the Softmax function to determine the occurrence probability of each word in the vocabulary, and the occurrence probability is used as the occurrence probability of the predicted word.
[0020] The word with the highest predicted probability of occurrence is determined as the supplementary field for the prompt word.
[0021] Furthermore, when performing a repetition analysis on the target text to determine the order of each keyword in the target text, the process includes:
[0022] Extract all keywords from the target text, input all keywords into the keyword entropy model, and determine the keyword entropy value of each keyword;
[0023] Create a sorted sequence from all the distinct keyword entropy values;
[0024] Construct several keyword entropy value sequences from all identical keyword entropy values, extract one keyword entropy value from each sequence, and delete the remaining keyword entropy values.
[0025] The extracted keyword entropy values are added to the sequence to be sorted, and the keywords are sorted in descending order of their entropy values to determine the order of each keyword in the target text.
[0026] Furthermore, when setting up a multimodal large language model, training the multimodal large language model using a diffusion model and a masking algorithm, determining a target multimodal large language model, and determining the visual image of the target text based on the arrangement order and the target multimodal large language model, the process includes:
[0027] Obtain a text-image pairing dataset, divide the text-image pairing dataset into a training set and a test set, use grid search to find the hyperparameters of the multimodal large language model, and establish the multimodal large language model;
[0028] The multimodal large language model is iteratively trained based on the training set, and the iteratively trained multimodal large language model is evaluated based on the test set.
[0029] If the evaluation value of the multimodal large language model after the current iteration is less than the evaluation value of the multimodal large language model after the previous iteration, then the diffusion model and masking algorithm are used to train the multimodal large language model after the current iteration, and the iterative training continues until the preset number of iterations is reached.
[0030] If the evaluation value of the multimodal large language model after the current iteration of training is greater than or equal to the evaluation value of the multimodal large language model after the previous iteration of training, then stop the iterative training and determine the multimodal large language model after the current iteration of training as the target multimodal large language model.
[0031] Each keyword is substituted into the target multimodal large language model according to the stated order to determine the visual image of the target text.
[0032] Furthermore, when training the multimodal large language model after the current iteration using the diffusion model and masking algorithm, the following are included:
[0033] The training image mapped by the training data in the training set is determined, Gaussian noise is added to the training image using a diffusion model, the noisy image is determined, the mean squared error loss is determined based on the training image and the noisy image, and the parameters of the multimodal large language model after the current iteration of training are updated based on the mean squared error loss.
[0034] A binary mask is created for the training images mapped from the training data in the training set, and image regions of the training images are randomly occluded to determine the mask multimodal data;
[0035] The masked multimodal data is substituted into the multimodal large language model trained in the current iteration to predict the pixel information of the image region, the prediction result is determined, the mean absolute error loss of the prediction result and the training data is calculated, and the parameters of the multimodal large language model trained in the current iteration are updated according to the mean absolute error loss.
[0036] Furthermore, when determining the mean squared error loss based on the training image and the noisy image, the process includes:
[0037] Record the noise information of Gaussian noise added to the training image by the diffusion model;
[0038] The noisy image is substituted into the multimodal large language model trained in the current iteration to predict the noise information that the noisy image is restored to the training image. The mean squared error loss is determined based on the noise information and the predicted noise information.
[0039] Furthermore, when performing image processing on the visual image using a preset image algorithm to determine the target visual image, the process includes:
[0040] The visual image is denoised, and the resolution of the denoised visual image is reconstructed based on the first preset algorithm of the image. The contrast of the reconstructed visual image is then adjusted.
[0041] The adjusted visual image is processed by edge processing based on the second preset algorithm of the image to obtain the image to be processed, and the image to be processed is color balanced to determine the target visual image.
[0042] Compared with existing technologies, the beneficial effects of this invention are as follows: By parsing the input text and extracting the initial field of prompt words, the semantics of the input text are fully explored, and the corresponding initial field of prompt words is supplemented, laying the design foundation for the target text. This ensures that the visual image and the target text are well-matched at the content level. By analyzing the order of keywords, the target multimodal large language model can grasp the logical structure of the text. When processing cover illustrations for academic publishing, it can highlight the core based on the order of keywords and reasonably arrange the elements of the image, improving the ability of the visual image to present complex information. By training the multimodal large language model with diffusion model and masking algorithm, the model can deeply learn image features and the mapping relationship between text and image. The diffusion model and masking algorithm enable the multimodal large language model to focus on the connection between image regions and semantics, improving the model's generalization ability and enabling it to adapt to different styles and types of text input, meeting diverse style design needs. By using preset image algorithms to process the generated visual image, the image quality is optimized, improving the image quality of the target visual image and reducing the dependence on the designer's professional skills.
[0043] On the other hand, this application also provides an artificial intelligence-based image style design system, applied to the aforementioned artificial intelligence-based image style design method, including:
[0044] The text extraction module is configured to determine the input text, extract the initial field of prompt words from the input text, generate a high-dimensional vector based on the initial field of prompt words, sample in the probability distribution space based on the high-dimensional vector, determine the supplementary field of prompt words based on the predicted word occurrence probability, and combine the initial field of prompt words and the supplementary field of prompt words to determine the preliminary text.
[0045] The sentence analysis module is configured to process the preliminary text, determine the target text, perform repetition analysis on the target text, and determine the order of each keyword in the target text.
[0046] The image generation module is configured to set a multimodal large language model, train the multimodal large language model using a diffusion model and a masking algorithm, determine a target multimodal large language model, and determine a visual image of the target text based on the arrangement order and the target multimodal large language model.
[0047] The image processing module is configured to perform image processing on the visual image using a preset image algorithm to determine the target visual image.
[0048] It is understandable that the above-mentioned image style design method and system based on artificial intelligence have the same beneficial effects, and will not be elaborated further here. Attached Figure Description
[0049] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:
[0050] Figure 1 A flowchart illustrating an artificial intelligence-based image style design method provided in this embodiment of the invention;
[0051] Figure 2 This is a functional block diagram of an artificial intelligence-based image style design system provided in an embodiment of the present invention. Detailed Implementation
[0052] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the disclosure to those skilled in the art. It should be noted that, unless otherwise specified, embodiments and features in the embodiments of the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.
[0053] In some embodiments of this application, see Figure 1 As shown, an image style design method based on artificial intelligence includes:
[0054] S100: Determine the input text, extract the initial field of prompt words from the input text, generate a high-dimensional vector based on the initial field of prompt words, sample in the probability distribution space based on the high-dimensional vector, determine the supplementary field of prompt words based on the probability of predicted word occurrence, and combine the initial field of prompt words and the supplementary field of prompt words to determine the preliminary text.
[0055] S200: Process the initial text to determine the target text, perform a repetition analysis on the target text, and determine the order of each keyword in the target text.
[0056] S300: Set up a multimodal large language model, and train the multimodal large language model using a diffusion model and a masking algorithm. Determine the target multimodal large language model, and determine the visual image of the target text based on the permutation order and the target multimodal large language model.
[0057] S400: Uses a preset image algorithm to process the visual image and determine the target visual image.
[0058] Specifically, the process involves acquiring user input text, which represents one or more paragraphs of text from which the user intends to generate the target visual image. Natural language processing (NLP) is used to extract initial cue word fields from the input text. These initial fields contain key vocabulary such as shape, color, and theme. NLP is then used to perform a deep semantic understanding of the input text, ensuring that the generated initial cue word fields fully cover the theme and style requirements of the input text. A BERT model is employed to generate high-dimensional vectors from the initial cue word fields and samples them in a probability distribution space. Based on the predicted word occurrence probabilities, supplementary cue word fields are determined, fully mining the latent semantics of the text and enriching the cue word content. Finally, the initial and supplementary fields of the prompt words are combined to determine the preliminary text. For example, if the initial prompt word is "eagle," the supplementary field might be words like "flying." By combining the initial and supplementary fields, the preliminary text becomes more complete, improving the subsequent understanding of the text and enhancing the design precision of the image style. The preliminary text undergoes text processing, including punctuation correction and capitalization standardization. A repetition analysis is performed on the target text to determine the order of each keyword. For example, keywords like "scenery," "painting," "beautiful," "fresh," "nature," "green trees," "blue," "sky," "sunshine," and "pretty" can be used to identify redundant keywords like "beautiful" and "pretty" in the image design. This allows for filtering and avoids the risk of image repetition or inconsistency in the target visual image, thereby improving the design quality of the target visual image. The diffusion model can progressively add noise to the data space and then determine the mean squared error loss to improve the parameters of the multimodal large language model, making the elements such as color, shape, and texture in the target visual image more consistent with the description of the target text. The masking algorithm helps the model understand the mapping relationship between text and image. After training, the target multimodal large language model is finally obtained. Based on the previously determined arrangement order, each keyword is substituted into the target multimodal large language model according to the arrangement order to determine the visual image of the target text, thus improving the accuracy and reliability of image style design.
[0059] It is understandable that visual images generated by the target multimodal large language model contain noise and other interference factors. By using a preset image processing algorithm to process the visual images, noise and other interference factors are eliminated, thus ensuring the details and colors of the target visual images.
[0060] In some embodiments of this application, the extraction of the initial field of prompt words from the input text includes:
[0061] Based on NLP analysis of the input text, perform word segmentation, stop word removal, and stemming on the input text to determine the actual text. Use a syntactic analysis model to parse the relationships between words in the actual text and extract all noun phrases. Take the noun phrases that conform to the syntactic structure as the initial fields of the prompt words for the input text.
[0062] Specifically, based on NLP, text analysis of the input text includes the following three main steps: word segmentation, stop word removal, and stemming. Word segmentation divides the continuous text of the input text into independent words or phrases for subsequent processing. Stop word removal removes common words that do not affect the actual semantics (such as "of", "is", "in", etc.) to improve the effectiveness of the analysis. Stemming reduces semantic redundancy by reducing words to their root forms (e.g., "beautiful" and "beautifully" are unified as "beautiful"). Use a syntactic analysis model to parse the grammatical and semantic relationships between words to ensure that the initial fields of the extracted prompt words accurately reflect the content of the input text. Noun phrases that conform to the syntactic structure can provide clear and logical semantic information for the target multi-modal large language model, enabling the target multi-modal large language model to understand the relationships between these noun phrases. By combining NLP technology and the syntactic analysis model, precise extraction and effective recognition of the input text are achieved. The extraction of the initial fields of the prompt words is based on the syntactic structure, ensuring the semantic value of the extracted content, thereby improving the efficiency of the entire image style design process.
[0063] In some embodiments of the present application, when generating a high-dimensional vector according to the initial fields of the prompt words, it includes: using the BERT model to convert the initial fields of the prompt words into word embedding vectors, performing self-attention calculations according to each attention head of each layer of the encoder, determining the high-dimensional vector based on the self-attention calculation results. Self-attention calculation performs a linear transformation on the word embedding vector to determine the value vector, query vector, and key vector, determines the dot product of the query vector and the key vector, normalizes and performs residual connection on the dot product, and inputs it into a feed-forward neural network for feature transformation.
[0064] Specifically, the BERT (Bidirectional Encoder Representations from Transformers) model is used to transform the initial fields of the prompt words into word embedding vectors. BERT is a pre-trained language model based on the Transformer architecture. It learns the semantic information of the initial fields of the prompt words and generates corresponding word embedding vectors for them. These word embedding vectors represent the position and features of the initial fields of the prompt words in the semantic space. Self-attention is calculated for each attention head of each layer of the encoder. During self-attention calculation, the word embedding vectors undergo a linear transformation to obtain value vectors, query vectors, and key vectors. The dot product reflects the similarity between the query vector and the key vector, i.e., the degree of association between elements at one position and elements at other positions. To avoid excessively large dot product results leading to gradient instability, normalization is performed. After normalization, residual connections are established, enabling the model to effectively learn deeper features. The results after normalization and residual connections are then input into a feedforward neural network for feature transformation. The feedforward neural network is part of the BERT model and can further extract and transform features. Finally, a high-dimensional vector was determined based on the self-attention calculation results, providing a solid semantic foundation for subsequent image style design.
[0065] In some embodiments of this application, when sampling in the probability distribution space based on a high-dimensional vector and determining the supplementary field of prompt words based on the predicted word occurrence probability, the process includes: inputting the high-dimensional vector into a fully connected layer, the output dimension of the fully connected layer being equal to the size of the vocabulary, applying the Softmax function to the output of the fully connected layer, converting the output of the fully connected layer into a probability distribution, determining the occurrence probability of each word in the vocabulary, using the occurrence probability as the predicted word occurrence probability, and determining the word corresponding to the highest predicted word occurrence probability as the supplementary field of prompt words.
[0066] Specifically, a high-dimensional vector is input into a fully connected layer. Fully connected layers are a common layer structure in neural networks. The output dimension of a fully connected layer is equal to the size of the vocabulary, meaning each output node of the fully connected layer corresponds to a word in the vocabulary. The output of the fully connected layer is processed by the Softmax function, a commonly used activation function that transforms the output of the fully connected layer into a probability distribution. Specifically, the Softmax function performs an exponential operation on each output value of the fully connected layer and then normalizes these exponents so that the sum of all output values is 1. At this point, each output value represents the probability of occurrence of the corresponding word in the vocabulary. The vocabulary is a collection of large amounts of text data related to image style design, such as art reviews, design descriptions, and image descriptions. This text data is segmented, and the frequency of each word is counted. Then, the most common words are selected based on their frequency. Choosing the word with the highest predicted probability as the supplementary field for prompt words ensures that the supplementary field conforms to the current semantic context. This avoids introducing irrelevant or incorrect words, thereby improving the quality and accuracy of the initial text.
[0067] In some embodiments of this application, when performing repetition analysis on target text to determine the order of each keyword in the target text, the process includes: extracting all keywords from the target text, inputting all keywords into a keyword entropy model, determining the keyword entropy value of each keyword, establishing a sorting sequence for all different keyword entropy values, constructing several keyword entropy value sequences for all identical keyword entropy values, extracting one keyword entropy value from each keyword entropy value sequence and deleting the remaining keyword entropy values, adding the extracted keyword entropy values to the sorting sequence, and sorting them based on the descending order of keyword entropy values to determine the order of each keyword in the target text.
[0068] Specifically, keyword extraction from the target text is still performed by combining NLP technology and syntactic analysis models, which will not be repeated here. The keyword entropy model is pre-trained and is trained based on different keyword samples and the keyword entropy values matched by the keyword samples. It can determine the keyword entropy value of each keyword.
[0069] In some embodiments of this application, when setting up a multimodal large language model and training it using a diffusion model and a masking algorithm, determining a target multimodal large language model, and determining the visual image of the target text based on the permutation order and the target multimodal large language model, the process includes: acquiring a text-image pairing dataset, dividing the text-image pairing dataset into a training set and a test set, using a grid search to find the hyperparameters of the multimodal large language model, establishing the multimodal large language model, iteratively training the multimodal large language model based on the training set, and evaluating the iteratively trained multimodal large language model based on the test set. If the current iterative training... If the evaluation value of the multimodal large language model is less than the evaluation value of the multimodal large language model after the previous iteration of training, then the multimodal large language model after the current iteration of training is trained using the diffusion model and the masking algorithm, and iterative training continues until the preset number of iterations is reached. If the evaluation value of the multimodal large language model after the current iteration of training is greater than or equal to the evaluation value of the multimodal large language model after the previous iteration of training, then iterative training is stopped, and the multimodal large language model after the current iteration of training is determined as the target multimodal large language model. Each keyword is substituted into the target multimodal large language model according to the order of arrangement to determine the visual image of the target text.
[0070] Specifically, the text-image pairing dataset contains various mapping relationships between keywords and images. Typically, 70%-80% of the data is used as the training set, and the remainder as the test set to improve the generalization ability of the multimodal large language model. The training set is used to train the multimodal large language model, while the test set is used to evaluate the performance of the trained model. Grid search searches for the hyperparameters of the multimodal large language model in the parameter space to build the model. The multimodal large language model includes an input encoding layer, modality alignment, and fusion layers, designed to capture complex relationships in the data. During iterative training of the multimodal large language model using data from the training set, the model attempts to learn patterns and relationships in the data during each iteration to improve its prediction or image processing capabilities. After each training iteration, the model is evaluated using data from the test set. Evaluation metrics include accuracy, loss function value, and recall, used to measure the model's performance.
[0071] Understandably, if the evaluation value of the multimodal language model after the current iteration is lower than that after the previous iteration, it indicates a performance degradation. In this case, a diffusion model and masking algorithm are used to train the multimodal language model after the current iteration, and iterative training continues until the preset number of iterations is reached. This helps the model stably approach the global style design. If the evaluation value of the multimodal language model after the current iteration is greater than or equal to that after the previous iteration, it indicates that the model performance has improved or remained stable. At this point, iterative training is stopped, and the multimodal language model trained in the current iteration is used as the target multimodal language model. Each keyword is then substituted into the target multimodal language model according to the order of arrangement to determine the visual image of the target text, improving the accuracy of keyword matching and visual image style design, and ensuring the image quality of the visual image.
[0072] In some embodiments of this application, when training the multimodal large language model after the current iteration of training using a diffusion model and a masking algorithm, the process includes: determining the training image mapped by the training data in the training set; adding Gaussian noise to the training image using a diffusion model; determining the noisy image; determining the mean squared error loss based on the training image and the noisy image; updating the parameters of the multimodal large language model after the current iteration of training based on the mean squared error loss; creating a binary mask for the training image mapped by the training data in the training set and randomly occluding image regions of the training image; determining the masked multimodal data; substituting the masked multimodal data into the multimodal large language model after the current iteration of training; predicting the pixel information of the image region; determining the prediction result; calculating the mean absolute error loss of the prediction result and the training data; and updating the parameters of the multimodal large language model after the current iteration of training based on the mean absolute error loss.
[0073] In some embodiments of this application, determining the mean squared error loss based on the training image and the noise image includes: recording the noise information of the diffusion model adding Gaussian noise to the training image, substituting the noise image into the multimodal large language model after the current iteration of training, predicting the noise information of the noise image to recover the training image, and determining the mean squared error loss based on the noise information and the predicted noise information.
[0074] Specifically, by adding Gaussian noise to the training image to obtain a noisy image, the model predicts and recovers the noise information needed, and updates the parameters with mean squared error loss. This enables the model to learn the distribution characteristics of the image data, thereby enhancing the model's ability to reconstruct images from noise. While adding Gaussian noise to the training image, a binary mask is created and image regions of the training image are randomly occluded. The model then predicts the pixel information of the occluded regions, and updates the parameters with mean absolute error loss. The process of obtaining mean squared error loss and mean absolute error loss is redundant and mature, and will not be described in detail here. The parameters of the multimodal large language model after the current iteration of training are updated with mean absolute error loss and mean squared error loss, which prompts the model to learn the contextual information and semantic relationships of the image, improves the understanding of the overall structure and features of the keyword mapping image, and thus improves the accuracy of the visual image.
[0075] In some embodiments of this application, when performing image processing on a visual image using a preset image algorithm to determine a target visual image, the process includes: denoising the visual image, reconstructing the resolution of the denoised visual image based on a first preset image algorithm, adjusting the contrast of the reconstructed visual image, performing edge processing on the adjusted visual image based on a second preset image algorithm to obtain an image to be processed, and performing color equalization on the image to be processed to determine the target visual image.
[0076] Specifically, the first preset image algorithm represents bilinear interpolation and bicubic interpolation, one of which can be selected according to actual needs. The second preset image algorithm represents the Sobel operator and the Canny edge detection algorithm, one of which can be selected according to actual needs. By removing noise from the visual image, the clarity and image quality of the visual image are improved, providing a foundation for subsequent processing. The first preset image algorithm is used for resolution reconstruction, making the image clearer and more delicate, thereby enhancing the visual effect of the image. The contrast of the reconstructed visual image is adjusted to highlight the details and features of the image, improving the visual expressiveness of the image. The second preset image algorithm is used to detect and enhance the edges of the image, making the outlines of people or objects in the image clearer. After denoising, resolution reconstruction, contrast adjustment, and edge processing, the image quality of the image to be processed is improved. Based on the image to be processed, color equalization is performed to balance the color distribution in the image to eliminate color deviations. While meeting the style design of the keywords, the color quality and visual effect of the target visual image are ensured.
[0077] In summary, the beneficial effects of this invention are as follows: By parsing the input text and extracting the initial field of prompt words, the semantics of the input text are fully explored, and the corresponding initial field of prompt words is supplemented, laying the design foundation for the target text. This ensures that the visual image and the target text are well-matched at the content level. By analyzing the order of keywords, the target multimodal large language model can grasp the logical structure of the text. When processing cover illustrations for academic publishing, it can highlight the core based on the order of keywords and reasonably arrange the elements of the image, improving the ability of the visual image to present complex information. By training the multimodal large language model with diffusion model and masking algorithm, the model can deeply learn image features and the mapping relationship between text and image. The diffusion model and masking algorithm enable the multimodal large language model to focus on the connection between image regions and semantics, improving the model's generalization ability and enabling it to adapt to different styles and types of text input, meeting diverse style design needs. By using preset image algorithms to process the generated visual image, the image quality is optimized, improving the image quality of the target visual image and reducing the dependence on the designer's professional skills.
[0078] In another preferred embodiment based on the above embodiments, see [reference] Figure 2 As shown, this embodiment provides an AI-based image style design system, applied to the aforementioned AI-based image style design method, including:
[0079] The text extraction module is configured to determine the input text, extract the initial field of prompt words from the input text, generate a high-dimensional vector based on the initial field of prompt words, sample in the probability distribution space based on the high-dimensional vector, determine the supplementary field of prompt words based on the predicted word occurrence probability, and combine the initial field of prompt words and the supplementary field of prompt words to determine the preliminary text.
[0080] The sentence analysis module is configured to process the initial text, identify the target text, perform repetition analysis on the target text, and determine the order of each keyword in the target text.
[0081] The image generation module is configured to set up a multimodal large language model, train the multimodal large language model using a diffusion model and a masking algorithm, determine the target multimodal large language model, and determine the visual image of the target text based on the permutation order and the target multimodal large language model.
[0082] The image processing module is configured to use a preset image algorithm to process the visual image and determine the target visual image.
[0083] Specifically, by parsing the input text and extracting the initial fields of prompt words, the semantics of the input text are fully explored, and the corresponding initial fields of prompt words are supplemented, laying the design foundation for the target text. This ensures that the visual image and the target text are well-matched at the content level. By analyzing the order of keywords, the target multimodal large language model can grasp the logical structure of the text. When processing cover illustrations for academic publishing, it can highlight the core based on the order of keywords and reasonably arrange image elements, improving the ability of the visual image to present complex information. By training the multimodal large language model with diffusion model and masking algorithm, the model can deeply learn image features and the mapping relationship between text and image. The diffusion model and masking algorithm enable the multimodal large language model to focus on the connection between image regions and semantics, improving the model's generalization ability and enabling it to adapt to different styles and types of text input, meeting diverse style design needs. The generated visual image is processed using preset image algorithms to optimize image quality, improving the image quality of the target visual image and reducing the dependence on the designer's professional skills.
[0084] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program goods. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program goods embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0085] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program goods according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0086] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0087] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0088] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.
Claims
1. An image style design method based on artificial intelligence, characterized in that, include: Determine the input text, extract the initial field of prompt words from the input text, generate a high-dimensional vector based on the initial field of prompt words, sample the high-dimensional vector in the probability distribution space, determine the supplementary field of prompt words based on the predicted word occurrence probability, and combine the initial field of prompt words and the supplementary field of prompt words to determine the preliminary text; The initial text is processed to determine the target text, and a repetition analysis is performed on the target text to determine the order of each keyword in the target text. A multimodal large language model is set up, and the multimodal large language model is trained using a diffusion model and a masking algorithm. A target multimodal large language model is determined, and a visual image of the target text is determined based on the arrangement order and the target multimodal large language model. The visual image is processed using a preset image algorithm to determine the target visual image; When performing a repeatability analysis on the target text to determine the order of each keyword in the target text, the process includes: Extract all keywords from the target text, input all keywords into the keyword entropy model, and determine the keyword entropy value of each keyword; Create a sorted sequence from all the distinct keyword entropy values; Construct several keyword entropy value sequences from all identical keyword entropy values, extract one keyword entropy value from each sequence, and delete the remaining keyword entropy values. The extracted keyword entropy values are added to the sequence to be sorted, and the keywords are sorted in descending order of their entropy values to determine the order of each keyword in the target text. When setting up a multimodal large language model, training the multimodal large language model using a diffusion model and a masking algorithm, determining a target multimodal large language model, and determining the visual image of the target text based on the arrangement order and the target multimodal large language model, the process includes: Obtain a text-image pairing dataset, divide the text-image pairing dataset into a training set and a test set, use grid search to find the hyperparameters of the multimodal large language model, and establish the multimodal large language model; The multimodal large language model is iteratively trained based on the training set, and the iteratively trained multimodal large language model is evaluated based on the test set. If the evaluation value of the multimodal large language model after the current iteration is less than the evaluation value of the multimodal large language model after the previous iteration, then the diffusion model and masking algorithm are used to train the multimodal large language model after the current iteration, and the iterative training continues until the preset number of iterations is reached. If the evaluation value of the multimodal large language model after the current iteration of training is greater than or equal to the evaluation value of the multimodal large language model after the previous iteration of training, then stop the iterative training and determine the multimodal large language model after the current iteration of training as the target multimodal large language model. Each keyword is substituted into the target multimodal large language model according to the arrangement order to determine the visual image of the target text; When training the multimodal large language model after the current iteration using the diffusion model and masking algorithm, the following are included: The training image mapped by the training data in the training set is determined, Gaussian noise is added to the training image using a diffusion model, the noisy image is determined, the mean squared error loss is determined based on the training image and the noisy image, and the parameters of the multimodal large language model after the current iteration of training are updated based on the mean squared error loss. A binary mask is created for the training images mapped from the training data in the training set, and image regions of the training images are randomly occluded to determine the mask multimodal data; The masked multimodal data is substituted into the multimodal large language model trained in the current iteration to predict the pixel information of the image region, the prediction result is determined, the mean absolute error loss of the prediction result and the training data is calculated, and the parameters of the multimodal large language model trained in the current iteration are updated according to the mean absolute error loss. When performing image processing on the visual image using a preset image algorithm to determine the target visual image, the process includes: The visual image is denoised, and the resolution of the denoised visual image is reconstructed based on the first preset algorithm of the image. The contrast of the reconstructed visual image is then adjusted. The adjusted visual image is processed by edge processing based on the second preset algorithm of the image to obtain the image to be processed, and the image to be processed is color balanced to determine the target visual image.
2. The image style design method based on artificial intelligence according to claim 1, characterized in that, When extracting the initial field of prompt words from the input text, the following is included: Based on NLP analysis of the input text, word segmentation, stop word removal, and stemming are performed on the input text to determine the actual text; A syntactic analysis model is used to analyze the relationships between words in the actual text and extract all noun phrases. Noun phrases that conform to the syntactic structure are used as the initial field of prompt words for the input text.
3. The image style design method based on artificial intelligence according to claim 2, characterized in that, When generating a high-dimensional vector based on the initial field of the prompt word, the process includes: The BERT model is used to transform the initial field of the prompt word into a word embedding vector, and self-attention is calculated based on each attention head of each layer of the encoder. The high-dimensional vector is determined based on the self-attention calculation result. The self-attention calculation performs a linear transformation on the word embedding vector to determine the value vector, query vector, and key vector. It then determines the dot product of the query vector and the key vector, normalizes the dot product, performs residual connection, and inputs it into a feedforward neural network for feature transformation.
4. The image style design method based on artificial intelligence according to claim 3, characterized in that, When sampling in the probability distribution space based on the high-dimensional vector and determining the prompt word supplementary field based on the predicted word occurrence probability, the process includes: The high-dimensional vector is input into a fully connected layer, and the output dimension of the fully connected layer is equal to the size of the vocabulary. The output of the fully connected layer is converted into a probability distribution by applying the Softmax function to determine the occurrence probability of each word in the vocabulary, and the occurrence probability is used as the occurrence probability of the predicted word. The word with the highest predicted probability of occurrence is determined as the supplementary field for the prompt word.
5. The image style design method based on artificial intelligence according to claim 4, characterized in that, Determining the mean squared error loss based on the training image and the noisy image includes: Record the noise information of Gaussian noise added to the training image by the diffusion model; The noisy image is substituted into the multimodal large language model trained in the current iteration to predict the noise information that the noisy image is restored to the training image. The mean squared error loss is determined based on the noise information and the predicted noise information.
6. An AI-based image style design system, applied to the AI-based image style design method as described in any one of claims 1-5, characterized in that, include: The text extraction module is configured to determine the input text, extract the initial field of prompt words from the input text, generate a high-dimensional vector based on the initial field of prompt words, sample in the probability distribution space based on the high-dimensional vector, determine the supplementary field of prompt words based on the predicted word occurrence probability, and combine the initial field of prompt words and the supplementary field of prompt words to determine the preliminary text. The sentence analysis module is configured to process the preliminary text, determine the target text, perform repetition analysis on the target text, and determine the order of each keyword in the target text. The image generation module is configured to set a multimodal large language model, train the multimodal large language model using a diffusion model and a masking algorithm, determine a target multimodal large language model, and determine a visual image of the target text based on the arrangement order and the target multimodal large language model. The image processing module is configured to perform image processing on the visual image using a preset image algorithm to determine the target visual image; When performing a repeatability analysis on the target text to determine the order of each keyword in the target text, the process includes: Extract all keywords from the target text, input all keywords into the keyword entropy model, and determine the keyword entropy value of each keyword; Create a sorted sequence from all the distinct keyword entropy values; Construct several keyword entropy value sequences from all identical keyword entropy values, extract one keyword entropy value from each sequence, and delete the remaining keyword entropy values. The extracted keyword entropy values are added to the sequence to be sorted, and the keywords are sorted in descending order of their entropy values to determine the order of each keyword in the target text. When setting up a multimodal large language model, training the multimodal large language model using a diffusion model and a masking algorithm, determining a target multimodal large language model, and determining the visual image of the target text based on the arrangement order and the target multimodal large language model, the process includes: Obtain a text-image pairing dataset, divide the text-image pairing dataset into a training set and a test set, use grid search to find the hyperparameters of the multimodal large language model, and establish the multimodal large language model; The multimodal large language model is iteratively trained based on the training set, and the iteratively trained multimodal large language model is evaluated based on the test set. If the evaluation value of the multimodal large language model after the current iteration is less than the evaluation value of the multimodal large language model after the previous iteration, then the diffusion model and masking algorithm are used to train the multimodal large language model after the current iteration, and the iterative training continues until the preset number of iterations is reached. If the evaluation value of the multimodal large language model after the current iteration of training is greater than or equal to the evaluation value of the multimodal large language model after the previous iteration of training, then stop the iterative training and determine the multimodal large language model after the current iteration of training as the target multimodal large language model. Each keyword is substituted into the target multimodal large language model according to the arrangement order to determine the visual image of the target text; When training the multimodal large language model after the current iteration using the diffusion model and masking algorithm, the following are included: The training image mapped by the training data in the training set is determined, Gaussian noise is added to the training image using a diffusion model, the noisy image is determined, the mean squared error loss is determined based on the training image and the noisy image, and the parameters of the multimodal large language model after the current iteration of training are updated based on the mean squared error loss. A binary mask is created for the training images mapped from the training data in the training set, and image regions of the training images are randomly occluded to determine the mask multimodal data; The masked multimodal data is substituted into the multimodal large language model trained in the current iteration to predict the pixel information of the image region, the prediction result is determined, the mean absolute error loss of the prediction result and the training data is calculated, and the parameters of the multimodal large language model trained in the current iteration are updated according to the mean absolute error loss. When performing image processing on the visual image using a preset image algorithm to determine the target visual image, the process includes: The visual image is denoised, and the resolution of the denoised visual image is reconstructed based on the first preset algorithm of the image. The contrast of the reconstructed visual image is then adjusted. The adjusted visual image is processed by edge processing based on the second preset algorithm of the image to obtain the image to be processed, and the image to be processed is color balanced to determine the target visual image.