Painting processing method and device, storage medium and electronic equipment

By obtaining the user's painting style information and initial prompt words, determining the painting style mode and optimizing the prompt words, and using the painting large language model to generate target painting, the problem of the painting effect in the existing technology is solved, and higher quality is achieved And more suitable for user expectations of paintings.

CN118918199AActive Publication Date: 2024-11-08SHENZHEN QIHOO INTELLIGENT TECH CO LTD

Patent Information

Application Number
CN202410612576.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-05-16
Publication Date
2024-11-08
Estimated Expiration
2044-05-16

AI Technical Summary

Technical Problem

The overall effect of existing large-scale art drawing technology generated by paintings is often far from users' expectations, and it is difficult to accurately capture the user's painting style and creativity.

Method used

By obtaining the user's initial painting prompt word, determine the painting style mode for the prompt word, and use the painting large language model to perform textual drawing processing in this mode to generate the target painting. The method includes obtaining user's painting style interest information, historical painting style and attribute information, generating drawing scene description data based on this information, performing style pattern matching and style guidance information enhancement, and optimizing initial prompt words to guide the model to generate more suitable for user expectations. painting.

Benefits of technology

The painting effect generated in the painting style mode is better, the overall effect is more in line with the user's painting expectations, the generated image quality is higher, and it can express the user's creativity and intention more accurately.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118918199A_ABST
    Figure CN118918199A_ABST
Patent Text Reader

Abstract

The embodiment of the invention discloses a drawing processing method and device, a storage medium and electronic device.The method comprises the steps that an initial drawing prompt word input by a user is obtained, and a drawing style mode corresponding to the initial drawing prompt word is determined, and under the painting style mode, based on the initial painting cue word, through the painting big language model, performing text drawing processing to obtain a target painting, and based on the target painting, performing painting display processing to the user.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a painting processing method, device, storage medium and electronic device. Background Art

[0002] With the development of computer technology, electronic devices such as smart watches and smart bracelets are rapidly becoming popular. Electronic devices usually have the function of intelligent voice question and answer, and support the function of waking up the device through voice, including making calls, setting alarms, opening applications, playing music stories, etc., and can also perform painting processing. Summary of the invention

[0003] The present application provides a method, device, storage medium and electronic device for processing a painting, and the technical solution is as follows:

[0004] In a first aspect, an embodiment of the present application provides a painting processing method, the method comprising:

[0005] Acquire an initial painting prompt word input by a user, determine a painting style mode corresponding to the initial painting prompt word, and perform text-based image processing based on the initial painting prompt word through the painting large language model in the painting style mode to obtain a target painting;

[0006] A painting display process is performed to the user based on the target painting.

[0007] In a feasible implementation manner, determining the painting style mode corresponding to the initial painting prompt word includes:

[0008] Determining painting scene description data for the user;

[0009] The painting scene description data is subjected to style pattern matching processing to obtain a painting style pattern corresponding to the initial painting prompt word.

[0010] In a feasible implementation manner, determining the painting scene description data for the user includes:

[0011] Acquiring the painting style interest information of the user, and generating painting scene description data for the user based on the painting style interest information; and / or,

[0012] A painting theme for the user is determined based on the initial painting prompt word, user attribute information and a historical painting style for the user are determined, and painting scene description data for the user is generated based on the painting theme, the user attribute information and the historical painting style.

[0013] In a feasible implementation manner, performing style pattern matching processing on the painting scene description data to obtain a painting style pattern corresponding to the initial painting prompt word includes:

[0014] Inputting the painting scene description data into a painting style recommendation model to perform painting style recommendation processing, and outputting a painting style model corresponding to the initial painting prompt word;

[0015] The painting style recommendation model is obtained by training an initial painting style recommendation model using sample painting scene description data that has been annotated with painting style pattern labels.

[0016] In a feasible implementation manner, the step of obtaining a target painting by performing text image processing based on the initial painting prompt word through the painting large language model in the painting style mode includes:

[0017] Determine the painting style guidance information corresponding to the painting style mode, and perform painting style guidance enhancement processing on the initial painting prompt word based on the painting style guidance information to obtain a target painting prompt word;

[0018] In the painting style mode, the target painting prompt word is input into the painting language model for text-based image processing to obtain the target painting.

[0019] In a feasible implementation manner, the determining the painting style guidance information corresponding to the painting style mode, and performing painting style guidance enhancement processing on the initial painting prompt word based on the painting style guidance information to obtain a target painting prompt word, includes:

[0020] Determine a painting style positive guiding word and / or a painting style negative guiding word corresponding to the painting style mode;

[0021] Based on the painting style positive guiding words and / or the painting style negative guiding words, the initial painting prompt words are subjected to prompt word style optimization processing to obtain target painting prompt words.

[0022] In a feasible implementation manner, the method further includes:

[0023] Performing prompt sensitivity detection on the initial drawing prompt word to obtain prompt sensitivity information, and performing prompt correction processing on the initial drawing prompt word based on the prompt sensitivity information to obtain the initial drawing prompt word after prompt correction processing; and / or

[0024] A painting sensitivity detection is performed on the target painting to obtain painting sensitivity information, and a painting correction process is performed on the target painting based on the painting sensitivity information to obtain the target painting after the painting correction process.

[0025] In a feasible implementation manner, the method further includes:

[0026] The basic large language model is used to create an initial painting large language model for the painting generation scenario;

[0027] Obtaining sample painting prompt words, and marking painting labels on the sample painting prompt words;

[0028] The sample painting prompt words are used to perform model transfer training on the initial painting large language model,

[0029] During the model transfer training process, a sample painting style mode corresponding to the sample painting prompt word is determined, and a predicted painting is obtained by performing Wensheng graph processing based on the sample painting prompt word through the painting large language model under the sample painting style mode, and model parameters of the initial painting large language model are adjusted based on the predicted painting and the painting label to obtain a painting large language model after transfer training.

[0030] In a second aspect, an embodiment of the present application provides a painting processing device, the device comprising:

[0031] A Wenshengtu module is used to obtain an initial painting prompt word input by a user, determine a painting style mode corresponding to the initial painting prompt word, and perform Wenshengtu processing based on the initial painting prompt word through the painting large language model in the painting style mode to obtain a target painting;

[0032] The painting display module is used to display the target painting to the user.

[0033] In a feasible implementation manner, the Wensheng diagram module includes:

[0034] A description determining unit, used to determine the painting scene description data for the user;

[0035] A painting processing unit is used to perform style pattern matching processing on the painting scene description data to obtain a painting style pattern corresponding to the initial painting prompt word.

[0036] In a feasible implementation manner, the description determining unit is used to:

[0037] Acquiring the painting style interest information of the user, and generating painting scene description data for the user based on the painting style interest information; and / or,

[0038] A painting theme for the user is determined based on the initial painting prompt word, user attribute information and a historical painting style for the user are determined, and painting scene description data for the user is generated based on the painting theme, the user attribute information and the historical painting style.

[0039] In a feasible implementation manner, the painting processing unit is used to:

[0040] Inputting the painting scene description data into a painting style recommendation model to perform painting style recommendation processing, and outputting a painting style model corresponding to the initial painting prompt word;

[0041] The painting style recommendation model is obtained by training an initial painting style recommendation model using sample painting scene description data that has been annotated with painting style pattern labels.

[0042] In a feasible implementation manner, the Wensheng diagram module is used to:

[0043] a prompt enhancement unit, configured to determine the painting style guidance information corresponding to the painting style mode, and perform painting style prompt enhancement processing on the initial painting prompt word based on the painting style guidance information to obtain a target painting prompt word;

[0044] In the painting style mode, the target painting prompt word is input into the painting language model for text-based image processing to obtain the target painting.

[0045] In a feasible implementation manner, the prompt enhancement unit is used to:

[0046] Determine a painting style positive guiding word and / or a painting style negative guiding word corresponding to the painting style mode;

[0047] Based on the painting style positive guiding words and / or the painting style negative guiding words, the initial painting prompt words are subjected to prompt word style optimization processing to obtain target painting prompt words.

[0048] In a feasible embodiment, the device is also used for:

[0049] Performing prompt sensitivity detection on the initial drawing prompt word to obtain prompt sensitivity information, and performing prompt correction processing on the initial drawing prompt word based on the prompt sensitivity information to obtain the initial drawing prompt word after prompt correction processing; and / or

[0050] A painting sensitivity detection is performed on the target painting to obtain painting sensitivity information, and a painting correction process is performed on the target painting based on the painting sensitivity information to obtain the target painting after the painting correction process.

[0051] In a feasible embodiment, the device is also used for:

[0052] The basic large language model is used to create an initial painting large language model for the painting generation scenario;

[0053] Obtaining sample painting prompt words, and marking painting labels on the sample painting prompt words;

[0054] The sample painting prompt words are used to perform model transfer training on the initial painting large language model. During the model transfer training process, a sample painting style mode corresponding to the sample painting prompt words is determined. Under the sample painting style mode, the sample painting prompt words are used to perform Wensheng graph processing through the painting large language model to obtain a predicted painting. Based on the predicted painting and the painting label, model parameters of the initial painting large language model are adjusted to obtain a painting large language model after transfer training.

[0055] In a third aspect, an embodiment of the present application provides a computer storage medium, wherein the computer storage medium stores a plurality of instructions, wherein the instructions are suitable for being loaded by a processor and executing the above-mentioned method steps.

[0056] In a fourth aspect, an embodiment of the present application provides an electronic device, which may include: a processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the above-mentioned method steps.

[0057] The beneficial effects brought about by the technical solutions provided by some embodiments of the present application include at least:

[0058] In one or more embodiments of the present application, by obtaining the initial painting prompt word input by the user, the painting style mode corresponding to the initial painting prompt word is determined, and the target painting is obtained by performing text-generated image processing based on the initial painting prompt word through the painting large language model in the painting style mode, rather than directly indicating the initial painting prompt word to instruct the large model to perform text-generated image. In this way, the painting large language model is guided in advance to fully refer to the style characteristics of the painting style mode, and the painting processing flow is optimized, so that the overall effect of generating a painting under the painting style mode can be better, more in line with the user's painting expectations, and a target painting with better quality can be generated for display to the user. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0060] Figure 1 It is a flowchart of a painting processing method provided in an embodiment of the present application;

[0061] Figure 2 is a schematic diagram of a text map interface provided in an embodiment of the present application;

[0062] Figure 3 It is a schematic diagram of a scene of Wensheng graph processing provided by an embodiment of the present application;

[0063] Figure 4 It is a flowchart of another embodiment of a painting processing method provided in an embodiment of the present application;

[0064] Figure 5 It is a schematic diagram of a model training process of a large painting language model provided in an embodiment of the present application;

[0065] Figure 6 is a structural schematic diagram of a painting processing device provided in an embodiment of the present application;

[0066] Figure 7 It is a structural schematic diagram of a Vincent diagram provided in an embodiment of the present application;

[0067] Figure 8 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application;

[0068] Fig. 9 It is a schematic diagram of the structure of the operating system and user space provided in the embodiment of the present application;

[0069] Fig.10 yes Fig. 9 The architecture diagram of the Android operating system;

[0070] Fig.11 yes Fig. 9 Architecture diagram of the IOS operating system. DETAILED DESCRIPTION

[0071] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0072] In the description of the present application, it should be understood that the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In the description of the present application, it should be noted that, unless otherwise clearly specified and limited, "including" and "having" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally also includes other steps or units inherent to these processes, methods, products or devices. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood in specific circumstances. In addition, in the description of the present application, unless otherwise specified, "multiple" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B, which can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects before and after are an "or" relationship.

[0073] In related technologies, with the development of big models and artificial intelligence (AIGC) technology, AI technology is increasingly used in bank marketing, poster generation, web page background image generation and even PPT background image generation. However, there are some problems in the current big model-based text image technology, mainly manifested in that the overall effect of the generated painting is often far from the overall effect expected by the user.

[0074] The present application is described in detail below with reference to specific embodiments.

[0075] In one embodiment, Figure 1 As shown, a painting processing method is proposed, which can be implemented by a computer program and can be run on a painting processing device based on the von Neumann system. The computer program can be integrated in an application or run as an independent tool application. The painting processing device can be an electronic device, including but not limited to: a personal computer, a tablet computer, a handheld device, a vehicle-mounted device, a wearable device, a computing device or other processing device connected to a wireless modem, etc. In different networks, terminal devices can be called different names, such as: user equipment, access terminal, user unit, user station, mobile station, mobile station, remote station, remote terminal, mobile device, user terminal, terminal, wireless communication device, user agent or user device, cellular phone, cordless phone, 5G network or terminal device in future evolution network, etc.; electronic device can also be a service platform.

[0076] Specifically, the painting processing method includes:

[0077] S102: Obtaining the initial drawing prompt word input by the user;

[0078] The initial painting prompt words are the description of the painting generated by the user input. The initial painting prompt words include the description information of the painting that the user expects to generate, such as the image style description, image content description, image specification description and other description information of the expected image. The initial painting prompt words are used to instruct the smart object service to generate painting content that meets the user's expectations.

[0079] The initial painting prompt is usually a descriptive text input by the user, and a target painting with corresponding content can be automatically generated based on the initial painting prompt with or without a sample image (the user can selectively input the sample image). That is, the user can choose whether to input the sample image;

[0080] The example image is a painting reference provided for the painting large language model to generate corresponding content based on the painting prompt words. It is used to instruct the painting large language model to generate a target painting that matches the example painting characteristics and corresponds to the painting prompt words with reference to the example painting characteristics of the example image. In this way, the target painting can be highly consistent with the example image in terms of example painting characteristics such as image depth and layout.

[0081] Illustratively, the electronic device obtains an initial painting prompt word input by a user. The initial painting prompt word may carry an example image when the user inputs an example image. A painting style mode corresponding to the initial painting prompt word is subsequently determined. In the painting style mode, a target painting is obtained by performing text-based image processing based on the initial painting prompt word with reference to the example image through a large painting language model.

[0082] Illustratively, the initial drawing prompt word may not carry an example image when the user does not input an example image.

[0083] The initial drawing prompt word can be a piece of text input by the user, or can be a text converted from the user's voice. Figure 2 As shown, Figure 2 It is a schematic diagram of a text map interface. Figure 2 In the interface shown, the user can choose to manually input or voice input the initial drawing prompt word; this specification does not impose any limitation on the form of obtaining the initial drawing prompt word and the content of the initial drawing prompt word.

[0084] S104: determining a painting style mode corresponding to the initial painting prompt word, and performing text-based image processing based on the initial painting prompt word through the painting large language model in the painting style mode to obtain a target painting;

[0085] The painting style mode may be one or more of a plurality of pre-configured reference painting style modes, such as a universal painting style mode, a two-dimensional painting style mode, a science fiction painting style mode, a traditional Chinese painting style mode, and the like.

[0086] For example, the user may pre-select a painting style mode for this painting, and then determine the painting style mode corresponding to the initial painting prompt word;

[0087] Exemplarily, style analysis can be performed based on the initial painting prompt words and / or sample images, and the initial painting prompt words and / or sample images provided by the user can be analyzed. These initial painting prompt words and / or sample images may contain painting features related to themes, descriptions, painting details, scene descriptions, and picture ratios. By analyzing the painting features, the painting style mode desired by the user can be further clarified, rather than directly indicating that the initial painting prompt words instruct the large model to perform text drawing. In this way, the painting large language model is guided into the painting style mode in advance, which can make the overall effect of generating paintings in this painting style mode better.

[0088] Furthermore, a style parsing model is pre-trained based on the machine learning model, and the initial painting prompt words and / or sample images are input into the style parsing model to extract painting features. The painting features are analyzed by the style parsing model to determine the painting style pattern, and the painting style pattern is output.

[0089] Optionally, after determining the painting style mode, the painting language model is controlled to enter the painting style mode, and the target painting is obtained by performing text-based image processing based on the initial painting prompt words.

[0090] Exemplarily, a painting language model that can support various painting style texts is pre-trained for different painting style modes. After the painting style mode is determined, the style mode prompt words corresponding to the painting style mode are input into the painting language model. Then, the painting language model of the current painting style mode can be selected to control the painting language model to enter the current painting style mode, and the text is processed based on the initial painting prompt words in the current painting style mode to obtain the target painting.

[0091] For example, Figure 3 As shown, Figure 3 It is a scene diagram of Wensheng picture processing. After the electronic device determines the painting style mode, it can be displayed on the user input interface as follows Figure 3 The text of the painting style mode shown, such as Figure 3In the "Painting Style: Chinese Style (Mode)", further, the user can also adjust the painting style mode, such as adjusting from Chinese style (mode) to realistic style. After the user determines the current painting style mode and enters the initial painting prompt word "draw a monkey catching the moon", he can start painting, such as clicking the start painting button, and then controlling the painting language model to enter the painting style mode. In this painting style mode, the text image is processed based on the initial painting prompt word to obtain the target painting that matches the painting style mode.

[0092] In a feasible implementation, the drawing large language model may directly use the basic large language model.

[0093] In a feasible implementation, the painting large language model can be obtained by training the model for the painting scene based on the basic large language model. The basic large language model (LLM) is an artificial intelligence content generation model that aims to understand and generate human language. The basic large language model is trained on a large amount of data and can perform a wide range of tasks, including text summarization, translation, sentiment analysis, etc.

[0094] Optionally, the basic large language model can be an AIGC model on the public network;

[0095] Optionally, you can obtain a trained basic large language model and adapt the basic large language model to the painting generation scenario. First, obtain the basic large language model to create an initial painting large language model, and obtain sample painting prompt words in the new painting scenario. Use the sample painting prompt words to perform model fine-tuning training on the initial painting large language model. After the model fine-tuning training is completed, obtain the painting large language model adapted to the painting generation scenario.

[0096] S106: Perform painting display processing to the user based on the target painting.

[0097] In some embodiments, the user terminal has an input function, which can be used for user input of data. For example, typing input, voice input, etc. The service platform can be centralized or distributed. In some embodiments, the user terminal can send the initial painting prompt word input by the user and the determined painting style mode to the service platform to instruct the service platform to perform text image processing based on the initial painting prompt word through the painting large language model in the painting style mode to obtain the target painting, and then the user terminal obtains the target painting and displays the target painting to the user;

[0098] In some embodiments, a user terminal (such as a smart terminal) receives an initial painting prompt word input by a user, determines a painting style mode, performs text-based image processing based on the initial painting prompt word through a painting language model in the painting style mode to obtain a target painting, and displays the target painting to the user;

[0099] In a feasible implementation manner, after obtaining the initial drawing prompt word input by the user, a prompt sensitivity detection may be performed on the initial drawing prompt word to obtain prompt sensitivity information, and a prompt correction process is performed on the initial drawing prompt word based on the prompt sensitivity information to obtain the initial drawing prompt word after the prompt correction process;

[0100] Optionally, a content sensitive constraint library is set up. The content sensitive constraint library is established and maintained based on a content sensitive constraint target. The content sensitive constraint target may be to avoid generating paintings involving sensitive content. By writing some keywords involving the content sensitive constraint target into the content sensitive constraint library, the content sensitive constraint library may be used in actual applications to perform prompt sensitivity detection on the initial painting prompt words to obtain prompt sensitive information.

[0101] Then, based on the prompt sensitive information, the initial painting prompt word is subjected to prompt correction processing to obtain the initial painting prompt word after prompt correction processing;

[0102] For example, if it is determined based on the prompt sensitive information that the initial painting prompt word contains sensitive keywords, the sensitive keywords of the initial painting prompt word are deleted.

[0103] For another example, sensitive keywords can be replaced with conventional keywords. When establishing a content-sensitive constraint library, corresponding conventional keywords are set for keywords related to content-sensitive constraint targets. In actual applications, conventional keywords corresponding to corresponding sensitive keywords can be obtained from the content-sensitive constraint library based on prompt sensitive information, and then sensitive keywords can be replaced with conventional keywords.

[0104] In a feasible implementation manner, a painting sensitivity detection is performed on the target painting to obtain painting sensitivity information, and a painting correction process is performed on the target painting based on the painting sensitivity information to obtain the target painting after the painting correction process.

[0105] In one or more embodiments of the present application, by obtaining the initial painting prompt word input by the user, the painting style mode corresponding to the initial painting prompt word is determined, and the target painting is obtained by performing text-generated image processing based on the initial painting prompt word through the painting large language model in the painting style mode, rather than directly indicating the initial painting prompt word to instruct the large model to perform text-generated image. In this way, the painting large language model is guided in advance to fully refer to the style characteristics of the painting style mode, and the painting processing flow is optimized, so that the overall effect of generating a painting under the painting style mode can be better, more in line with the user's painting expectations, and a target painting with better quality can be generated for display to the user.

[0106] See also Figure 4 , Figure 4 This is a flow chart of another embodiment of a painting processing method proposed in this application. Specifically:

[0107] S202: Obtaining the initial drawing prompt word input by the user;

[0108] For details, please refer to the method steps in other embodiments of this specification, which will not be repeated here.

[0109] S204: Determine painting scene description data for the user;

[0110] The painting scene description data can be understood as metadata describing the user's literary scene. The painting scene description data can include painting themes, painting style interest information, historical painting styles, user attributes (such as age, identity, knowledge level, etc.), etc., which can describe the user's literary scene.

[0111] In this specification, by introducing painting scene description data into the initial painting prompt words originally input by the user to adjust the prompt words, the target painting prompt words finally input into the large model can include components describing the user's literary and artistic scene and the matched painting style guidance information. Subsequently, the large model can be instructed that the user's literary and artistic scene and painting style can be converted into literary and artistic scenes more accurately, thereby improving the efficiency and accuracy of generating paintings.

[0112] In a feasible implementation manner, obtaining the painting style interest information of the user, and generating the painting scene description data for the user based on the painting style interest information;

[0113] Collect the user's historical painting data and identify the painting style interest information for the historical painting data. Identify the user's painting style interest information by analyzing the user's past painting data in the historical painting data. For example, identify the user's preferred color, line, composition, theme and other elements, and summarize these elements as the user's unique painting style interest information. In some embodiments, the painting style interest information can be used as the painting style interest information matched by the current user and saved. In actual applications, the user's painting style interest information can be directly obtained to generate a series of painting scene description data that meets the user's preferences. These description data include elements such as interest color, interest light, interest composition, interest object and interest theme to form a separate painting scene description data for the user.

[0114] It can be understood that by obtaining the user's painting style interest information to generate painting scene description data, the initial painting prompt words can be modified later, so as to provide the user with more personalized painting prompt words that are in line with their painting style interest information, and better express the user's creativity and ideas during the painting process.

[0115] Exemplary:

[0116] In a feasible implementation, a painting theme for the user can be determined based on the initial painting prompt words, user attribute information and historical painting style for the user can be determined, and painting scene description data for the user can be generated based on the painting theme, the user attribute information and the historical painting style.

[0117] Painting theme: The theme semantics of the initial painting prompt words can be analyzed to obtain the painting theme generated by the user's intention;

[0118] User attribute information: including the (current) user's age, gender, occupation, educational background, interests, etc. User attribute information can provide clues about the user's possible interests, knowledge level, and experience;

[0119] Historical painting style: The user's painting style is identified based on the user's historical painting data. Specifically, the historical painting style can be determined by performing style analysis on painting elements such as color, lines, composition, and theme in the historical painting data.

[0120] Then, the painting theme, user attribute information and historical painting style are combined to obtain the painting scene description data for the user, and the painting scene description data is used to subsequently determine the painting style mode that matches it, so as to match the painting style mode that is highly consistent with the user's interests.

[0121] S206: Performing style pattern matching processing on the painting scene description data to obtain a painting style pattern corresponding to the initial painting prompt word.

[0122] Exemplarily, a plurality of reference painting style modes are predefined, and a painting style library is established by collecting paintings of different artistic painting styles, and key painting features such as color matching, line application, composition method, theme tendency, etc. are extracted from the paintings in the style library, and then each reference painting style mode is defined based on the extracted key painting features;

[0123] Specifically, key description elements are extracted from the painting scene description data, the extracted key description elements are matched with the defined reference painting style patterns to obtain style pattern similarity, and the painting style pattern indicated by the highest style pattern similarity is selected.

[0124] In a feasible implementation manner, performing style pattern matching processing on the painting scene description data to obtain a painting style pattern corresponding to the initial painting prompt word may be:

[0125] Inputting the painting scene description data into a painting style recommendation model to perform painting style recommendation processing, and outputting a painting style model corresponding to the initial painting prompt word;

[0126] The painting style recommendation model is obtained by training an initial painting style recommendation model using sample painting scene description data that has been annotated with painting style pattern labels.

[0127] Exemplarily, the model training process of a painting style recommendation model is illustrated below:

[0128] Model creation: Create a painting style recommendation model based on the machine learning model;

[0129] Sample data acquisition: acquiring sample painting scene description data, where the sample painting scene description data is the painting scene description data in the model training phase;

[0130] Sample data annotation: Use expert services to annotate sample painting scene description data with painting style pattern labels;

[0131] Model training process: input the sample painting scene description data into the initial painting style recommendation model for at least one round of model training. In each round of model training, the initial painting style recommendation model recommends the painting style of the sample painting scene description data to obtain a predicted painting style mode; based on the predicted query scene description and the painting style mode label, a model loss function is used to determine the model loss value, and based on the model loss value, the model parameters of the initial painting style recommendation model are adjusted until the model training end condition is met to obtain a painting style recommendation model;

[0132] Optionally, the model loss function can be a hinge loss function, a cross entropy loss function, a contrast loss function, etc.

[0133] Optionally, the model training end condition of the model may include, for example, the value of the loss function is less than or equal to a preset loss function threshold, the number of iterations reaches a preset number threshold, etc. The specific model training end condition can be determined based on actual conditions and is not specifically limited here.

[0134] It should be noted that the machine learning models involved in one or more embodiments of the present specification include but are not limited to the fitting of one or more machine learning models such as convolutional neural network (CNN) model, deep neural network (DNN) model, recurrent neural network (RNN) model, embedding model, gradient boosting decision tree (GBDT) model, logistic regression (LR) model, etc.

[0135] S208: Determine the painting style guidance information corresponding to the painting style mode, and perform painting style guidance enhancement processing on the initial painting prompt word based on the painting style guidance information to obtain a target painting prompt word;

[0136] Exemplarily, reference painting style guidance information of each reference painting style mode is predefined for one or more reference painting style modes, such as color preference guidance words, line and brushstroke guidance words, composition suggestion guidance words, light and shadow guidance words, texture guidance words, etc.;

[0137] After the painting style mode is determined, a set of predefined painting style guidance information is extracted or obtained for the painting style mode, and then the painting style guidance information is added to the initial painting prompt word to enhance the painting style prompt, so as to obtain the target painting prompt word.

[0138] Optionally, the painting style guidance information may include painting style positive guidance words and / or painting style negative guidance words;

[0139] In a feasible implementation, the following method may be referred to:

[0140] A2: Determine a painting style positive guiding word and / or a painting style negative guiding word corresponding to the painting style pattern;

[0141] For example, based on the painting style guidance information and the content of the initial painting prompt words, the following prompt word enhancement processing is performed:

[0142] Color enhancement: Add color description guide words that match the painting style to the prompt words, such as positive guide words "use warm tones to depict the sunset". For example, negative guide words "do not use cold tones to depict the sunset"

[0143] Tips for brushstrokes and lines: Add tips and guides about the use of brushstrokes and lines, such as "Use rough brushstrokes to express the texture of rocks";

[0144] Composition guidance: Give specific composition suggestions, such as positive guidance words "focus on the center of the picture, surrounded by trees and rivers"; negative guidance words such as "focus on the periphery of the picture"

[0145] Description of light, shadow and texture: Add descriptive guide words for light, shadow and texture, such as positive guide words "emphasize the contrast of light and shadow to highlight the three-dimensional sense of the building";

[0146] Emphasis on themes and elements: If the painting style has specific themes or elements, you can emphasize these elements in the prompts, such as positive guides such as "add flying birds to the picture to express the theme of freedom"; negative guides such as "do not add fallen leaves to the picture to express the theme of desolation"

[0147] The enhanced content is integrated into the initial painting prompt words to generate target painting prompt words. It is ensured that the target prompt words contain the core content of the initial painting prompt words and the elements of the painting style guidance information, so as to guide the painting creation more accurately.

[0148] A4: Based on the painting style positive guiding words and / or the painting style negative guiding words, the initial painting prompt words are optimized in prompt word style to obtain target painting prompt words.

[0149] Exemplary:

[0150] Initial drawing prompt: Draw a cabin in the forest with sunlight shining through the leaves.

[0151] Painting style mode:

[0152] Impressionism

[0153] Drawing style guide information:

[0154] Color preference: warm and bright colors, emphasizing the changes in light and shadow, colors are not cold, and it is forbidden to paint a cold atmosphere.

[0155] Brushstrokes and lines: Use loose brushstrokes to avoid fine brushwork and capture the fluidity of light and shadow.

[0156] Target painting prompt: Use warm and bright colors to paint a cabin in the forest. The sunlight shines through the leaves on the cabin, creating a mottled light and shadow effect. Use loose brushstrokes to capture the flow of light and shadow, emphasizing the changes in light and shadow between the sunlight and the leaves.

[0157] S210: In the painting style mode, the target painting prompt word is input into the painting language model to perform text-based image processing to obtain a target painting.

[0158] For example, the generated target painting prompt words have advantages such as improved accuracy, enhanced style consistency, creative stimulation, improved flexibility and enhanced interpretability for the painting language model. These advantages enable the painting language model to more accurately understand user needs, generate images that meet the requirements, and play an important role in a wide range of literary and graphic application scenarios.

[0159] Improved accuracy: The target painting prompt word combines the initial description and painting style guidance information to enable the large model to understand the user's intention more accurately. This accuracy is reflected not only in the accuracy of the content, but also in the accuracy of the style. The large model can more accurately grasp the scene, emotion, and style that the user wants to express, thereby generating images that better meet the user's needs.

[0160] Enhanced style consistency: By adding painting style guidance information, the target painting prompt word can ensure that the generated image is consistent in style. The model can generate images with consistent style characteristics according to the style requirements in the prompt word, improving the integrity and aesthetics of the output.

[0161] Creative stimulation: The style guidance information in the target painting prompt can also stimulate the creativity of the model. By introducing different painting styles and elements, the large model can incorporate more creativity and inspiration in the process of generating images. At the same time, it can not only enrich the expressiveness of the image, but also meet the user's demand for novel and unique images.

[0162] Improved flexibility: Since the target painting prompt words are generated based on the initial description and style guidance information, it has high flexibility. Users can modify the initial description or add different style guidance information as needed to generate images with different content and styles. This flexibility enables the model to be applicable to a wider range of application scenarios and user needs.

[0163] Enhanced interpretability: By generating target painting prompt words, the model can more clearly express the process and basis of its image generation. This helps to enhance the interpretability of the model, allowing users to better understand the working principle and output results of the model. At the same time, it also helps to discover possible problems and defects in the model generation process, so as to carry out targeted optimization and improvement.

[0164] S212: Perform painting display processing to the user based on the target painting.

[0165] For details, please refer to the method steps in other embodiments of this specification, which will not be repeated here.

[0166] In one or more embodiments of the present application, the target painting prompt words generated by the above method have advantages such as improved accuracy, enhanced style consistency, creative stimulation, improved flexibility and enhanced interpretability for the painting language model. These advantages enable the painting language model to more accurately understand user needs, generate images that meet the requirements, and play an important role in a wide range of literary and artistic application scenarios.

[0167] See also Figure 5 , Figure 5 This is a schematic diagram of the model training process of a large painting language model proposed in this application. Specifically:

[0168] S302: Using the basic large language model to create an initial painting large language model for the painting generation scenario;

[0169] Specifically, a basic large language model is obtained; based on the basic large language model, an initial painting large language model is created which includes at least a painting generation scene adaptation module and a large language generation module.

[0170] Optionally, different painting style generation scene adaptation modules can be created for different reference painting style modes; for example, a universal painting style generation scene adaptation module corresponding to the universal painting style mode, a two-dimensional painting style generation scene adaptation module corresponding to the two-dimensional painting style mode, and so on; subsequently, sample painting prompt words of different reference painting styles are used for model transfer training.

[0171] A trained basic large language model can be obtained, and the basic large language model can be adapted to a painting-based generation scenario to obtain a painting large language model. Usually, the basic large language model is directly applied to generate different painting styles in the painting generation scenario, which is difficult to adapt to the new painting scenario. Based on this, the basic large language model is first obtained to create an initial painting large language model, and sample painting prompt words in the new painting scenario or sample painting prompt words of different reference painting styles in the painting scenario are obtained to perform model migration training for different painting styles. Since the basic large language model is usually a trained open source AIGC model with content generation capabilities, based on this, this manual only needs to adapt it to the painting scenario. Specifically, the sample data can be used to perform model fine-tuning training on the initial painting large language model, and after the model fine-tuning training is completed, a painting large language model adapted to the painting scenario is obtained.

[0172] S304: Obtain sample painting prompt words, and label the sample painting prompt words with painting labels;

[0173] Sample data acquisition: obtain sample painting prompt words, call expert services to annotate painting labels for the sample painting prompt words (usually painting images drawn by expert services based on the sample painting prompt words); the sample painting prompt words may come from a historical cultural image database, the Internet, and the like.

[0174] S306: Using the sample painting prompt words to perform model transfer training on the initial painting language model,

[0175] S308: During the model transfer training process, a sample painting style mode corresponding to the sample painting prompt word is determined, and a predicted painting is obtained by performing Wensheng graph processing based on the sample painting prompt word through the painting large language model in the sample painting style mode, and model parameters of the initial painting large language model are adjusted based on the predicted painting and the painting label to obtain a painting large language model after transfer training.

[0176] Model transfer training process: input sample painting prompt words into the initial painting large language model for at least one round of model training to obtain predicted paintings, calculate the model comprehensive loss based on the predicted paintings and the painting labels, adjust the model parameters of the painting scene adaptation module in the initial painting large language model based on the comprehensive model loss, and control the model parameters of the large language generation module to remain unchanged. Until the model training end conditions are met to obtain the large language generation module and the painting scene adaptation module, complete the model fusion of the large language generation module and the painting scene adaptation module, and obtain a trained painting large language model.

[0177] In some embodiments, different painting style generation scene adaptation modules can be created for different reference painting style modes; for example, a universal painting style generation scene adaptation module corresponding to a universal painting style mode, a two-dimensional painting style generation scene adaptation module corresponding to a two-dimensional painting style mode, and so on; subsequently, the model migration training is performed in stages using sample painting prompt words of different reference painting styles, and a painting style generation scene adaptation module that completes the end condition of the stage model training can be obtained. At this time, a combination of "n painting style generation scene adaptation modules + 1 large language generation module" can be obtained. After the combination is completed, a trained painting large language model can be obtained. In actual application deployment, the current painting style generation scene adaptation module corresponding to the painting style mode is selected according to "determining the painting style mode corresponding to the initial painting prompt word", and the current painting style generation scene adaptation module is linked and integrated with the painting scene adaptation module, thereby completing the control of the painting large language model to enter the current painting style mode;

[0178] The link fusion of the large language generation module and the painting scene adaptation module is to fuse the model structure layer weights of a painting scene adaptation module with the large language generation module, and by determining the target model structure layer to which the model structure layer weights correspond in the large language generation module, the model structure layer parameters of the target model structure layer are fused with the model structure layer weights. The model structure layer weights of the painting scene adaptation module may only partially correspond to and have model structure layer weights in all model structure layers in the basic large language model. By completing the parameter update of the model structure layer based on the model structure layer weights for this part of the target model structure layer, the reference update process of all model structure layer weights is completed by analogy, thereby completing the control of the painting large language model to enter the current painting style mode.

[0179] Optionally, the model training end condition of the model may include, for example, the value of the loss function is less than or equal to a preset loss function threshold, the number of iterations reaches a preset number threshold, etc. The specific model training end condition can be determined based on actual conditions and is not specifically limited here.

[0180] In one or more embodiments of the present application, a large painting language model can be trained by the above method. The large painting language model can be used to perform text-generated image processing based on the initial painting prompt words through the large painting language model in the painting style mode to obtain a target painting, rather than directly indicating the initial painting prompt words to instruct the large model to perform text-generated image. In this way, the large painting language model is guided in advance to fully refer to the style characteristics of the painting style mode, and the painting processing flow is optimized, so that the overall effect of generating paintings in this painting style mode is better, more in line with the user's painting expectations, and a target painting with better quality can be generated for display to the user.

[0181] The following will be combined Figure 6 , the painting processing device provided in the embodiment of the present application is introduced in detail. It should be noted that, Figure 6 The painting processing device shown is used to execute the present application Figure 1 to Figure 5 For the convenience of explanation, only the part related to the embodiment of the present application is shown. For the specific technical details not disclosed, please refer to the present application. Figure 1 to Figure 5 The embodiment shown.

[0182] See also Figure 6 , which shows a schematic diagram of the structure of a painting processing device according to an embodiment of the present application. The painting processing device 1 can be implemented as all or part of a device through software, hardware, or a combination of both. According to some embodiments, the painting processing device 1 includes a Wensheng image module 11 and a painting display module 12, which are specifically used to:

[0183] The Wenshengtu module 11 is used to obtain the initial painting prompt word input by the user, determine the painting style mode corresponding to the initial painting prompt word, and perform Wenshengtu processing based on the initial painting prompt word through the painting large language model in the painting style mode to obtain the target painting;

[0184] The painting display module 12 is used to display the target painting to the user.

[0185] Optional, such as Figure 7 As shown, the Wensheng diagram module 11 includes:

[0186] A description determining unit 111, configured to determine painting scene description data for the user;

[0187] The painting processing unit 112 is used to perform style pattern matching processing on the painting scene description data to obtain a painting style pattern corresponding to the initial painting prompt word.

[0188] Optionally, the description determining unit 111 is configured to:

[0189] Acquiring the painting style interest information of the user, and generating painting scene description data for the user based on the painting style interest information; and / or,

[0190] A painting theme for the user is determined based on the initial painting prompt word, user attribute information and a historical painting style for the user are determined, and painting scene description data for the user is generated based on the painting theme, the user attribute information and the historical painting style.

[0191] Optionally, the painting processing unit 112 is used to: input the painting scene description data into a painting style recommendation model to perform painting style recommendation processing, and output a painting style model corresponding to the initial painting prompt word;

[0192] The painting style recommendation model is obtained by training an initial painting style recommendation model using sample painting scene description data that has been annotated with painting style pattern labels.

[0193] Optionally, the Wensheng diagram module 11 is used to:

[0194] A prompt enhancement unit 113 is used to determine the painting style guidance information corresponding to the painting style mode, and perform painting style prompt enhancement processing on the initial painting prompt word based on the painting style guidance information to obtain a target painting prompt word;

[0195] In the painting style mode, the target painting prompt word is input into the painting language model for text-based image processing to obtain the target painting.

[0196] Optionally, the prompt enhancement unit 113 is used to:

[0197] Determine a painting style positive guiding word and / or a painting style negative guiding word corresponding to the painting style mode;

[0198] Based on the painting style positive guiding words and / or the painting style negative guiding words, the initial painting prompt words are subjected to prompt word style optimization processing to obtain target painting prompt words.

[0199] Optionally, the device 1 is also used for:

[0200] Performing prompt sensitivity detection on the initial drawing prompt word to obtain prompt sensitivity information, and performing prompt correction processing on the initial drawing prompt word based on the prompt sensitivity information to obtain the initial drawing prompt word after prompt correction processing; and / or

[0201] A painting sensitivity detection is performed on the target painting to obtain painting sensitivity information, and a painting correction process is performed on the target painting based on the painting sensitivity information to obtain the target painting after the painting correction process.

[0202] Optionally, the device 1 is also used for:

[0203] The basic large language model is used to create an initial painting large language model for the painting generation scenario;

[0204] Obtaining sample painting prompt words, and marking painting labels on the sample painting prompt words;

[0205] The sample painting prompt words are used to perform model transfer training on the initial painting large language model. During the model transfer training process, a sample painting style mode corresponding to the sample painting prompt words is determined. Under the sample painting style mode, the sample painting prompt words are used to perform Wensheng graph processing through the painting large language model to obtain a predicted painting. Based on the predicted painting and the painting label, model parameters of the initial painting large language model are adjusted to obtain a painting large language model after transfer training.

[0206] It should be noted that the painting processing device provided in the above embodiment only uses the division of the above functional modules as an example when executing the painting processing method. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the painting processing device provided in the above embodiment and the painting processing method embodiment belong to the same concept, and the implementation process thereof is detailed in the method embodiment, which will not be repeated here.

[0207] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0208] In an embodiment of the present application, by obtaining the initial painting prompt word input by the user, the painting style mode corresponding to the initial painting prompt word is determined, and the target painting is obtained by performing text-generated image processing based on the initial painting prompt word through the painting large language model in the painting style mode, rather than directly indicating the initial painting prompt word to instruct the large model to perform text-generated image. In this way, the painting large language model is guided in advance to fully refer to the style characteristics of the painting style mode, and the painting processing flow is optimized, so that the overall effect of generating a painting under the painting style mode can be better, more in line with the user's painting expectations, and a target painting with better quality can be generated for display to the user.

[0209] The present application also provides a computer storage medium, which can store multiple instructions, and the instructions are suitable for being loaded and executed by a processor as described above. Figure 1 to Figure 5 The specific execution process of the painting processing method in the embodiment shown can be found in Figure 1 to Figure 5 The specific description of the illustrated embodiment will not be repeated here.

[0210] The present application also provides a computer program product, which stores at least one instruction, and the at least one instruction is loaded and executed by the processor as described above. Figure 1 to Figure 5 The specific execution process of the painting processing method in the embodiment shown can be found in Figure 1 to Figure 5 The specific description of the illustrated embodiment will not be repeated here.

[0211] Please refer to Figure 8 , which shows a block diagram of the structure of an electronic device provided by an exemplary embodiment of the present application. The electronic device in the present application may include one or more of the following components: a processor 110, a memory 120, an input device 130, an output device 140, and a bus 150. The processor 110, the memory 120, the input device 130, and the output device 140 may be connected via the bus 150.

[0212] The processor 110 may include one or more processing cores. The processor 110 uses various interfaces and lines to connect various parts of the entire electronic device, and executes various functions and processes data of the electronic device 100 by running or executing instructions, programs, code sets or instruction sets stored in the memory 120, and calling data stored in the memory 120. Optionally, the processor 110 can be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor 110 can integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing display content; and the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 110, but may be implemented separately through a communication chip.

[0213] The memory 120 may include a random access memory (RAM) or a read-only memory (ROM). Optionally, the memory 120 includes a non-transitory computer-readable storage medium. The memory 120 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 120 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the following various method embodiments, etc. The operating system may be an Android system, including a system deeply developed based on the Android system, an IOS system developed by Apple, including a system deeply developed based on the IOS system or other systems. The data storage area may also store data created by the electronic device during use, such as a phone book, audio and video data, chat record data, etc.

[0214] See also Fig. 9As shown, the memory 120 can be divided into an operating system space and a user space. The operating system runs in the operating system space, and native and third-party applications run in the user space. In order to ensure that different third-party applications can achieve good operating results, the operating system allocates corresponding system resources to different third-party applications. However, different application scenarios in the same third-party application also have different requirements for system resources. For example, in the local resource loading scenario, the third-party application has higher requirements for disk reading speed; in the animation rendering scenario, the third-party application has higher requirements for GPU performance. The operating system and third-party applications are independent of each other, and the operating system often cannot perceive the current application scenario of the third-party application in a timely manner, resulting in the operating system being unable to perform targeted system resource adaptation according to the specific application scenario of the third-party application.

[0215] In order for the operating system to distinguish the specific application scenarios of third-party applications, it is necessary to open up data communication between third-party applications and the operating system so that the operating system can obtain the current scenario information of third-party applications at any time, and then perform targeted system resource adaptation based on the current scenario.

[0216] Taking the Android operating system as an example, the programs and data stored in the memory 120 are as follows: Fig.10As shown, the memory 120 may store a Linux kernel layer 320, a system runtime library layer 340, an application framework layer 360 and an application layer 380, wherein the Linux kernel layer 320, the system runtime library layer 340 and the application framework layer 360 belong to the operating system space, and the application layer 380 belongs to the user space. The Linux kernel layer 320 provides underlying drivers for various hardware of electronic devices, such as display drivers, audio drivers, camera drivers, Bluetooth drivers, Wi-Fi drivers, power management, etc. The system runtime library layer 340 provides the main feature support for the Android system through some C / C++ libraries. For example, the SQLite library provides database support, the OpenGL / ES library provides 3D drawing support, and the Webkit library provides browser kernel support, etc. The system runtime library layer 340 also provides an Android runtime library (Android runtime), which mainly provides some core libraries that allow developers to use Java language to write Android applications. The application framework layer 360 provides various APIs that may be used when building applications. Developers can also use these APIs to build their own applications, such as activity management, window management, view management, notification management, content provider, package management, call management, resource management, and location management. At least one application runs in the application layer 380. These applications can be native applications that come with the operating system, such as contact applications, text messaging applications, clock applications, camera applications, etc.; they can also be third-party applications developed by third-party developers, such as game applications, instant messaging applications, photo beautification applications, etc.

[0217] Taking the operating system as an IOS system as an example, the programs and data stored in the memory 120 are as follows: Fig.11As shown, the IOS system includes: a core operating system layer 420 (Core OS layer), a core service layer 440 (Core Services layer), a media layer 460 (Media layer), and a touchable layer 480 (Cocoa Touch Layer). The core operating system layer 420 includes the operating system kernel, drivers, and underlying program frameworks, which provide functions closer to the hardware for use by the program framework located in the core service layer 440. The core service layer 440 provides system services and / or program frameworks required by the application, such as the foundation framework, account framework, advertising framework, data storage framework, network connection framework, geographic location framework, motion framework, etc. The media layer 460 provides audio-visual interfaces for the application, such as graphics and image related interfaces, audio technology related interfaces, video technology related interfaces, and wireless playback (AirPlay) interfaces for audio and video transmission technologies. The touchable layer 480 provides various commonly used interface-related frameworks for application development, and the touchable layer 480 is responsible for the user's touch interaction operations on the electronic device. For example, local notification service, remote push service, advertising framework, game tool framework, message user interface (UI) framework, user interface UIKit framework, map framework, etc.

[0218] exist Fig.11 Among the frameworks shown, the frameworks related to most applications include but are not limited to: the basic framework in the core service layer 440 and the UIKit framework in the touchable layer 480. The basic framework provides many basic object classes and data types, provides the most basic system services for all applications, and has nothing to do with UI. The classes provided by the UIKit framework are basic UI class libraries for creating touch-based user interfaces. iOS applications can provide UIs based on the UIKit framework, so it provides the basic architecture of applications for building user interfaces, drawing, processing and user interaction events, responding to gestures, etc.

[0219] Among them, the method and principle of implementing data communication between third-party applications and the operating system in the IOS system can be referred to the Android system, and this application will not go into details here.

[0220] Among them, the input device 130 is used to receive input instructions or data, and the input device 130 includes but is not limited to a keyboard, a mouse, a camera, a microphone or a touch device. The output device 140 is used to output instructions or data, and the output device 140 includes but is not limited to a display device and a speaker. In one example, the input device 130 and the output device 140 can be combined, and the input device 130 and the output device 140 are touch screen displays, which are used to receive touch operations on or near the user using any suitable object such as a finger or a touch pen, and to display the user interface of each application. The touch screen display is usually set on the front panel of the electronic device. The touch screen display can be designed as a full screen, a curved screen or a special-shaped screen. The touch screen display can also be designed as a combination of a full screen and a curved screen, or a combination of a special-shaped screen and a curved screen, which is not limited in the embodiments of the present application.

[0221] In addition, those skilled in the art will appreciate that the structure of the electronic device shown in the above drawings does not constitute a limitation on the electronic device, and the electronic device may include more or fewer components than shown, or combine certain components, or arrange the components differently. For example, the electronic device also includes a radio frequency circuit, an input unit, a sensor, an audio circuit, a wireless fidelity (WiFi) module, a power supply, a Bluetooth module and other components, which will not be described in detail here.

[0222] In the embodiment of the present application, the execution subject of each step may be the electronic device described above. Optionally, the execution subject of each step is the operating system of the electronic device. The operating system may be an Android system, an IOS system, or other operating systems, which is not limited in the embodiment of the present application.

[0223] The electronic device of the embodiment of the present application may also be equipped with a display device, which may be any device capable of realizing a display function, such as a cathode ray tube display (CR), a light-emitting diode display (LED), an electronic ink screen, a liquid crystal display (LCD), a plasma display panel (PDP), etc. The user may use the display device on the electronic device 101 to view displayed text, images, videos and other information. The electronic device may be a smart phone, a tablet computer, a gaming device, an AR (Augmented Reality) device, a car, a data storage device, an audio playback device, a video playback device, a notebook, a desktop computing device, a wearable device such as an electronic watch, an electronic glasses, an electronic helmet, an electronic bracelet, an electronic necklace, electronic clothing and other devices.

[0224] exist Figure 8 In the electronic device shown, the processor 110 may be used to call an application stored in the memory 120 and specifically perform the following operations:

[0225] Acquire an initial painting prompt word input by a user, determine a painting style mode corresponding to the initial painting prompt word, and perform text-based image processing based on the initial painting prompt word through the painting large language model in the painting style mode to obtain a target painting;

[0226] A painting display process is performed to the user based on the target painting.

[0227] In one embodiment, the processor 110 performs the following operations when determining the painting style mode corresponding to the initial painting prompt word:

[0228] Determining painting scene description data for the user;

[0229] The painting scene description data is subjected to style pattern matching processing to obtain a painting style pattern corresponding to the initial painting prompt word.

[0230] In one embodiment, the processor 110 performs the following steps when determining the painting scene description data for the user:

[0231] Acquiring the painting style interest information of the user, and generating painting scene description data for the user based on the painting style interest information; and / or,

[0232] A painting theme for the user is determined based on the initial painting prompt word, user attribute information and a historical painting style for the user are determined, and painting scene description data for the user is generated based on the painting theme, the user attribute information and the historical painting style.

[0233] In one embodiment, the processor 110 performs the following steps when performing the style pattern matching process on the painting scene description data to obtain the painting style pattern corresponding to the initial painting prompt word:

[0234] Inputting the painting scene description data into a painting style recommendation model to perform painting style recommendation processing, and outputting a painting style model corresponding to the initial painting prompt word;

[0235] The painting style recommendation model is obtained by training an initial painting style recommendation model using sample painting scene description data that has been annotated with painting style pattern labels.

[0236] In a feasible implementation manner, in the painting style mode, based on the initial painting prompt word, the target painting is obtained by performing Wensheng picture processing through the painting large language model, and the following steps are performed:

[0237] Determine the painting style guidance information corresponding to the painting style mode, and perform painting style guidance enhancement processing on the initial painting prompt word based on the painting style guidance information to obtain a target painting prompt word;

[0238] In the painting style mode, the target painting prompt word is input into the painting language model for text-based image processing to obtain the target painting.

[0239] In one embodiment, the processor 110 performs the following steps when determining the painting style guidance information corresponding to the painting style mode and performing painting style guidance enhancement processing on the initial painting prompt word based on the painting style guidance information to obtain a target painting prompt word:

[0240] Determine a painting style positive guiding word and / or a painting style negative guiding word corresponding to the painting style mode;

[0241] Based on the painting style positive guiding words and / or the painting style negative guiding words, the initial painting prompt words are subjected to prompt word style optimization processing to obtain target painting prompt words.

[0242] In one embodiment, the processor 110 further performs the following steps when executing the method:

[0243] Performing prompt sensitivity detection on the initial drawing prompt word to obtain prompt sensitivity information, and performing prompt correction processing on the initial drawing prompt word based on the prompt sensitivity information to obtain the initial drawing prompt word after prompt correction processing; and / or

[0244] A painting sensitivity detection is performed on the target painting to obtain painting sensitivity information, and a painting correction process is performed on the target painting based on the painting sensitivity information to obtain the target painting after the painting correction process.

[0245] In one embodiment, when executing the painting processing method, the processor 110 further performs the following steps:

[0246] The basic large language model is used to create an initial painting large language model for the painting generation scenario;

[0247] Obtaining sample painting prompt words, and marking painting labels on the sample painting prompt words;

[0248] The sample painting prompt words are used to perform model transfer training on the initial painting large language model,

[0249] During the model transfer training process, a sample painting style mode corresponding to the sample painting prompt word is determined, and a predicted painting is obtained by performing Wensheng graph processing based on the sample painting prompt word through the painting large language model under the sample painting style mode, and model parameters of the initial painting large language model are adjusted based on the predicted painting and the painting label to obtain a painting large language model after transfer training.

[0250] In an embodiment of the present application, by obtaining the initial painting prompt word input by the user, the painting style mode corresponding to the initial painting prompt word is determined, and the target painting is obtained by performing text-generated image processing based on the initial painting prompt word through the painting large language model in the painting style mode, rather than directly indicating the initial painting prompt word to instruct the large model to perform text-generated image. In this way, the painting large language model is guided in advance to fully refer to the style characteristics of the painting style mode, and the painting processing flow is optimized, so that the overall effect of generating a painting under the painting style mode can be better, more in line with the user's painting expectations, and a target painting with better quality can be generated for display to the user.

[0251] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only storage memory, or a random access memory, etc.

[0252] The above disclosure is only the preferred embodiment of the present application, which certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. A painting processing method, characterized in that: The method comprises: Acquire an initial painting prompt word input by a user, determine a painting style mode corresponding to the initial painting prompt word, and perform text-based image processing based on the initial painting prompt word through the painting large language model in the painting style mode to obtain a target painting; A painting display process is performed to the user based on the target painting.

2. The method according to claim 1, characterized in that The determining of the painting style mode corresponding to the initial painting prompt word includes: Determining painting scene description data for the user; The painting scene description data is subjected to style pattern matching processing to obtain a painting style pattern corresponding to the initial painting prompt word.

3. The method according to claim 2, characterized in that The determining of the painting scene description data for the user includes: Acquiring the painting style interest information of the user, and generating painting scene description data for the user based on the painting style interest information; and / or, A painting theme for the user is determined based on the initial painting prompt word, user attribute information and a historical painting style for the user are determined, and painting scene description data for the user is generated based on the painting theme, the user attribute information and the historical painting style.

4. The method according to claim 2, characterized in that: The performing style pattern matching processing on the painting scene description data to obtain a painting style pattern corresponding to the initial painting prompt word includes: Inputting the painting scene description data into a painting style recommendation model to perform painting style recommendation processing, and outputting a painting style model corresponding to the initial painting prompt word; The painting style recommendation model is obtained by training an initial painting style recommendation model using sample painting scene description data that has been annotated with painting style pattern labels.

5. The method according to claim 1, characterized in that The step of obtaining a target painting by performing text image processing based on the initial painting prompt word through the painting large language model in the painting style mode includes: Determine the painting style guidance information corresponding to the painting style mode, and perform painting style guidance enhancement processing on the initial painting prompt word based on the painting style guidance information to obtain a target painting prompt word; In the painting style mode, the target painting prompt word is input into the painting large language model for text-based image processing to obtain the target painting.

6. The method according to claim 5, characterized in that The determining of the painting style guidance information corresponding to the painting style mode, and performing painting style guidance enhancement processing on the initial painting prompt word based on the painting style guidance information to obtain a target painting prompt word, includes: Determine a painting style positive guiding word and / or a painting style negative guiding word corresponding to the painting style mode; Based on the painting style positive guiding words and / or the painting style negative guiding words, the initial painting prompt words are subjected to prompt word style optimization processing to obtain target painting prompt words.

7. The method according to claim 1, characterized in that The method further comprises: Performing prompt sensitivity detection on the initial drawing prompt word to obtain prompt sensitivity information, and performing prompt correction processing on the initial drawing prompt word based on the prompt sensitivity information to obtain the initial drawing prompt word after prompt correction processing; and / or A painting sensitivity detection is performed on the target painting to obtain painting sensitivity information, and a painting correction process is performed on the target painting based on the painting sensitivity information to obtain the target painting after the painting correction process.

8. The method according to claim 1, characterized in that The method further comprises: The basic large language model is used to create an initial painting large language model for the painting generation scenario; Obtaining sample painting prompt words, and marking painting labels on the sample painting prompt words; The sample painting prompt words are used to perform model transfer training on the initial painting large language model. During the model transfer training process, a sample painting style mode corresponding to the sample painting prompt words is determined. Under the sample painting style mode, the sample painting prompt words are used to perform Wensheng graph processing through the painting large language model to obtain a predicted painting. Based on the predicted painting and the painting label, model parameters of the initial painting large language model are adjusted to obtain a painting large language model after transfer training.

9. A painting processing device, characterized in that: The device comprises: A Wenshengtu module is used to obtain an initial painting prompt word input by a user, determine a painting style mode corresponding to the initial painting prompt word, and perform Wenshengtu processing based on the initial painting prompt word through the painting large language model in the painting style mode to obtain a target painting; The painting display module is used to display the target painting to the user.

10. An electronic device, characterized in that: include: A processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the method steps as claimed in any one of claims 1 to 8.

Citation Information

Patent Citations

  • Brain-machine art system and working method thereof

    CN115981466A

  • Image generation method and device, computer equipment and storage medium

    CN116363242A

  • Processing system and method for generating image based on user text cue word

    CN116680425A

  • Ancient poetry artistic conception map generation method based on chain analysis

    CN117274411A

  • Image generation method and device, terminal and storage medium

    CN117392254A

Cited By

  • Method and apparatus for generating picture, computing device and computer program product

    CN120032002A

  • Natural-language-driven artificial intelligence sub-mirror drawing method and natural-language-driven artificial intelligence sub-mirror drawing system

    CN120279126A

  • Method and apparatus for generating image, computing device and program product

    CN120451295A

  • Automatic situation plotting method and device and electronic equipment

    CN120804223A

  • Image generation method and device, equipment, storage medium and product

    CN120997628A