Personalized image generation method and model training method

Abstract

Provided in one or more embodiments of the present specification are a personalized image generation method, a model training method, a device, a medium and a product. The personalized image generation method comprises: acquiring scene description information and image generation requirements of an interaction scenario, and acquiring user information and image preference information of a target user (S501); generating a first prompt on the basis of the scene description information and the user information, and inputting the first prompt into a trained text generation model, so as to generate, by means of the text generation model, a personalized copy that conforms to the interaction scenario and is oriented to the target user (S502); and generating a second prompt on the basis of the image generation requirements, the image preference information and the personalized copy, and inputting the second prompt into a trained image generation model, so as to generate, by means of the image generation model, a personalized image that conforms to the image generation requirements and the image preference information and comprises the personalized copy (S503).

Description

Personalized image generation method and model training method TECHNICAL FIELD

[0001] One or more embodiments of the present specification relate to the technical field of data processing, and in particular to a personalized image generation method, a text generation model training method, an electronic device, a computer readable medium, and a computer program product. BACKGROUND

[0002] In various interactive scenarios that need to convey information to users, such as product marketing, advertisement promotion, e-commerce recommendation, and the like, in order to improve the conveying effect and the interactive willingness of users, the initiator of the activity will usually convey relevant information (such as marketing activity information, product information, advertisement content, and the like) to users in a combination of text and images. This way, through the combination of vision and text, it can attract the attention of users and enhance the readability and attractiveness of information. However, in the face of a large and diverse user group, a single marketing style often cannot meet the needs of all users. SUMMARY

[0003] Therefore, one or more embodiments of the present specification provide a personalized image generation method, a text generation model training method, an electronic device, a computer readable medium, and a computer program product.

[0004] To achieve the above object, one or more embodiments of the present specification provide the technical solutions as follows.

[0005] According to a first aspect of one or more embodiments of the present specification, a personalized image generation method is provided, including: obtaining scene description information of an interactive scenario and image generation requirements, and obtaining user information and image preference information of a target user; generating a first prompt word based on the scene description information and the user information, and inputting the first prompt word into a trained text generation model to generate a personalized script that conforms to the interactive scenario and is targeted at the target user by the text generation model; generating a second prompt word based on the image generation requirements, the image preference information, and the personalized script, and inputting the second prompt word into a trained image generation model to generate a personalized image that conforms to the image generation requirements and the image preference information and includes the personalized script by the image generation model.

[0006] According to a second aspect of the embodiments of the present specification, a method for training a text generation model is provided, including: obtaining a plurality of training samples, each of the training samples including input information and a label, the input information including scene description information of an interaction scene and user information of an interacted user, the label including a script in an image corresponding to the interaction scene, the interacted user having performed a specified interaction behavior on the image corresponding to the interaction scene; inputting a prompt word generated by the input information in the training sample into a text generation model to be trained, to generate, by the text generation model to be trained, a predicted script conforming to the interaction scene and facing the interacted user based on the prompt word, and to train the text generation model to be trained with a minimum difference between the predicted script and the label as an optimization objective; and wherein the trained text generation model is applied to the method of the first aspect.

[0007] According to a third aspect of the embodiments of the present specification, an electronic device is provided, including: a processor; a memory for storing processor executable instructions; wherein the processor executes the executable instructions to implement the method of the first aspect.

[0008] According to a fourth aspect of the embodiments of the present specification, a computer readable storage medium is provided, having a computer program stored thereon, the program being executed by a processor to implement the steps of the method of the first aspect.

[0009] According to a fifth aspect of the embodiments of the present specification, a computer program product is provided, including a computer program, the computer program being executed by a processor to implement the steps of the method of the first aspect.

[0010] The technical solutions provided in the embodiments of this specification may include the following beneficial effects: In the embodiments of this specification, scene description information and image generation requirements of the interaction scenario are obtained, as well as user information and image preference information of the target user are obtained. Then, a first prompt word is generated based on the scene description information and the user information, and the first prompt word is input into a pre-trained text generation model so that the text generation model generates personalized copy that conforms to the interaction scenario and is geared towards the target user. On the one hand, this ensures that the personalized copy is highly relevant to the interaction scenario, avoiding irrelevant promotion or information display; on the other hand, it ensures that the personalized copy conforms to user preferences, which can increase user engagement with the copy. This increases the acceptance rate of users and effectively enhances their willingness to interact. Furthermore, based on the image generation requirements, image preference information, and personalized text, a second prompt word is generated. This second prompt word is then input into a pre-trained image generation model, which generates a personalized image that conforms to the image generation requirements and image preference information, and includes the personalized text. The content of the generated personalized image is highly relevant to the interaction scenario and meets the user's visual needs. Both the text and image content in the personalized image conform to the user's preferences, achieving a synergistic effect between text and images, creating a dual appeal of visual and linguistic elements, and enhancing the effectiveness and interest of information delivery.

[0011] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this specification. Attached Figure Description

[0012] Figure 1 is a schematic diagram of the architecture of a personalized image generation service system provided in an exemplary embodiment.

[0013] Figure 2 is a schematic diagram of the training of a text generation model provided in an exemplary embodiment.

[0014] Figure 3 is a schematic diagram of the structure of a text generation model provided in an exemplary embodiment.

[0015] Figure 4 is a schematic diagram of the structure of another text generation model provided in an exemplary embodiment.

[0016] Figure 5 is a flowchart of a personalized image generation method provided in an exemplary embodiment.

[0017] Figure 6 is a flowchart of another personalized image generation method provided in an exemplary embodiment.

[0018] Figure 7 is a schematic diagram of the structure of an electronic device provided in an exemplary embodiment. Detailed Implementation

[0019] The exemplary embodiments will be described in detail herein with reference to the attached drawings. The following description is made with reference to the accompanying drawings in which like reference numerals represent like elements or similar elements, unless otherwise indicated. The following exemplary embodiments described herein represent implementations consistent with the embodiments of the present specification and are intended to provide an overview or framework for understanding the nature and character of the embodiments of the specification. However, unless otherwise indicated herein, the following examples are not intended to limit or restrict the various embodiments of the present specification to the examples described herein.

[0020] It should be noted that the steps of the methods performed by the respective methods in other embodiments are not necessarily performed in the order described in the specification. In some other embodiments, the steps included in the methods can be more or less than described in the specification. In addition, a single step described in the specification can be divided into multiple steps for description in other embodiments, and multiple steps described in the specification can be combined into a single step for description in other embodiments.

[0021] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the specification are information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation portal for user to choose authorization or refusal.

[0022] The related terms appearing in the embodiments of the present application are explained and described herein.

[0023] 1、Transformer: A deep learning model architecture, is a model based on attention mechanism, is specially used for processing sequence data, especially in the field of natural language processing has made remarkable success.

[0024] 2、Attention mechanism (Attention): is the core component of Transformer, it is a mechanism for processing sequence data, especially suitable for natural language processing tasks. Attention mechanism allows the model to consider the information of all other elements in the sequence when processing each element in the sequence, so as to capture long distance dependencies and better understand the context.

[0025] For example, the implementation of attention mechanism includes the following steps.

[0026] Firstly, Query, Key, Value transformation is involved, for each input position, three vectors are obtained by linear transformation: Query vector (q), Key vector (k) and Value vector (v). These vectors are used to calculate attention scores and generate output. By calculating the similarity between Query vector and each Key vector (usually using dot product or other similarity function), the attention score of each position to all other positions is obtained, and finally the weighted sum of all Value vectors using attention score is used to generate the final output.

[0027] which can be represented as follows: q = XW Q ; k = XW K ; v = XW V ; out = attnW o .

[0028] where X represents the input vector, W Q represents the query matrix learned during model training, W K represents the key matrix learned during model training, W V represents the value matrix learned during model training, q represents the Query vector, k represents the Key vector, v represents the Value vector, softmax() is a scaling and normalization function, is a scaling factor (used to scale the dot product result to a moderate range, thereby alleviating the problem of numerical instability), d is the dimension of the Query vector, k T is the transpose of the Key vector, W o represents the output weight learned during model training, and out represents the final output.

[0029] 3. Self-attention mechanism (Self-attention) allows each element in the input sequence to pay attention to all elements in the sequence (including itself) in order to calculate its output representation. This process helps to capture long-distance dependencies in the sequence. The implementation of Self-attention usually contains the following steps.

[0030] (1) Linear transformation: for each token in the input sequence, it is mapped to Query (Q), Key (K) and Value (V) vector space by three learnable weight matrices W Q , W V and W K .

[0031] (2) Calculate attention scores: Use dot product or other similarity functions to calculate the degree of match or relevance between each query and all other keys. The result of this step is a score matrix that represents the strength of association between different elements.

[0032] (3) Softmax normalization: Perform a softmax operation on the attention scores to ensure that the scores of each row add up to 1, and obtain a probability distribution that reflects the importance of each element in the current context.

[0033] (4) Weighted summation: Using the probability distribution obtained in the previous step as weights, the Values ​​are weighted and summed to generate the final output representation for each position.

[0034] To enhance the model's expressive power, the self-attention mechanism typically employs a multi-head approach, executing multiple processes simultaneously, each using a different parameter set. Finally, the outputs of each head are concatenated and passed through an additional linear layer to obtain the final output.

[0035] 4. Cross-attention is a mechanism for calculating attention between two different input sequences. Specifically, when processing one sequence (e.g., the target sequence), it relies on another sequence (e.g., the source sequence) to generate a better representation. In cross-attention, one sequence is the Key and Value, and the other is the Query.

[0036] Suppose there are two input sequences: (1) Sequence A (Query sequence): used to generate Query vector (q); (2) Sequence B (Key-Value sequence): used to generate Key vector (k) and Value vector (v).

[0037] First, calculate the Query vector (q), Key vector (k), and Value vector (v): q = X A W Q k = X B W K v = X B W V ;X A X is the representation of sequence A. B W is the representation of sequence B. Q W represents the query matrix learned during model training. K W represents the key matrix learned during model training. V This represents the value matrix learned during model training, where q represents the Query vector, k represents the Key vector, and v represents the Value vector.

[0038] The attention score is then calculated by the following equation: Finally, the value vectors are weighted and summed by the attention scores to obtain the enhanced representation of sequence A in the context of sequence B.

[0039] In some embodiments, FIG. 1 is a schematic diagram of an architecture of a personalized image generation service system according to an example embodiment. As shown in FIG. 1, the system can include a server 11, a network 12, and a plurality of user terminals such as a PC (Personal Computer) 13, a mobile phone 14, and the like.

[0040] The server 11 can be a physical server including a standalone host, or the server 11 can be a virtual server carried by a host cluster. In operation, the server 11 can run a server-side program of a personalized image generation application to implement a corresponding personalized image generation platform.

[0041] The PC 13 and the mobile phone 14 are only some types of user terminals that can be used by users. In fact, users can also use other types of user terminals such as tablet devices, notebook computers, PDAs (Personal Digital Assistants), wearable devices (such as smart glasses, smart watches, and the like), and the like, without limitation. In operation, the user terminals can run a client-side program of a personalized image generation application to implement a client of the personalized image generation service. The application program of the client of the personalized image generation service can be started and run on the user terminals. The client-side program can be a native application installed on the user terminals, or the client-side program can be an applet, a fast application, or other similar forms. Of course, when using web page technologies such as HTML5 or the like, the relevant functions can be implemented through a page displayed by a browser, which can be a standalone browser application or a browser module embedded in some application.

[0042] The network 12 between the user terminals such as the PC 13 and the mobile phone 14 and the server 11 can be implemented based on a communication mode supported by the corresponding user terminals, and can be implemented by wired or wireless networks without limitation. For example, the PC 13 can support wired and wireless communication, and can be implemented by wired or wireless networks as needed. The mobile phone 14 usually only supports wireless communication, and can be implemented by a wireless network.

[0043] To address the problems in related technologies, server 11 can deploy pre-trained models, such as text generation models, image generation models, image style description models, and image detection models, to execute the personalized image generation method provided in this embodiment. This generates personalized images for different users, catering to the diverse needs of a large and varied user group and enhancing user engagement. Alternatively, other electronic devices connected to server 11 can also execute the method and transmit the generated personalized images to server 11; this embodiment does not impose any limitations on this.

[0044] (1) Regarding the text generation model used in the embodiments of this specification.

[0045] One possible implementation is to directly use pre-trained language models, such as Large Language Models (LLMs). Pre-trained language models are based on deep learning techniques, especially artificial intelligence models trained using large corpora, and aim to understand and generate text similar to human language, possessing powerful natural language understanding and generation capabilities. The goal of LLMs is to leverage natural language processing capabilities to achieve various applications, such as text generation, translation, summarization, question answering, and dialogue systems, thereby helping to improve the efficiency and automation of human-computer interaction.

[0046] In another possible implementation, to improve the generation of personalized copy, the pre-trained language model can be fine-tuned. This fine-tuning process can be performed by any computing-capable electronic device; this embodiment does not impose any restrictions on this. Referring to Figure 2, the pre-trained language model can be used as the text generation model to be trained, and the following training process can be performed.

[0047] First, the electronic device can acquire multiple pre-prepared training samples. Each training sample includes input information and labels. The input information includes scene descriptions of the interaction scenario and user information of the users who have interacted with the scenario. The labels include text in the images corresponding to the interaction scenario, and the user's specified interactive behavior in response to the images. The training samples clearly demonstrate the relationship between the input information and the labels, providing high-quality labeled data for model learning.

[0048] For example, the interaction scenario is a marketing scenario for a certain product, the scenario description information is the relevant introduction information of the product, the marketing scenario of the product produces images for marketing, and the interacting users refer to users who have performed interactive behaviors such as clicking, collecting, and liking the images produced by the marketing scenario of the product. The user information of the interacting users includes, but is not limited to, user behavior data, interest data, basic attributes, etc.

[0049] In one example, assuming the interaction scenario is a marketing scenario for beauty products, a training sample includes: (1) Scenario description information includes: {Product type: lipstick; Core selling points: matte texture, long-lasting and non-fading, 20 shades; Target audience: working women aged 25-35. Marketing theme: Confident makeup for the workplace}. (2) User information of the interacting users includes but is not limited to: {User ID: U2023_Beauty_001, Consumption habits: {Monthly average beauty consumption: 800 yuan, Preferred categories: [lipstick, foundation], Purchase channel: e-commerce live broadcast}, Basic attributes: {Age: 28, Occupation: financial analyst}, Behavioral characteristics: {Recently viewed: [Workplace outfit tutorial, makeup tips video]}}. (3) Tags: [Workplace essential] 12-hour makeup without embarrassment, XX matte lipstick creates a confident lip look.

[0050] In another example, assuming the interaction scenario is a game promotion scenario, a training sample includes: (1) Scenario description information includes: {Product type: mobile game, core selling points: [open world exploration, multiplayer dungeons, character development], target audience: students aged 18-25, marketing theme: new user registration}. (2) User information of the interacting user includes but is not limited to: {User ID: U2023_Game_045, consumption habits: {average monthly game consumption: 300 yuan, preferred categories: [MMORPG, competitive games], purchase channel: in-game store}, basic attributes: {age: 22, occupation: college student}, behavioral characteristics: {recently viewed: [game strategy videos, e-sports competition live broadcasts]}}. (3) Tags: [Epic open world] Explore the fantasy realm with millions of players, log in to receive an SSR character!

[0051] After acquiring multiple training samples, the electronic device can generate prompt words based on the input information in the training samples. Then, the prompt words generated from the input information in the training samples are input into the text generation model to be trained. The text generation model to be trained generates predicted text that conforms to the interaction scenario and is aimed at the interactive user based on the prompt words. The optimization objective is to minimize the difference between the predicted text and the label.

[0052] The role of cue words is to transform the scene description information of the interaction scenario and the user information of the interacting users into a structured input, helping the text generation model understand the task objective and generate personalized copy that meets user needs. Cue words integrate the input information (including the description of the interaction scenario and user information) into a concise, structured prompt, enabling the model to extract important features from this information. Through these cue words, the model can better understand the relationship between the interaction scenario and user needs. Cue words are not merely a way to organize information; they also clarify the goal and constraints of the generation task. They instruct the model on which inputs to use to generate copy that meets the needs of the target user and ensure that the generated copy matches the specific interaction scenario and user profile.

[0053] In some embodiments, user information includes personalized user information represented in text form and a first embedding vector representing a user profile of an interacting user. The personalized user information represented in text form and the first embedding vector representing a user profile of an interacting user each play different but complementary roles, providing the model with user information at different levels and in different forms, thereby making the generated copy more personalized and accurate.

[0054] Personalized user information, represented in text form, can be tailored to the characteristics of the interaction scenario and includes detailed descriptions reflecting user preferences, needs, and behaviors. This information helps the model to more intuitively understand the user's basic characteristics and needs, and allows for flexible adjustments based on the interaction scenario. It serves the following purposes.

[0055] (1) Strong scenario relevance: Textual information is usually tailored to specific interaction scenarios, accurately describing user behavior and needs within those scenarios. For example, in e-commerce product marketing scenarios, personalized user information includes historical purchase records, browsing history, shopping cart data, and basic attribute information, directly reflecting user behavior patterns, preferences, and sensitivities during shopping. In travel / travel scenarios, personalized user information includes travel history, destination preferences, budget, travel frequency, mode of transportation, and basic attribute information, showcasing user travel interests and needs. This enables textual information to help models accurately capture users' immediate needs in specific scenarios.

[0056] (2) High interpretability: Personalized information represented in text form is intuitive and easy to understand, and can provide clearer user characteristics. For example, information such as a user's preference for a certain brand and price sensitivity can be clearly described, which helps the model adjust its output to match the user's specific needs.

[0057] The first embedding vector used to characterize the user profile of an interacting user is a dense vector representation that integrates multiple relevant user information, typically composed of historical data and global information about behavioral patterns. It serves the following purposes.

[0058] (1) Comprehensive User Features: The first embedding vector is not merely a single user behavior or attribute information, but is created through comprehensive analysis of various user data (such as behavioral data, interests, historical interaction records, etc.). This allows it to capture the full picture of the user and their potential needs. For example, in e-commerce scenarios, the first embedding vector may integrate information such as the user's purchasing habits, price sensitivity, and brand preferences, thus providing a more comprehensive reference for generating personalized recommendations or advertisements.

[0059] (2) Deep Pattern Learning: The first embedding vector can reflect deep patterns and preference trends in user behavior, which may not be intuitively expressed by pure text information. For example, a user's repeated browsing or purchasing behavior of a certain type of product can be reflected by the embedding vector, and this information may not be fully presented by simple text description.

[0060] In summary, personalized user information represented in text form focuses on describing user needs and behaviors in specific interaction scenarios through concrete and interpretable information. It provides the model with intuitive and highly context-relevant input, supporting accurate content generation and recommendation. The first embedding vector, on the other hand, transforms all relevant user information into a numerical representation, capturing the user's potential interests, behavioral patterns, and global features. This vectorized representation provides the model with deep personalized insights, supporting efficient personalized recommendations and content generation. The combination of these two approaches enables the model to handle complex user needs by providing both clear and interpretable personalized information and leveraging embedding vectors for deep learning, thereby generating content that better aligns with user interests and needs.

[0061] To adapt to the aforementioned user information, for each training sample, the electronic device can generate textual prompts based on the user's personalized information, scene description information, and specified placeholders in each training sample. Then, the prompts and the first embedding vector are input into the text generation model to be trained. The text generation model to be trained generates predicted text that conforms to the interaction scenario and is geared towards the interacting user based on the prompts and the first embedding vector. The optimization objective is to minimize the difference between the predicted text and the label.

[0062] Please refer to Figure 3. The text generation model to be trained includes a linear processing layer, an embedding layer, and multiple Transformer structures.

[0063] The embedding layer is used to map the text in the prompt words, except for the specified placeholders, into embedding vectors based on learnable embedding vector mapping relationships.

[0064] In this model, the text generation model to be trained is a pre-trained neural network model. During the pre-training phase, the text generation model establishes initial embedding vector mapping relationships by learning from existing large-scale text data. These initial embedding vector mapping relationships are typically learned based on common language features and patterns (such as word meanings and grammatical structures of words and sentences). For example, the model can map different words, phrases, or grammatical structures to a vector space, so that semantically similar words or sentences have similar vector representations in the vector space. Understandably, in addition to establishing initial embedding vector mapping relationships, the text generation model also accumulates certain general knowledge during the pre-training phase, such as common facts, historical events, cultural backgrounds, and common sense. This embodiment does not impose any restrictions on this.

[0065] Next, this learnable embedding vector mapping relationship can be further adjusted and optimized during the training process with the help of personalized information in the training samples.

[0066] During this training process, the personalized information of each interacting user (such as purchase history, interests, and behavioral data) is mapped into an initial embedding vector representation. The model optimizes through backpropagation, updating the learnable parameters of the embedding layer based on the error between the predicted and tagged text. In this way, the model gradually learns the mapping relationship between the user's personalized information and the embedding vector. As training progresses, the model's embedding layer gradually increases its ability to map the personalized information of interacting users. In other words, the embedding vector mapping relationships contained in the embedding layer further enhance the mapping relationship between the personalized information of interacting users and the embedding vectors, thereby increasing the control of personalized information over the model's generated results and making the model-generated text more in line with the needs of individual users.

[0067] The linear processing layer processes the vector dimension of the first embedding vector to align it with the embedding vector obtained by the embedding layer in the same vector space. In this embodiment, the first embedding vector and the embedding vector generated by the embedding layer may reside in different vector spaces. The linear processing layer learns appropriate weights to transform the first embedding vector into a new representation, enabling this vector to align with the embedding vectors of other inputs in a unified vector space. This ensures that vectors from different inputs can be effectively fused and processed in the same feature space. It is understood that if the first embedding vector itself is already aligned with the embedding vector obtained by the embedding layer in the same vector space, no further processing by the linear processing layer is required.

[0068] The embedding layer is also used to replace the specified placeholder in the prompt with the processed first embedding vector, thereby introducing the first embedding vector and effectively combining the first embedding vector with other parts of the prompt to obtain a second embedding vector that can integrate the first embedding vector and textual user personalization information. The second embedding vector is then input into multiple subsequent Transformer structures for further processing.

[0069] It is understandable that the parameters of the linear processing layer, embedding layer, and multiple Transformer structures included in the text generation model can be updated during this training process based on the aforementioned optimization objectives, making the fine-tuned text generation model more suitable for personalized copywriting generation scenarios. As for the specific parameter update method, parameter update algorithms from relevant technologies can be used, and this embodiment does not impose any restrictions on this.

[0070] In one possible implementation, if it is only necessary to input the second embedding vector into a number of subsequent Transformer structures for processing, the number of Transformer structures can refer to the model architecture in related technologies, and this embodiment does not impose any restrictions on this.

[0071] In one possible implementation, to further enhance the control of the first embedding vector over the model generation results, the first embedding vector can be introduced again in the last Transformer structure among multiple Transformer structures, and the second embedding vector can be used as the input of the first Transformer structure among multiple Transformer structures; the first embedding vector can be used as a part of the input of the last Transformer structure among multiple Transformer structures.

[0072] For example, the last Transformer structure in a plurality of Transformer structures includes a self-attention structure and a cross-attention structure; the input of the self-attention structure is the output of the previous Transformer structure; the input of the cross-attention structure includes the output of the self-attention structure and a first embedding vector. The output of the self-attention structure is used to calculate the query vector in the cross-attention structure, and the first embedding vector is used to calculate the key vector and value vector in the cross-attention structure. In this embodiment, by introducing the first embedding vector only in the last Transformer structure, the model can learn the general context and patterns in the preceding Transformer structures, thereby constructing a more generalized representation. Introducing the first embedding vector again in the last Transformer structure allows the model to make personalized fine-tuning during the generation process. At this point, the model already has basic semantic and contextual information, and by introducing the first embedding vector again, it can ensure that the generated content is more in line with the interests, needs, or behavioral characteristics of a specific user.

[0073] The other Transformer structures, except for the last one, can refer to the model architecture in related technologies. This embodiment does not impose any restrictions on them.

[0074] For example, please refer to Figure 4, which illustrates N Transformer structures, where N is an integer greater than 0. The last Transformer structure in the multiple Transformer structures includes a self-attention structure, a normalization layer, a cross-attention structure, a fusion layer, and a feedforward neural network.

[0075] The self-attention structure is used to perform calculations based on the output of the previous Transformer structure. The specific calculation process can be found in the description above, and will not be repeated here.

[0076] The normalization layer is used to normalize the output of the self-attention structure.

[0077] The input to the cross-attention structure includes the output of the normalization layer and the first embedding vector. The output of the normalization layer is used to calculate the query vector in the cross-attention structure, and the first embedding vector is used to calculate the key vector and value vector in the cross-attention structure. The specific calculation process of the cross-attention structure can be found in the above description, and will not be repeated here.

[0078] The fusion layer is used to fuse the outputs of the self-attention structure and the cross-attention structure; for example, it can be fused based on bitwise addition, and the fusion result can be further normalized.

[0079] Feedforward neural networks are used to process the output of the fusion layer.

[0080] Of course, in addition to linear processing layers, embedding layers, and multiple Transformer structures, the text generation model to be trained can also contain other structures, such as an LM head (Language Model Head) structure containing multiple Transformer structures. The LM head structure can convert the output of the last Transformer structure into a probability distribution for predicting the next word.

[0081] In another possible implementation, the user information of the interacting user can also contain only the user's personalized information in text form. In this case, the prompt does not need to contain a specified placeholder. The prompt only needs to be converted into an embedding vector by the embedding layer, and then the embedding vector can be input into multiple Transformer structures for further processing.

[0082] (2) Regarding the image generation model used in the embodiments of this specification, the image generation model is capable of generating another image that meets the requirements based on at least one of the input text and image.

[0083] In one possible implementation, pre-trained image generation models from relevant technologies can be directly adopted, such as text-to-image diffusion models and generative adversarial networks.

[0084] In another possible implementation, to improve the generation of personalized images, the pre-trained image generation model in the relevant technology can be fine-tuned using relevant training samples. This fine-tuning process can be performed by any computing-capable electronic device; this embodiment does not impose any restrictions on this. The training samples include input information and labels, and the output information includes the image generation requirements of the interactive scenario and the image preference information of the interacting users. The labels include the images corresponding to the interactive scenario.

[0085] (3) Regarding the image style description model used in the embodiments of this specification, the image style description model is used to analyze the input image to generate corresponding image style description information.

[0086] In one possible implementation, pre-trained image style description models from related technologies can be directly adopted, such as the BLIP (Bootstrapping Language-Image Pretraining) model.

[0087] In another possible implementation, to improve the generation of personalized images, the pre-trained image style description model in the relevant technology can be fine-tuned using relevant training samples. This fine-tuning process can be performed by any computing-capable electronic device; this embodiment does not impose any restrictions on this. The training samples include input information and labels, and the output information includes images from which the user has performed specified interactive actions, with the labels being the style description information of those images.

[0088] (4) Regarding the image detection model used in the embodiments of this specification, the image detection model is used to detect the content of a specified dimension in the input image to determine whether the requirements are met.

[0089] In one possible implementation, a pre-trained image detection model from related technologies can be directly used.

[0090] In another possible implementation, to improve the detection effect of image quality, the pre-trained image detection model in the relevant technology can be fine-tuned using relevant training samples. The fine-tuning process can be performed by any electronic device with computing power, and this embodiment does not impose any restrictions on this. The training samples include input information and labels. The input information for positive samples includes qualified images, and the label is an identifier representing that the detection has passed; the input information for negative samples includes unqualified images, and the label is an identifier representing that the detection has failed.

[0091] In some embodiments, the following provides an exemplary description of a personalized image generation method provided in the embodiments of this specification, which can be executed by the server 11 in FIG1. ​​Referring to FIG5, the method includes the following steps.

[0092] In S501, the scene description information and image generation requirements of the interaction scene are obtained, as well as the user information and image preference information of the target user are obtained.

[0093] The scenario description information for an interactive scenario involves the basic background and requirements of the scenario. For example, in an advertising and marketing scenario, the scenario description might include information such as product category, core selling points, and target audience. If it's a travel recommendation, the scenario description might involve destination, season, and type of activity.

[0094] Image generation requirements include basic requirements for image generation, such as image style, size, content, and quality. These requirements clearly define the general outline of the image relevant to the interactive scenario.

[0095] The target user's information includes the user's historical data, behavior, and basic attributes. For example, the user's age, gender, interests, and purchase history help the model understand the user's personalized needs. For instance, the user information includes: personalized user information represented in text form, and a first embedding vector used to characterize the target user's profile.

[0096] Image preference information refers to a user's specific requirements regarding image style, color, content, and other aspects. It helps refine the generated image, making it more aligned with the user's aesthetic or needs. For example, a user might prefer a clean, modern style or have a preference for a particular color.

[0097] For example, if the target user is not a cold-start user, since the target user already has a certain interaction history, the target user's image preference information includes at least one of the following: historical images related to the target user's specified interaction behavior, and the target user's image style preference information; wherein, the target user's image style preference information is obtained by recognizing the historical images related to the target user's specified interaction behavior using a pre-trained image style description model. Specified interaction behaviors include, but are not limited to, clicking, saving, liking, and forwarding. Since non-cold-start users have accumulated a certain amount of historical data, the server can infer their preferences based on the user's historical interaction data, and make more accurate personalized recommendations.

[0098] For example, if the target user is a cold-start user, the server cannot directly infer the target user's image preferences based on the target user's historical interaction behavior. In this case, it can find reference users similar to the target user based on the target user's first embedding vector. The similarity between the first embedding vector of the reference user and the first embedding vector of the target user meets a preset similarity condition. For example, the preset similarity condition is that the similarity between the first embedding vector of the reference user and the first embedding vector of the target user is greater than a preset threshold, or that the similarity between the first embedding vector of the reference user and the first embedding vector of the target user is the highest.

[0099] Image preference information from reference users can be used to infer the preferences of cold-start users. The image preference information of the target user includes at least one of the following: historical images related to a specified interaction behavior of the reference user, and image style preference information of the reference user; wherein the image style preference information of the reference user is obtained by identifying historical images related to the specified interaction behavior of the reference user using a pre-trained image style description model. By leveraging information from reference users similar to the target user, effective personalized recommendations can be made even in the absence of direct data from the target user, providing a reasonable initial recommendation for cold-start users.

[0100] In S502, a first prompt word is generated based on scene description information and user information. The first prompt word is then input into a pre-trained text generation model, which generates personalized copy that conforms to the interaction scenario and is geared towards the target user.

[0101] In this step, the generated personalized copy includes promotional slogans and advertising copy for products or activities, and will be adjusted according to users' interests and needs to increase user acceptance of the copy in order to better attract target users.

[0102] For example, referring to Figure 6, when the user information includes: personalized user information represented in text form, and a first embedding vector representing the user profile of the target user, the server can generate a first prompt word based on the personalized user information, scene description information, and a specified placeholder. The first prompt word and the first embedding vector are then input into a text generation model. The text generation model converts the text in the first prompt word (excluding the specified placeholder) into an embedding vector, and replaces the specified placeholder with the first embedding vector to obtain a second embedding vector corresponding to the first prompt word. Personalized copy that conforms to the interaction scenario and is geared towards the target user is then generated, at least based on the second embedding vector. The combination of the personalized user information represented in text form and the first embedding vector representing the user profile of the target user enables the text generation model to generate content that better matches user interests and needs when dealing with complex user requirements. This provides both clearly interpretable personalized information and the ability to utilize the embedding vector for deep learning.

[0103] The text generation model includes a linear processing layer and an embedding layer. When the first embedding vector is introduced for the first time, the embedding layer is used to map the text in the first prompt word, except for the specified placeholder, into an embedding vector based on the embedding vector mapping relationship learned during the pre-training and fine-tuning process of the text generation model. The linear processing layer is used to process the first embedding vector so that the processed first embedding vector is aligned with the embedding vector obtained by the embedding layer to the same vector space. The embedding layer is also used to replace the specified placeholder in the first prompt word with the processed first embedding vector, effectively combining the first embedding vector with other parts of the prompt word to obtain a second embedding vector corresponding to the first prompt word, thus realizing the integration of the first embedding vector and the textualized user personalized information.

[0104] To enhance the control of the first embedding vector over the model's generated results, the first embedding vector can be introduced again. The text generation model is used to generate personalized text that conforms to the interaction scenario and is geared towards the target user based on the first and second embedding vectors. The text generation model includes multiple Transformer structures. The second embedding vector serves as the input to the first Transformer structure among the multiple Transformer structures. The first embedding vector serves as a part of the input to the last Transformer structure among the multiple Transformer structures.

[0105] The last Transformer structure in a series of Transformer structures includes a self-attention structure and a cross-attention structure. The input to the self-attention structure is the output of the previous Transformer structure. The input to the cross-attention structure includes the output of the self-attention structure and a first embedding vector. The output of the self-attention structure is used to compute the query vector in the cross-attention structure, and the first embedding vector is used to compute the key vector and value vector in the cross-attention structure.

[0106] Referring to Figure 4, the last Transformer structure among the multiple Transformer structures also includes a normalization layer, a fusion layer, and a feedforward neural network; the normalization layer is used to normalize the output of the self-attention structure; the input of the cross-attention structure includes the output of the normalization layer and the first embedding vector, and the output of the normalization layer is used to calculate the query vector in the cross-attention structure; the fusion layer is used to fuse the output of the self-attention structure and the output of the cross-attention structure; the feedforward neural network is used to process the output of the fusion layer.

[0107] In S503, a second prompt word is generated based on image generation requirements, image preference information, and personalized text. The second prompt word is then input into a pre-trained image generation model, which generates a personalized image that meets the image generation requirements and image preference information and includes personalized text.

[0108] In this step, the server generates a second prompt based on the already generated personalized text, image generation requirements, and image preference information. This second prompt is then input into a trained image generation model. The model can then generate an image that both meets the image requirements and effectively conveys the personalized text, based on the user's image preferences (such as color, style, and composition) and the generated personalized text. For example, in an e-commerce scenario, the generated image might contain a high-quality product display image, while also embedding a slogan that matches the text content, enhancing the image's marketing effectiveness.

[0109] In this embodiment, a language generation model can be used to generate personalized copy that conforms to the interaction scenario and is geared towards the target user. On the one hand, this ensures that the personalized copy is highly relevant to the interaction scenario, avoiding irrelevant promotion or information display. On the other hand, it ensures that the personalized copy conforms to user preferences, which can increase user acceptance of the copy and thus effectively improve users' willingness to interact. Furthermore, an image generation model is used to generate personalized images that meet image generation requirements and image preference information and include personalized copy. The image content of the generated personalized images is not only highly relevant to the interaction scenario but also meets the user's visual needs. Both the copy and image content in the personalized images conform to user preferences, which can achieve the effect of text-image synergy, forming a dual attraction of visual and linguistic elements, and enhancing the effectiveness and interest of information transmission.

[0110] For example, referring to Figure 6, the second prompt also includes the target user's information, ensuring that the personalized image generated by the image generation model matches the target user's information. By incorporating the target user's information (such as age, gender, interests, behavioral data, etc.) into the second prompt, the image generation model can more accurately generate images that match the user's characteristics. For instance, for a young female user, the generated image might better match her preferences in style, tone, and composition; similarly, for a male user, an image might be generated that better matches his aesthetic or preferences.

[0111] In some embodiments, after obtaining a personalized image, the server can also use a trained image detection model to detect content in specified dimensions of the personalized image and obtain detection results. If the detection results indicate that the personalized image passes detection, the personalized image is output to the target user's terminal. In this embodiment, by detecting content in specified dimensions of the image, it can be ensured that the generated personalized image not only meets the user's preferences, needs, and scenario requirements, but also has high quality and adaptability. This detection helps ensure the quality and suitability of the image before output, thereby improving the user experience and the effectiveness of the image, and avoiding the sending of inappropriate or low-quality images to the target user.

[0112] The content of the specified dimensions can be set according to the actual application scenario, and this embodiment does not impose any restrictions on this.

[0113] For example, if the personalized image is a product marketing image, the specified dimensions can include whether the product appears in the image and whether the product's core selling points (such as size, color, function, etc.) are clearly displayed.

[0114] Alternatively, it can detect whether the image matches the expected interactive scenario. For example, in travel advertising images, a specified dimension could be checking whether the image displays travel scenarios preferred by the target user (such as beaches, cities, mountains, etc.).

[0115] Alternatively, check if the image resolution meets the target output requirements, ensuring the image quality is high enough, clear, and free of noticeable noise or distortion. Or, check if the image's brightness and contrast meet standards to ensure visual effectiveness on different devices.

[0116] The various technical features in the above embodiments can be combined arbitrarily, as long as there is no conflict or contradiction between the combinations of features. However, due to space limitations, they are not described one by one. Therefore, the arbitrary combination of various technical features in the above embodiments is also within the scope of this specification.

[0117] In some embodiments, this specification also provides an electronic device, including: a processor; and a memory for storing processor-executable instructions; wherein the processor implements the method described in any one of the above embodiments by executing the executable instructions.

[0118] Figure 7 is a schematic structural diagram of a device provided in an exemplary embodiment. Referring to Figure 7, at the hardware level, the device includes a processor 702, an internal bus 704, a network interface 706, a memory 708, and a non-volatile memory 710, and may also include other hardware required for its functions. One or more embodiments of this specification can be implemented in software, for example, the processor 702 reads the corresponding computer program from the non-volatile memory 710 into the memory 708 and then runs it. Of course, in addition to software implementation, one or more embodiments of this specification do not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. That is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0119] In some embodiments, the personalized image generation apparatus can be applied to the device shown in FIG7 to implement the technical solution of this specification. The personalized image generation apparatus may include the following modules.

[0120] The acquisition module is used to acquire scene description information and image generation requirements of the interaction scenario, as well as user information and image preference information of the target user.

[0121] The personalized copy generation module is used to generate a first prompt word based on the scene description information and the user information, and input the first prompt word into a pre-trained text generation model so that the text generation model can generate personalized copy that conforms to the interaction scene and is geared towards the target user.

[0122] A personalized image generation module is used to generate a second prompt word based on the image generation requirements, the image preference information, and the personalized text, and input the second prompt word into a pre-trained image generation model so that the image generation model can generate a personalized image that conforms to the image generation requirements and the image preference information and includes the personalized text.

[0123] For example, the user information includes: user personalization information represented in text form, and a first embedding vector for characterizing the user profile of the target user.

[0124] The personalized copy generation module is specifically used to generate a first prompt word based on the user's personalized information, the scene description information, and a specified placeholder; input the first prompt word and the first embedding vector into the text generation model, so that the text generation model converts the text in the first prompt word other than the specified placeholder into an embedding vector, and replaces the specified placeholder with the first embedding vector to obtain a second embedding vector corresponding to the first prompt word, and generates personalized copy that conforms to the interaction scene and is aimed at the target user based at least on the second embedding vector.

[0125] For example, the text generation model includes a linear processing layer and an embedding layer; the embedding layer is used to map the text in the first prompt word, excluding the specified placeholder, into embedding vectors based on the embedding vector mapping relationship learned during the pre-training and fine-tuning processes of the text generation model; the linear processing layer is used to process the first embedding vector so that the processed first embedding vector is aligned with the embedding vector mapped by the embedding layer to the same vector space; the embedding layer is also used to replace the specified placeholder in the first prompt word with the processed first embedding vector.

[0126] For example, the text generation model is used to generate personalized text that conforms to the interaction scenario and is geared towards the target user based on the first embedding vector and the second embedding vector; wherein, the text generation model includes multiple Transformer structures; the second embedding vector serves as the input of the first Transformer structure among the multiple Transformer structures; and the first embedding vector serves as a part of the input of the last Transformer structure among the multiple Transformer structures.

[0127] For example, the last Transformer structure among the plurality of Transformer structures includes a self-attention structure and a cross-attention structure; the input of the self-attention structure is the output of the previous Transformer structure; the input of the cross-attention structure includes the output of the self-attention structure and the first embedding vector, the output of the self-attention structure is used to calculate the query vector in the cross-attention structure, and the first embedding vector is used to calculate the key vector and value vector in the cross-attention structure.

[0128] For example, the last Transformer structure among the plurality of Transformer structures further includes a normalization layer, a fusion layer, and a feedforward neural network; the normalization layer is used to normalize the output of the self-attention structure; the input of the cross-attention structure includes the output of the normalization layer and the first embedding vector, and the output of the normalization layer is used to calculate the query vector in the cross-attention structure; the fusion layer is used to fuse the output of the self-attention structure and the output of the cross-attention structure; the feedforward neural network is used to process the output of the fusion layer.

[0129] For example, if the target user is not a cold start user, the target user's image preference information includes: historical images related to the target user's specified interaction behavior, and / or, the target user's image style preference information; wherein, the target user's image style preference information is obtained by recognizing the historical images related to the target user's specified interaction behavior using a trained image style description model.

[0130] For example, if the target user is a cold-start user, the image preference information of the target user includes: historical images related to a specified interaction behavior of a reference user, and / or, the image style preference information of the reference user; wherein, the image style preference information of the reference user is obtained by recognizing the historical images related to the specified interaction behavior of the reference user using a trained image style description model; the similarity between the first embedding vector corresponding to the reference user and the first embedding vector corresponding to the target user satisfies a preset similarity condition.

[0131] For example, the second prompt word also includes the user information of the target user, so that the personalized image generated by the image generation model matches the user information of the target user.

[0132] For example, the device further includes an image detection module, which is used to detect content in a specified dimension of the personalized image using a trained image detection model to obtain a detection result; if the detection result indicates that the personalized image has passed detection, the personalized image is output to the user terminal of the target user.

[0133] In some embodiments, the training device for the text generation model can be applied to the device shown in FIG. 7 to implement the technical solution of this specification. The training device for the text generation model may include the following modules.

[0134] The sample acquisition module is used to acquire multiple training samples. Each training sample includes input information and labels. The input information includes scene description information of the interaction scene and user information of the users who have interacted. The labels include text in the image corresponding to the interaction scene. The users who have interacted have performed a specified interaction behavior on the image corresponding to the interaction scene.

[0135] The training module is used to input prompt words generated from the input information in the training samples into the text generation model to be trained, so that the text generation model to be trained can generate predicted text that conforms to the interaction scenario and is aimed at the interactive user based on the prompt words, and train the text generation model to be trained with the optimization objective of minimizing the difference between the predicted text and the label; wherein, the trained text generation model is applied to the above-mentioned personalized image generation method.

[0136] For example, the user information includes: user personalization information represented in text form, and a first embedding vector for characterizing the user profile of the interacting user.

[0137] The training module is specifically used to input the prompt words generated by the user personalization information, the scene description information, and the specified placeholders, along with the first embedding vector, into the text generation model to be trained.

[0138] The text generation model to be trained includes a linear processing layer, an embedding layer, and multiple Transformer structures.

[0139] The embedding layer is used to map the text in the prompt word, excluding the specified placeholder, into embedding vectors based on a learnable embedding vector mapping relationship; wherein, the learnable embedding vector mapping relationship further enhances the mapping relationship between the personalized information of the interacting user and the embedding vector during this training process.

[0140] The linear processing layer is used to process the first embedding vector so that the processed first embedding vector is aligned with the embedding vector obtained by the embedding layer to the same vector space.

[0141] The embedding layer is also used to replace the specified placeholder in the prompt word with the processed first embedding vector.

[0142] The second embedding vector serves as the input to the first Transformer structure among the plurality of Transformer structures; the first embedding vector serves as a portion of the input to the last Transformer structure among the plurality of Transformer structures.

[0143] The specific implementation process of the functions and roles of each module in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0144] Based on the same concept as the methods described above, this specification also provides an electronic device, including: a processor; a memory for storing processor-executable instructions; wherein the processor performs the steps of the method as described in any of the above embodiments by executing the executable instructions.

[0145] Based on the same concept as the methods described above, this specification also provides a computer-readable storage medium having computer instructions stored thereon that, when executed by a processor, implement the steps of the methods as described in any of the above embodiments.

[0146] Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage, quantum memory, graphene-based storage media or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0147] Based on the same concept as the methods described above, this specification also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the methods as described in any of the above embodiments.

[0148] The above description is merely a preferred embodiment of one or more embodiments of this specification and is not intended to limit the scope of one or more embodiments of this specification. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of one or more embodiments of this specification should be included within the protection scope of one or more embodiments of this specification.

Claims

1. A personalized image generation method, comprising: Obtain scene description information and image generation requirements for the interaction scenario, as well as user information and image preference information of the target user; Based on the scene description information and the user information, a first prompt word is generated, and the first prompt word is input into a pre-trained text generation model so that the text generation model can generate personalized copy that conforms to the interaction scene and is aimed at the target user. Based on the image generation requirements, the image preference information, and the personalized text, a second prompt word is generated, and the second prompt word is input into a pre-trained image generation model so that the image generation model can generate a personalized image that meets the image generation requirements and the image preference information and includes the personalized text.

2. The method according to claim 1, wherein the user information includes: User-personalized information represented in text form, and a first embedding vector used to characterize the user profile of the target user; The step of generating a first prompt word based on the scene description information and the user information, and inputting the first prompt word into a pre-trained text generation model to generate personalized copy that conforms to the interaction scene and is geared towards the target user, includes: A first prompt word is generated based on the user's personalized information, the scene description information, and the specified placeholder; The first prompt word and the first embedding vector are input into the text generation model, so that the text generation model converts the text in the first prompt word other than the specified placeholder into an embedding vector, and replaces the specified placeholder with the first embedding vector to obtain a second embedding vector corresponding to the first prompt word, and generates personalized copy that conforms to the interaction scenario and is aimed at the target user based at least on the second embedding vector.

3. The method according to claim 2, wherein the text generation model comprises a linear processing layer and an embedding layer; The embedding layer is used to map the text in the first prompt word, excluding the specified placeholder, into embedding vectors based on the embedding vector mapping relationship learned during the pre-training and fine-tuning processes of the text generation model. The linear processing layer is used to process the first embedding vector so that the processed first embedding vector and the embedding vector obtained by mapping the embedding layer are aligned to the same vector space. The embedding layer is further configured to replace the specified placeholder in the first prompt word with the processed first embedding vector.

4. The method according to claim 2 or 3, wherein the text generation model is used to generate personalized text that conforms to the interaction scenario and is geared towards the target user based on the first embedding vector and the second embedding vector; in, The text generation model includes multiple Transformer structures; The second embedding vector serves as the input to the first Transformer structure among the plurality of Transformer structures; The first embedding vector serves as a part of the input to the last Transformer structure among the plurality of Transformer structures.

5. The method according to claim 4, wherein the last Transformer structure among the plurality of Transformer structures includes a self-attention structure and a cross-attention structure; The input to the self-attention structure is the output of the previous Transformer structure; The input to the cross-attention structure includes the output of the self-attention structure and the first embedding vector. The output of the self-attention structure is used to calculate the query vector in the cross-attention structure, and the first embedding vector is used to calculate the key vector and value vector in the cross-attention structure.

6. The method according to claim 5, wherein the last Transformer structure among the plurality of Transformer structures further comprises a normalization layer, a fusion layer, and a feedforward neural network; The normalization layer is used to normalize the output of the self-attention structure; The input to the cross-attention structure includes the output of the normalization layer and the first embedding vector, wherein the output of the normalization layer is used to calculate the query vector in the cross-attention structure; The fusion layer is used to fuse the output of the self-attention structure and the output of the cross-attention structure. The feedforward neural network is used to process the output of the fusion layer.

7. The method according to claim 1, If the target user is not a cold-start user, the target user's image preference information includes: Historical images related to a specified interaction behavior of the target user, and / or, image style preference information of the target user; wherein, the image style preference information of the target user is obtained by recognizing historical images related to the specified interaction behavior of the target user using a trained image style description model; If the target user is a cold-start user, the image preference information of the target user includes: historical images related to the specified interaction behavior of the reference user, and / or, the image style preference information of the reference user; wherein, the image style preference information of the reference user is obtained by recognizing the historical images related to the specified interaction behavior of the reference user using a trained image style description model; the similarity between the first embedding vector corresponding to the reference user and the first embedding vector corresponding to the target user satisfies a preset similarity condition.

8. The method according to claim 1, wherein the second prompt word further includes the user information of the target user, so that the personalized image generated by the image generation model conforms to the user information of the target user.

9. The method according to claim 1, further comprising: The trained image detection model is used to detect content in a specified dimension in the personalized image to obtain the detection results; If the detection result indicates that the personalized image detection is successful, the personalized image is output to the user terminal of the target user.

10. A method for training a text generation model, comprising: Multiple training samples are obtained, each training sample including input information and labels. The input information includes scene description information of the interaction scene and user information of the users who have interacted. The labels include text in the image corresponding to the interaction scene. The users who have interacted have performed a specified interaction behavior on the image corresponding to the interaction scene. The prompt words generated from the input information in the training samples are input into the text generation model to be trained. The text generation model to be trained generates predicted text that conforms to the interaction scenario and is aimed at the interactive user based on the prompt words. The optimization objective is to minimize the difference between the predicted text and the tag. The trained text generation model is applied to the method described in any one of claims 1 to 9.

11. The method according to claim 10, wherein the user information includes: User-personalized information represented in text form, and a first embedding vector used to characterize the user profile of the interacting user; The prompt words generated from the input information in the training samples are input into the text generation model to be trained, including: The prompt words generated from the user's personalized information, the scene description information, and the specified placeholders, along with the first embedding vector, are input into the text generation model to be trained. The text generation model to be trained includes a linear processing layer, an embedding layer, and multiple Transformer structures. The embedding layer is used to map the text in the prompt word, excluding the specified placeholder, into embedding vectors based on a learnable embedding vector mapping relationship; wherein, the learnable embedding vector mapping relationship further enhances the mapping relationship between the personalized information of the interacting user and the embedding vector during this training process; The linear processing layer is used to process the first embedding vector so that the processed first embedding vector and the embedding vector obtained by mapping the embedding layer are aligned to the same vector space. The embedding layer is further configured to replace the specified placeholder in the prompt word with the processed first embedding vector; The second embedding vector serves as the input to the first Transformer structure among the plurality of Transformer structures; The first embedding vector serves as a part of the input to the last Transformer structure among the plurality of Transformer structures.

12. An electronic device, comprising: processor; A memory for storing processor-executable instructions; wherein the processor implements the steps of the method as described in any one of claims 1-11 by executing the executable instructions.

13. A computer-readable storage medium having stored thereon computer instructions that, when executed by a processor, implement the steps of the method as claimed in any one of claims 1-11.

14. A computer program product comprising a computer program / instructions that, when executed by a processor, implement the steps of the method as claimed in any one of claims 1-11.

Images