Prompt word optimization method and device, storage medium and program product

By optimizing the initial prompt words of the image generation large model, using the Chinese translation, English, expansion and style template application techniques of the large language model, the problem of poor comprehension ability of Chinese prompt words is solved, lowering the threshold for prompt word construction and improving Generate quality.

CN119088936APending Publication Date: 2024-12-06CHINA MERCHANTS BANK
View PDF 0 Cites 3 Cited by

Patent Information

Application Number
CN202411212105.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-30
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

The existing image generation model has poor understanding of Chinese prompt words, which leads to the generation of images not meeting user expectations, and users need to have high skills to construct appropriate prompt words, which increases the threshold for use.

Method used

By receiving initial prompt words input by users, using pre-optimized large language model combined with search enhancement generation principle, it is optimized to generate optimized prompt words, including Chinese translation, English, expansion, sorting and style template application, reducing the number of recommended prompt words Prompt word structures the threshold and improves the quality of generation.

Benefits of technology

It effectively reduces the threshold for prompt word construction of image generation large models, saves user learning costs, and significantly improves the quality and effect of image generation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119088936A_ABST
    Figure CN119088936A_ABST
Patent Text Reader

Abstract

The invention discloses a cue word optimization method and device, a storage medium and a program product, and relates to the technical field of information processing.The cue word optimization method comprises the steps that an initial cue word input by a user is received; and based on a pre-optimized large language model, in combination with a retrieval enhancement generation principle, optimizing the initial cue word to generate a first cue word. According to the scheme, the cue word construction threshold of the image generation large model can be effectively reduced, and the learning cost of a user is saved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of information processing technology, and in particular to a prompt word optimization method, device, storage medium and program product. Background Art

[0002] The current large-scale image generation model uses the text in the prompt word as a label and uses a diffusion model to continuously remove noise from the original image to generate an image that matches the text label. Therefore, the quality of the prompt word is directly related to the effect of the generated image.

[0003] The existing large-scale image generation model's training set prompt words are usually composed of English, and the understanding ability of Chinese prompt words is poor. Directly using Chinese as prompt words will result in problems such as inaccurate expression and inadequate expression. At the same time, large-scale image generation models have high skill requirements for user input of prompt words, and unprofessional input will lead to unsatisfactory output results.

[0004] Therefore, how to lower the threshold for constructing prompt words in large image generation models and improve the quality of image generation is an urgent problem to be solved.

[0005] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention

[0006] The main purpose of this application is to provide a prompt word optimization method, device, storage medium and program product, aiming to solve the technical problem of how to lower the threshold for prompt word construction of large image generation models and improve the quality of image generation.

[0007] To achieve the above objectives, the present application proposes a prompt word optimization method, which includes: Receive the initial prompt word input by the user; Based on a pre-optimized large language model and in combination with the retrieval enhancement generation principle, the initial prompt word is optimized to generate a first prompt word.

[0008] In one embodiment, before the step of optimizing the initial prompt word based on the pre-optimized large language model and combining the retrieval enhancement generation principle to generate the first prompt word, the step further includes: According to the preset large language model tasks, build several large language model roles; Based on the thought chain task prompts and preset sample data, a few sample learning is performed on the large language model role to learn the correspondence between the preset large language model tasks and subtasks, and obtain a pre-optimized large language model, wherein the thought chain task prompts are the correspondence between the preset large language model tasks and the subtasks obtained based on the thought chain principle.

[0009] In one embodiment, the step of optimizing the initial prompt word based on the pre-optimized large language model and combining the retrieval enhancement generation principle to generate the first prompt word includes: Based on the pre-optimized large language model, the initial prompt word is translated from Chinese to English and expanded to obtain a second prompt word; Based on the pre-optimized large language model and the preset prompt word sorting rule, the second prompt word is re-sorted to obtain a third prompt word; Based on the pre-optimized large language model and the preset style template library, the style template corresponding to the third prompt word is retrieved using the retrieval enhancement generation principle; The style template is applied to the third prompt word to generate a first prompt word.

[0010] In one embodiment, before the step of retrieving the style template corresponding to the third prompt word based on the pre-optimized large language model and the preset style template library by using the retrieval enhancement generation principle, the following steps are included: Collect several images to generate professional tips for a large model; Combining the professional prompt words to form a plurality of preset style templates; Based on the user data analysis and the template style characteristics, the plurality of preset style templates are classified to obtain a preset style template library.

[0011] In one embodiment, the step of retrieving the style template corresponding to the third prompt word based on the pre-optimized large language model and the preset style template library and using the retrieval enhancement generation principle includes: Extracting text from a plurality of preset style templates in a preset style template library, and performing text segmentation on the text to obtain text units; Using a word embedding model to embed the text unit into a word to obtain a first text vector; Using a word embedding model to embed the third prompt word to obtain a second text vector; Based on the pre-optimized large language model and the preset similarity rule, performing vector matching retrieval on the first text vector and the second text vector to obtain a retrieval result; Based on the search results, a style template associated with the first text vector having the highest similarity is obtained.

[0012] In one embodiment, after the step of optimizing the initial prompt word based on the pre-optimized large language model and combining the retrieval enhancement generation principle to generate the first prompt word, the step further includes: Based on the pre-optimized large language model and the preset sensitive word database, the first prompt word is reviewed for sensitive words to obtain a first review result; When the first audit result is successful, the first prompt word is input into the image generation model to generate a picture to obtain a generated image; Collecting user's evaluation of the quality of the first prompt word and the generated image to obtain quality feedback data; Based on the quality feedback data, the large language model is continuously optimized.

[0013] In one embodiment, after the step of receiving the initial prompt word input by the user, the method further includes: Based on the pre-optimized large language model and the preset sensitive word database, the initial prompt word is reviewed for sensitive words to obtain a second review result; When the second audit result is successful, the following steps are executed: based on the pre-optimized large language model, combined with retrieval enhancement generation, the initial prompt word is optimized to generate a first prompt word.

[0014] In addition, to achieve the above-mentioned purpose, the present application also proposes a prompt word optimization device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the prompt word optimization method described above.

[0015] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the prompt word optimization method described above are implemented.

[0016] In addition, to achieve the above-mentioned purpose, the present application also proposes a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps of the prompt word optimization method described above are implemented.

[0017] One or more technical solutions proposed in this application have at least the following technical effects: The prompt word optimization method, device, storage medium and program product proposed in the embodiments of the present application specifically receive an initial prompt word input by a user; based on a pre-optimized large language model and in combination with the retrieval enhancement generation principle, the initial prompt word is optimized to generate a first prompt word.

[0018] The present application receives the initial prompt word for the image generation large model input by the user, and automatically optimizes the received initial prompt word through a pre-optimized large language model and combined with the enhanced generation principle, thereby lowering the threshold for constructing the prompt word for the image generation large model and saving the user's learning cost. At the same time, the optimized first prompt word can be used for the image generation large model, effectively improving the quality and effect of image generation. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0020] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0021] Figure 1 A flowchart of the first embodiment of the method for optimizing prompt words of the present application is provided; Figure 2 A flowchart of the second embodiment of the method for optimizing prompt words of this application is provided; Figure 3 A flowchart of the third embodiment of the method for optimizing prompt words of the present application is provided; Figure 4 A flowchart of the fourth embodiment of the method for optimizing prompt words of the present application is provided; Figure 5 A flowchart of the fifth embodiment of the method for optimizing prompt words of the present application is provided; Figure 6 This is an example flow chart of a prompt word optimization method involved in an embodiment of the present application; Figure 7 It is a schematic diagram of the device structure of the hardware operating environment involved in the prompt word optimization method in the embodiment of the present application.

[0022] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0023] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0024] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0025] The main solution of the embodiment of the present application is: receiving an initial prompt word input by a user; optimizing the initial prompt word based on a pre-optimized large language model and combining the retrieval enhancement generation principle to generate a first prompt word.

[0026] Technical terms involved in the embodiments of this application: Large language model: A large language model is a natural language processing model based on deep learning, which is characterized by its ability to process and understand large amounts of text data. Large language models usually have hundreds of millions of parameters and can capture complex language structures and semantic information. Large language models are widely used in various natural language processing tasks, such as text generation, language understanding, and question-answering systems. By pre-training on large amounts of text data, large language models can generate high-quality natural language text and demonstrate strong generalization capabilities in a variety of tasks.

[0027] Image generation big model: Image generation big model refers to a deep learning model specifically used to generate images. Image generation big model is usually based on architectures such as convolutional neural network (CNN), generative adversarial network (GANs), variational autoencoder (VAEs). Image generation big model can generate realistic and diverse images by learning the rules and features in a large amount of image data. Image generation big model has broad application prospects in the fields of artistic creation, computer vision, virtual reality, etc.

[0028] Retrieval-Augmented Generation (RAG): RAG is a model that combines information retrieval and language generation techniques. When the model needs to generate text or answer questions, it first retrieves task-related information from a large collection of documents, and then uses the retrieved information to guide text generation. RAG methods can significantly improve the quality and accuracy of generated text because RAG can reference information from external knowledge bases to support its generated content. RAG models perform well in natural language processing tasks such as question-answering systems, document generation, and intelligent assistants.

[0029] Word embedding: Word embedding is a technique that converts text data into low-dimensional dense vector representations. These vectors can capture the semantic information in the text, so that the similarities and differences between texts can be measured by the distance between vectors. Word embedding is widely used in various natural language processing tasks, such as information retrieval, text classification, clustering, etc. Through word embedding, the model can better understand and process text data and improve the performance of the task.

[0030] BGE (BAAI General Embedding): BGE is an open source Chinese and English semantic vector model launched by Beijing Zhiyuan Artificial Intelligence Research Institute. The model has excellent performance in semantic retrieval accuracy and overall semantic representation ability. BGE uses RetroMAE, a pre-training algorithm for representation, which encodes low-mask input into semantic vectors, and then concatenates high-mask input with semantic vectors to reconstruct the original input, thereby improving the representation ability of word embedding.

[0031] Since the existing large-scale image generation models rely on prompt words input by users to generate images, however, since the prompt words in the training set of large-scale image generation models, such as Stable Diffusion, are usually composed of English, and the understanding ability of Chinese prompt words is poor, even if the Chinese prompt words can be translated into English through the built-in machine translation to generate images, ordinary machine translation often cannot accurately understand the user's intentions based on the context, resulting in errors in the translation results, which in turn affects the actual results and causes the image generation to fail to meet user expectations.

[0032] At the same time, the image generation quality of existing large image generation models is often highly correlated with the quality of the prompt words entered by the user, and has high requirements for the prompt words entered by the user, thereby increasing the user's usage threshold.

[0033] From the above analysis, it can be seen that there is a need for a prompt word optimization method for large image generation models to lower the threshold for prompt word construction of large image generation models and improve the image generation quality.

[0034] The present application provides a solution, which receives the initial prompt word for the image generation large model input by the user, and automatically optimizes the received initial prompt word through a pre-optimized large language model and combined with the enhanced generation principle, thereby lowering the prompt word construction threshold for the image generation large model and saving the user's learning cost. At the same time, the optimized first prompt word can be used for the image generation large model, effectively improving the quality and effect of image generation.

[0035] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or a prompt word optimization device capable of realizing the above functions, etc. The following takes the prompt word optimization device as an example to illustrate this embodiment and the following embodiments.

[0036] Based on this, the present application embodiment provides a prompt word optimization method, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the prompt word optimization method of this application.

[0037] In this embodiment, the prompt word optimization method includes steps S110 to S120: Step S110, receiving an initial prompt word input by a user; It is understandable that since the technical solution proposed in this application is actually applied to the optimization of prompt words of the large model for image generation, step S110 is required. First, the prompt word optimization device receives the initial prompt words input by the user, so that the large language model can be used to understand the user's image generation intention in the future, and the initial prompt words are further optimized, thereby improving the quality and effect of image generation. Among them, the initial prompt words refer to the user's requirements for the image generation large model to output images based on their own image usage needs.

[0038] Step S120 , based on the pre-optimized large language model and in combination with the retrieval enhancement generation principle, the initial prompt word is optimized to generate a first prompt word.

[0039] It should be noted that the pre-optimized large language model is obtained by relevant personnel pre-optimizing the large language model based on the user's image generation needs in order to make the large language model output higher-quality images to generate prompt words for the large model.

[0040] Specifically, by presetting an optimized large language model and based on the principle of retrieval enhancement generation, the database of image generation large model prompt words is used as the searchable knowledge base of the large language model, and the initial prompt words input by the user are recognized for their intent, so as to understand the image requirements required by the user. Then, based on the image requirements, the initial prompt words are further optimized, such as supplementing related prompt words, expanding prompt words, simplifying prompt words, formatting prompt words, etc., to obtain optimized prompt words, that is, the first prompt words.

[0041] In a feasible implementation manner, the step S120 may include steps A01 to A04: Step A01, based on a pre-optimized large language model, the initial prompt word is translated from Chinese into English and expanded to obtain a second prompt word; Step A02, based on the pre-optimized large language model and the preset prompt word sorting rule, the second prompt words are reordered to obtain a third prompt word; It should be noted that the preset prompt word sorting rule is a predefined and optimized sorting rule for the prompt words output by the large language model, which is used to guide how to reorder the prompt words to improve the quality, accuracy and relevance of the images generated by the image generation model. The preset prompt word sorting rule can be implemented by specifying the priority of core elements in the prompt words, limiting the image hierarchy, limiting the image style characteristics, etc.

[0042] Specifically, first, the prompt word optimization device uses a pre-optimized large language model to translate the initial prompt words received from Chinese to English, and then expands the prompt words based on the obtained English prompt words, allowing the large language model to supplement the initial prompt words with other elements of the picture to obtain the English translated and expanded prompt words, i.e., the second prompt words, to help the image generation model generate better quality images.

[0043] Then, the prompt word optimization device reorders the second prompt word according to the preset prompt word sorting rule through the pre-optimized large language model to ensure the primary and secondary status of each element in the second prompt word, and obtains the third prompt word.

[0044] For example, the preset prompt word sorting rule can be to sort the prompt words in the order of the main object, secondary object, foreground, middle and background, and environmental atmosphere of the picture. Through the above sorting, the large language model is helped to confirm that the image generation large model recognizes different elements in the third prompt word and arranges them in order, thereby improving the relevance between the picture and the prompt word.

[0045] Step A03, based on the pre-optimized large language model and the preset style template library, using the retrieval enhancement generation principle, retrieve and obtain the style template corresponding to the third prompt word; Step A04: Apply the style template to the third prompt word to generate a first prompt word.

[0046] It should be noted that the preset style template library is a collection of generation prompt words for several screen elements with different style characteristics based on the user's optimization requirements for image generation large model prompt words, and a text database containing several style templates is obtained based on these generation prompt words.

[0047] Specifically, according to the retrieval enhancement generation principle, the preset style template library is used as the searchable knowledge base of the large language model, and the pre-optimized large language model is used to perform retrieval matching according to the third prompt word to obtain a style template matching the third prompt word.

[0048] Then, the style template is applied to the third prompt word, and the natural language processing capability of the large language model is used to select or generate specific content suitable for filling in the style template from the third prompt word, and then the selected content is filled in according to the structure and placeholders of the style template, and finally the optimized first prompt word is obtained.

[0049] Alternatively, some elements in the third prompt word can be replaced with elements in the corresponding style template, and prompt words for corresponding screen elements can be added according to the style template to assist users in using the image generation model to generate higher quality images, while lowering the user's usage threshold.

[0050] In this implementation, the initial prompt words are translated from Chinese to English and expanded through a pre-optimized large language model, and the prompt words are reordered in combination with preset prompt word sorting rules to determine the weights of each element in the generated image, and the corresponding style template is retrieved and matched. By applying the style template, the final optimized prompt words are obtained, thereby lowering the prompt word construction threshold of the large model for image generation and effectively improving the quality and effect of image generation.

[0051] This embodiment provides a prompt word optimization method, which receives an initial prompt word input by a user; based on a pre-optimized large language model and in combination with the retrieval enhancement generation principle, optimizes the initial prompt word to generate a first prompt word.

[0052] The present application receives the initial prompt word for the image generation large model input by the user, and automatically optimizes the received initial prompt word through a pre-optimized large language model and combined with the enhanced generation principle, thereby lowering the threshold for constructing the prompt word for the image generation large model and saving the user's learning cost. At the same time, the optimized first prompt word can be used for the image generation large model, effectively improving the quality and effect of image generation.

[0053] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can refer to the above introduction, and will not be repeated later. Figure 2 , before the above step S120, it also includes steps S210 to S220: Step S210, constructing a plurality of large language model roles according to the preset large language model tasks; Based on the first embodiment, the preset large language model task is a pending task pre-set by relevant personnel based on the user's optimization requirements for image generation of large model prompt words, including but not limited to Chinese-English translation, prompt word expansion, prompt word sorting, prompt word style template retrieval, etc.

[0054] Specifically, according to the preset large language model tasks, the role of the large language model is set, which effectively narrows the overall problem domain, reduces ambiguity, and converts the versatility of the large language model into professionalism.

[0055] For example, when translating prompt words from Chinese to English is required, the large language model role will be set as "Chinese to English Translation Assistant"; when expanding and reordering prompt words, the large language model role will be set as "Prompt Word Creation Assistant"; when reviewing sensitive words, the large language model role will be set as "Text Review Expert".

[0056] Step S220, based on the thought chain task prompt and the preset sample data, the large language model role is subjected to a few-sample learning, the correspondence between the preset large language model task and the subtask is learned, and a pre-optimized large language model is obtained, wherein the thought chain task prompt is the correspondence between the preset large language model task and the subtask obtained based on the thought chain principle.

[0057] It is understandable that in order to enable the large language model to more accurately understand the user's image requirements and optimize according to the initial prompt words input by the user, the large language model needs to be trained with examples.

[0058] It should be noted that the thought chain task prompt is that the relevant personnel disassemble the preset large language model task based on the thought chain principle to obtain the correspondence between the preset large language model task and the associated subtasks, as well as the specific execution steps of the subtasks. Then, based on the correspondence between the preset large language model task and the associated subtasks, as well as the specific execution steps of the subtasks, the thought chain guidance prompt words for the large language model are constructed.

[0059] Specifically, in this embodiment, according to the sample data of the thinking chain task prompts and the preset task decomposition examples, small sample example training is performed on each role of the large language model, so that each role of the large language model can learn and understand the solution logic of the preset large language model task, and at the same time learn how to solve the preset large language model task according to this solution logic, so as to obtain an optimized large language model containing several large language model roles.

[0060] It should be understood that steps S210 to S220 may also be performed before step S110.

[0061] This embodiment provides a prompt word optimization method, by receiving an initial prompt word input by a user; According to the preset large language model task, several large language model roles are constructed; based on the thought chain task prompt and the preset sample data, a few-sample learning is performed on the large language model role to learn the correspondence between the preset large language model task and the subtask, and a pre-optimized large language model is obtained, wherein the thought chain task prompt is the correspondence between the preset large language model task and the subtask obtained based on the thought chain principle; based on the pre-optimized large language model and combined with the retrieval enhancement generation principle, the initial prompt word is optimized to generate a first prompt word.

[0062] Through the above scheme, this embodiment pre-optimizes the large language model, so that the optimized large language model can optimize the prompt words according to the initial prompt words input by the user, helping the user to use the image generation large model to generate higher quality pictures, and at the same time lowering the user's usage threshold by lowering the prompt word construction threshold of the image generation large model.

[0063] Based on the first embodiment and / or the second embodiment of the present application, in the third embodiment of the present application, the same or similar contents as those in the first embodiment and / or the second embodiment can refer to the above description, and will not be described in detail later. Figure 3 , before the above step A03, it also includes steps S310~S330: Step S310, collecting professional prompt words for generating a large model from a plurality of images; Step S320, combining the professional prompt words to form a plurality of preset style templates; Step S330: Based on the user data analysis and the template style characteristics, the plurality of preset style templates are classified to obtain a preset style template library.

[0064] It is understandable that the prompt word optimization device needs to collect style templates in advance to build a preset style template library, so as to optimize the prompt words according to the preset style template library, thereby improving the output quality of the user's use of images to generate large models.

[0065] Specifically, the prompt word optimization device needs to collect professional terms, keywords or phrases that can guide the image generation model to produce specific visual effects or image content from community forums and professional blogs in the field of image generation, that is, professional prompt words. Professional prompt words often have significant style characteristics. Professional prompt words can be descriptive of the environment (such as a clear sky), painting style (Impressionist style), atmosphere and emotion (such as a warm family atmosphere), with aperture, focal length, etc.

[0066] Then, the collected professional prompt words are combined according to common image elements and structures, and the unique style characteristics of the professional prompt words are used as labels to identify several prompt word style templates.

[0067] It is understandable that in order to achieve more accurate optimization of prompt words, the preset style templates can be used for classification, thereby improving the retrieval accuracy of the large language model.

[0068] Specifically, we first generate a large model of behavioral data based on historically collected user usage images, conduct data analysis and processing, understand user preferences, needs, and habits, and perform a primary classification of the preset style templates based on user preferences, needs, and habits. For example, we classify the preset style templates according to user usage needs into realism, art, marketing, design, etc.

[0069] Then, a finer-grained image scene division is performed according to the labels of the aforementioned preset style templates, and the preset style templates under the first-level classification are further divided into different categories, and finally a preset style template database of the second-level classification is obtained.

[0070] The present embodiment provides a prompt word optimization method, which includes receiving an initial prompt word input by a user; translating the initial prompt word into English and expanding it based on a pre-optimized large language model to obtain a second prompt word; reordering the second prompt word based on the pre-optimized large language model and a preset prompt word sorting rule to obtain a third prompt word; collecting professional prompt words of a large model generated by a plurality of images; combining the professional prompt words to form a plurality of preset style templates; classifying the plurality of preset style templates based on user data analysis and template style characteristics to obtain a preset style template library; retrieving a style template corresponding to the third prompt word based on the pre-optimized large language model and the preset style template library using a retrieval enhancement generation principle; applying the style template to the third prompt word to generate a first prompt word.

[0071] Through the above scheme, this embodiment pre-builds a prompt word style template database with secondary classification, so that the prompt word input by the user can be optimized using the style template database later, so that the generated picture of the user using the image to generate a large model has obvious style characteristics, thereby improving the output quality.

[0072] Based on the third embodiment of the present application, in the fourth embodiment of the present application, the same or similar contents as those in the third embodiment can be referred to the above description, and will not be described in detail later. Figure 4 , the step A03 includes steps S410 to S450: Step S410, extracting text from a plurality of preset style templates in a preset style template library, and performing text segmentation on the text to obtain text units; Specifically, in order to ensure that the large language model can understand and use the style templates for matching retrieval, it is necessary to convert several preset style templates in the preset style template library into a processable text format for text extraction, such as JSON format.

[0073] At the same time, the extracted text needs to be segmented to obtain independent text units to ensure that the text length is within the range that the large language model can handle, improve the accuracy of the large language model's answers, and the degree of matching between the style template and the prompt word.

[0074] Step S420, embedding the text unit using a word embedding model to obtain a first text vector; Step S430, embedding the third prompt word using a word embedding model to obtain a second text vector; Specifically, the word embedding model is used to perform vector retrieval and matching between the prompt word and the style template. First, the prompt word optimization device needs to use the word embedding model to vectorize the text unit corresponding to the style template and convert it into a first text vector in a high-dimensional space. Then, the word embedding model is used to vectorize the third prompt word and convert it into a second text vector in a high-dimensional space. Among them, the word embedding model includes but is not limited to Word2Vec, BERT (BidirectionalEncoder Representations from Transformers) and other models. In this embodiment, the BGE model is preferably used for word embedding operations.

[0075] Step S440, based on the pre-optimized large language model and the preset similarity rule, performing vector matching retrieval on the first text vector and the second text vector to obtain a retrieval result; Step S450: obtaining a style template associated with the first text vector having the highest similarity based on the search result.

[0076] The prompt word optimization device uses the vector library composed of the first text vector as the retrieval knowledge base based on the retrieval enhancement generation principle. Through the pre-optimized large language model, firstly, according to the primary classification of the style template, the associated first text vectors of several style templates in the primary classification are matched.

[0077] Then, the similarity or matching degree between the first text vector and the second text vector is measured in combination with a preset similarity rule, thereby obtaining a search result. The preset similarity rule is a vector similarity calculation method, such as a vector cosine similarity calculation method, a Euclidean distance calculation method, and the like.

[0078] Finally, in the search results, the style template associated with the first text vector with the highest similarity score is returned.

[0079] The present embodiment provides a prompt word optimization method, which includes receiving an initial prompt word input by a user; translating the initial prompt word into English and expanding it based on a pre-optimized large language model to obtain a second prompt word; reordering the second prompt word based on the pre-optimized large language model and a preset prompt word sorting rule to obtain a third prompt word; extracting text from a plurality of preset style templates in a preset style template library, and performing text segmentation on the text to obtain text units; performing word embedding on the text unit using a word embedding model to obtain a first text vector; performing word embedding on the third prompt word using a word embedding model to obtain a second text vector; performing vector matching retrieval on the first text vector and the second text vector based on the pre-optimized large language model and a preset similarity rule to obtain a retrieval result; obtaining a style template associated with the first text vector with the highest similarity based on the retrieval result; applying the style template to the third prompt word to generate a first prompt word.

[0080] Through the above scheme, this embodiment intelligently matches the style templates to introduce professional prompt words into the user's original prompt words, thereby improving the quality of raw images without affecting the user's original intention, helping users to use the image generation model to generate higher quality images, and at the same time lowering the user's usage threshold by lowering the prompt word construction threshold of the image generation model.

[0081] Based on the first embodiment and / or the second embodiment and / or the third embodiment and / or the fourth embodiment of the present application, in the fifth embodiment of the present application, the same or similar contents as those in the first embodiment, the second embodiment, the third embodiment and / or the fourth embodiment can refer to the above description and will not be described in detail later. Figure 5 , after the above step 120, further comprising steps S510 to S540: Step S510, based on the pre-optimized large language model and the preset sensitive word database, the first prompt word is reviewed for sensitive words to obtain a first review result; Step S520, when the first audit result is successful, input the first prompt word into the image generation model to generate a picture to obtain a generated image; Understandably, in order to avoid compliance issues with prompt words and reduce resource waste of large language models, it is necessary to conduct sensitive word review on the optimized prompt words.

[0082] It should be noted that the preset sensitive word database is a sensitive word database pre-set by relevant personnel based on actual needs.

[0083] Specifically, after obtaining the optimized prompt word, that is, the first prompt word, the prompt word optimization device needs to use the pre-optimized large language model to conduct a sensitive word review on the banned characters and words specified by the relevant departments in the initial prompt word in the preset sensitive word database to obtain the review result, that is, the first review result.

[0084] Only when the first audit result is a successful audit, the first prompt word is input into the image generation model for picture generation to obtain a generated image, so as to subsequently tune the prompt word optimization method based on the generated image, while avoiding the initial prompt word containing sensitive words, thereby affecting the subsequent prompt word optimization.

[0085] Step S530, collecting the user's quality evaluation of the first prompt word and the generated image to obtain quality feedback data; Step S540: Continuously optimize the large language model based on the quality feedback data.

[0086] It should be understood that the prompt word optimization device can adjust the prompt word optimization method according to the quality of the output of the optimized prompt words when applied to the image generation model.

[0087] Specifically, the prompt word optimization device first collects the first prompt word and the generated image obtained by inputting the first prompt word into the image generation model, and performs data cleaning and classification operations to obtain a data set. After the data set is cleaned and classified, a higher quality data set can be further obtained through manual review.

[0088] Then, we collect users’ quality evaluations on the dataset, i.e., the quality scores of the first prompt word and the generated image, and fine-tune the prompt word optimization method based on the quality score results, thereby further improving the prompt word optimization capability of the large language model.

[0089] It is understandable that in order to ensure the accuracy of the prompt word optimization and reduce the resource waste of the large language model, after the above step 110, steps B01 to B02 are also included: Step B01, based on a pre-optimized large language model and a preset sensitive word database, the initial prompt word is reviewed for sensitive words to obtain a second review result; Step B02, when the second audit result is successful audit, execute the following steps: based on the pre-optimized large language model, combined with retrieval enhancement generation, the initial prompt word is optimized to generate a first prompt word.

[0090] Specifically, after receiving the initial prompt word input by the user, it is first necessary to use the pre-optimized large language model to conduct a sensitive word review on the banned characters and words specified by the relevant departments in the initial prompt word in the preset sensitive word database to obtain the review result, that is, the second review result.

[0091] Only when the second audit result is successful, step S120 is executed to avoid the initial prompt word containing sensitive words, thereby affecting the subsequent prompt word optimization.

[0092] The present embodiment provides a prompt word optimization method, which includes receiving an initial prompt word input by a user; optimizing the initial prompt word based on a pre-optimized large language model and in combination with the retrieval enhancement generation principle to generate a first prompt word; performing a sensitive word review on the first prompt word based on the pre-optimized large language model and a preset sensitive word database to obtain a first review result; when the first review result is a successful review, inputting the first prompt word into an image generation large model to generate a picture to obtain a generated image; collecting the user's quality evaluation of the first prompt word and the generated image to obtain quality feedback data; and continuously optimizing the large language model based on the quality feedback data.

[0093] Through the above scheme, this embodiment avoids image compliance issues by conducting sensitive word review, and at the same time collects user feedback to adjust the prompt word optimization method, thereby lowering the threshold for using large image generation models, assisting users in generating high-quality prompt words, saving users' learning costs, and bringing a better user experience.

[0094] For example, to help understand the implementation process of the prompt word optimization method obtained by combining this embodiment with the above-mentioned embodiment 5, please refer to Figure 6 , Figure 6 An example flow chart of a prompt word optimization method is provided, specifically: First, the prompt words entered by the user are reviewed for sensitive words. After the user enters the initial text and image prompt words according to the needs, the large language model combined with the preset sensitive word library will be used to detect the banned characters and words in the initial prompt words.

[0095] Then, the initial prompt word is translated from Chinese to English using the large language model and prompt word engineering. Since the large language model training set is relatively new, after prompting the large language model through the prompt word engineering, it can understand the user's intention and give an accurate translation based on the context. It can accurately translate some polysemous and uncommon words.

[0096] Then, the translated English prompt words are expanded using the large language model. With the help of the large number of parameters of the large language model, after understanding the user's initial intention, the prompt word writing methods and techniques of the image generation large model can be combined, and a series of large language model tuning methods can be used to guide it to expand the prompt words, such as prompt word engineering, RAG, etc., to guide the large language model to describe different elements of the entire picture based on the existing prompt words, such as theme, environment, atmosphere, etc., so as to optimize the prompt word content and thus improve the richness of the picture.

[0097] Then, the expanded prompt words are reordered using the large language model and prompt word engineering. Since the large model for generating images has a default weight for prompt words, the default weight of prompt words that are closer to the front is greater, so by reordering the expanded prompt words, the primary and secondary status of each prompt word element can be ensured. After guiding the large language model, the elements in the optimized prompt words can be identified and sorted, thereby improving the relevance between the image and the prompt word.

[0098] Then, we use RAG and large language model retrieval to recommend style templates. By leveraging the long text understanding ability of the large language model and combining it with optimized prompt words, we search and recommend in the preset style templates, thereby lowering the threshold for using professional prompt words and improving the quality of the output. Users can choose whether to use the recommended style template to apply the prompt words.

[0099] The optimized prompt words are then tested for sensitive words again. Due to compliance requirements and the characteristics of the large language model, in order to ensure that the final prompt words are safe and usable, they are reviewed for sensitive words again and then sent to the image generation large model for image generation.

[0100] Finally, after the image is generated, the prompt word optimization device prompts the user to rate the prompt word and the image. By collecting user reviews and manually reviewing high- and low-scoring data, classifying data, and cleaning them, a batch of user feedback samples are obtained. By using the samples to fine-tune the large language model, the prompt word optimization effect of the large language model is continuously optimized.

[0101] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the prompt word optimization method of the present application. More simple transformations based on this technical concept are all within the protection scope of the present application.

[0102] The present application provides a prompt word optimization device, which includes: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the prompt word optimization method in the above-mentioned implementation example.

[0103] Reference below Figure 7 , which shows a schematic diagram of the structure of a prompt word optimization device suitable for implementing the embodiment of the present application. The prompt word optimization device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 7 The prompt word optimization device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0104] like Figure 7 As shown, the prompt word optimization device may include a processing device 1001 (such as a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the prompt word optimization device are also stored. The processing device 1001, ROM1002 and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the prompt word optimization device to communicate with other devices wirelessly or by wire to exchange data. Although the prompt word optimization device with various systems is shown in the figure, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have alternatively.

[0105] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0106] The prompt word optimization device provided by the present application adopts the prompt word optimization method in the above embodiment, which can solve the technical problem of how to lower the threshold of prompt word construction of the image generation large model and improve the image generation quality. Compared with the prior art, the beneficial effects of the prompt word optimization device provided by the present application are the same as the beneficial effects of the prompt word optimization method provided by the above embodiment, and the other technical features in the prompt word optimization device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0107] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0108] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0109] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the prompt word optimization method in the above-mentioned embodiment.

[0110] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

[0111] The computer-readable storage medium may be included in the prompt word optimization device; or may exist independently without being assembled into the prompt word optimization device.

[0112] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the prompt word optimization device, the prompt word optimization device: receives an initial prompt word input by a user; optimizes the initial prompt word based on a pre-optimized large language model and in combination with the retrieval enhancement generation principle to generate a first prompt word.

[0113] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0114] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0115] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0116] The readable storage medium provided by the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned prompt word optimization method, and can solve the technical problem of how to lower the threshold for constructing prompt words for large image generation models and improve the quality of image generation. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as the beneficial effects of the prompt word optimization method provided by the above-mentioned embodiment, and will not be repeated here.

[0117] An embodiment of the present application provides a computer program product, including a computer program, which implements the steps of the prompt word optimization method as described above when the computer program is executed by a processor.

[0118] The computer program product provided by this application can solve the technical problem of how to lower the threshold for constructing prompt words for large image generation models and improve the quality of image generation. Compared with the prior art, the beneficial effects of the computer program product provided by the embodiment of this application are the same as the beneficial effects of the prompt word optimization method provided by the above embodiment, which will not be repeated here.

[0119] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent processing scope of the present application.

Claims

1. A prompt word optimization method, characterized in that: The method comprises: Receive the initial prompt word input by the user; Based on a pre-optimized large language model and in combination with the retrieval enhancement generation principle, the initial prompt word is optimized to generate a first prompt word.

2. The method according to claim 1, characterized in that Before the step of optimizing the initial prompt word based on the pre-optimized large language model and combining the retrieval enhancement generation principle to generate the first prompt word, the step further includes: According to the preset large language model tasks, build several large language model roles; Based on the thought chain task prompts and preset sample data, a few sample learning is performed on the large language model role to learn the correspondence between the preset large language model tasks and subtasks, and obtain a pre-optimized large language model, wherein the thought chain task prompts are the correspondence between the preset large language model tasks and the subtasks obtained based on the thought chain principle.

3. The method according to claim 2, characterized in that The step of optimizing the initial prompt word based on the pre-optimized large language model and combining the retrieval enhancement generation principle to generate the first prompt word includes: Based on the pre-optimized large language model, the initial prompt word is translated from Chinese to English and expanded to obtain a second prompt word; Based on the pre-optimized large language model and the preset prompt word sorting rule, the second prompt word is re-sorted to obtain a third prompt word; Based on the pre-optimized large language model and the preset style template library, the style template corresponding to the third prompt word is retrieved using the retrieval enhancement generation principle; The style template is applied to the third prompt word to generate a first prompt word.

4. The method according to claim 3, characterized in that Before the step of retrieving the style template corresponding to the third prompt word based on the pre-optimized large language model and the preset style template library by using the retrieval enhancement generation principle, the method includes: Collect several images to generate professional tips for a large model; Combining the professional prompt words to form a plurality of preset style templates; Based on the user data analysis and the template style characteristics, the plurality of preset style templates are classified to obtain a preset style template library.

5. The method according to claim 4, characterized in that The step of retrieving the style template corresponding to the third prompt word based on the pre-optimized large language model and the preset style template library and using the retrieval enhancement generation principle includes: Extracting text from a plurality of preset style templates in a preset style template library, and performing text segmentation on the text to obtain text units; Using a word embedding model to embed the text unit into a word to obtain a first text vector; Using a word embedding model to embed the third prompt word to obtain a second text vector; Based on the pre-optimized large language model and the preset similarity rule, performing vector matching retrieval on the first text vector and the second text vector to obtain a retrieval result; Based on the search results, a style template associated with the first text vector having the highest similarity is obtained.

6. The method according to claim 1, characterized in that After the step of optimizing the initial prompt word based on the pre-optimized large language model and combining the retrieval enhancement generation principle to generate the first prompt word, the method further includes: Based on the pre-optimized large language model and the preset sensitive word database, the first prompt word is reviewed for sensitive words to obtain a first review result; When the first audit result is successful, the first prompt word is input into the image generation model to generate a picture to obtain a generated image; Collecting user's evaluation of the quality of the first prompt word and the generated image to obtain quality feedback data; Based on the quality feedback data, the large language model is continuously optimized.

7. The method according to claim 1, characterized in that After the step of receiving the initial prompt word input by the user, the method further includes: Based on the pre-optimized large language model and the preset sensitive word database, the initial prompt word is reviewed for sensitive words to obtain a second review result; When the second audit result is successful, the following steps are executed: based on the pre-optimized large language model, combined with retrieval enhancement generation, the initial prompt word is optimized to generate a first prompt word.

8. A prompt word optimization device, characterized in that: The prompt word optimization device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the prompt word optimization method according to any one of claims 1 to 7.

9. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the prompt word optimization method according to any one of claims 1 to 7 are implemented.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the prompt word optimization method according to any one of claims 1 to 7 are implemented.

Citation Information

Cited By

  • Traditional Chinese medicine prescription data analysis method and device based on OCR (Optical Character Recognition) and large model

    CN119811698A

  • Large model sensitive content review method and device based on thinking chain

    CN119862872A

  • Information extraction task-oriented cue word design and optimization method and system

    CN120671823A