Image generation method and device, storage medium and electronic equipment

Abstract

The invention discloses an image generation method and device, a storage medium and electronic equipment, and relates to the technical field of intelligent medical treatment, artificial intelligence and finance, and the method comprises the steps: receiving input business data of a target user; retrieving a preset prompt text library and a preset image library by adopting a preset retrieval method based on the input business data to obtain a retrieval business data set; in response to a selection operation of a user on target retrieval service data in the retrieval service data set, querying a preset image generation model library based on the target retrieval service data to obtain a first target image generation model corresponding to the target retrieval service data; and based on the target retrieval service data, carrying out image generation by adopting the first target image generation model to obtain a first target image. According to the method, the business integrating degree of image generation and the satisfaction degree of a customer on the generated image can be improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image generating method, device, storage medium and electronic device. Background Art

[0002] With the development of artificial intelligence technology, image generation methods can support functions such as disease auxiliary diagnosis, health management, and remote consultation. Traditional business image generation methods use diffusion models to generate images, but the images generated by the diffsion model are highly diverse and contain a variety of content. It is difficult to quickly generate images that meet complex business requirements and cannot meet the needs of business scenarios well. Summary of the invention

[0003] In view of this, the present invention provides an image generation method, device, storage medium and electronic device, the main purpose of which is to solve the problem that the current image generation process cannot be well matched with the business scenario, resulting in the image generation result not meeting the business needs.

[0004] To solve the above problems, the present application provides an image generation method, comprising:

[0005] Receive input business data from target users;

[0006] Based on the input business data, a preset prompt text library and a preset image library are searched using a preset search method to obtain a search business data set;

[0007] In response to a user's selection operation on target retrieval business data in the retrieval business data set, querying a preset image generation model library based on the target retrieval business data to obtain a first target image generation model corresponding to the target retrieval business data;

[0008] The first target image generation model is used to generate an image based on the target retrieval business data to obtain a first target image.

[0009] Optionally, before searching the preset prompt text library and the preset image library using a preset search method based on the input business data to obtain the search business data set, the method further includes: constructing the preset prompt text library;

[0010] The step of constructing the preset prompt text library specifically includes:

[0011] Constructing several initial prompt texts corresponding to the target business type;

[0012] Based on each of the initial prompt texts, a preset multilingual translation model is used to perform data expansion to obtain a prompt text set corresponding to the target business type, so as to obtain the prompt text set corresponding to each business type respectively;

[0013] The preset prompt text library is constructed based on each of the prompt text sets.

[0014] Optionally, before searching the preset prompt text library and the preset image library using a preset search method based on the input business data to obtain the search business data set, the method further includes: constructing the preset image library;

[0015] The constructing of the preset image library specifically includes:

[0016] Based on each of the prompt text sets, a preset stable diffusion model is used to generate an image to obtain an image data set corresponding to each of the business types;

[0017] Performing cluster analysis on each image data in the image data set corresponding to the same business category to obtain each cluster;

[0018] Scoring each of the clusters to obtain target clusters corresponding to the same business category;

[0019] The preset image library is constructed based on each of the target clusters.

[0020] Optionally, the step of searching a preset prompt text library and a preset image library using a preset search method based on the input business data to obtain a search business data set specifically includes:

[0021] Extracting features from the input service data to obtain an input feature vector;

[0022] Performing feature extraction on each preset prompt text in the preset prompt text library to obtain each first feature vector;

[0023] Performing feature extraction on each image data in the preset image library to obtain each second feature vector;

[0024] Performing similarity calculation processing based on the input feature vector and each of the first feature vectors to obtain each first similarity;

[0025] Performing similarity calculation processing based on the input feature vector and each of the second feature vectors to obtain each second similarity;

[0026] The preset prompt texts and the preset images are screened based on the first similarities, the second similarities and a first preset threshold to obtain the retrieval service data set.

[0027] Optionally, the filtering of the preset prompt texts and the preset images based on the first similarities, the second similarities and the first preset threshold to obtain the retrieval service data set specifically includes:

[0028] Sort the preset prompt texts based on the first similarities to obtain a first search service data queue;

[0029] Sort the preset images based on the second similarities to obtain a second search service data queue;

[0030] The first retrieval service data in the first retrieval service data queue ranked first is screened based on the first preset threshold, and the second retrieval service data in the second retrieval service data queue ranked first is screened based on the first preset threshold to obtain the retrieval service data set.

[0031] Optionally, in response to a user's selection operation on target retrieval business data in the retrieval business data set, before querying a preset image generation model library based on the target retrieval business data to obtain a first target image generation model corresponding to the target retrieval business data, the method further includes: constructing a preset image generation model library;

[0032] The construction of the preset image generation model library specifically includes:

[0033] Based on the preset prompt texts in the preset prompt text library, a preset low-rank adaptive transfer model is used to perform model training to obtain a plurality of first image generation models;

[0034] Based on the image data in the preset image library, a preset low-rank adaptive migration model is used to perform model training to obtain a plurality of second image generation models;

[0035] A model library is constructed based on each of the first image generation models and each of the second image generation models to obtain the preset image generation model library.

[0036] Optionally, after the first target image generation model is used to generate an image based on the target retrieval service data to obtain the first target image, the method further includes:

[0037] In response to an optimization operation performed by the target user on the first target image, querying the preset image generation model library based on the first target image to obtain a second image generation model corresponding to the first target image;

[0038] Based on the first target image, the second image generation model is used to generate an image to obtain a second target image.

[0039] To solve the above problems, the present application provides an image generating device, comprising:

[0040] A receiving module, used to receive input business data from a target user;

[0041] A retrieval module, used to search a preset prompt text library and a preset image library based on the input business data by using a preset retrieval method to obtain a retrieval business data set;

[0042] A query module, configured to query a preset image generation model library based on the target retrieval business data in response to a user's selection operation on the target retrieval business data in the retrieval business data set, and obtain a first target image generation model corresponding to the target retrieval business data;

[0043] A generation module is used to generate an image using the first target image generation model based on the target retrieval business data to obtain a first target image.

[0044] In order to solve the above problem, the present application provides a storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned image generation method are implemented.

[0045] To solve the above problem, the present application provides an electronic device, which at least includes a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps of the above-mentioned image generation method when executing the computer program in the memory.

[0046] Beneficial effects of the present application: The present application receives input business data from a target user; the present application can generate a final first target image based on the input image data or text data, can realize image generation for multi-dimensional input data, and improve practicality. Based on the input business data, a preset retrieval method is used to retrieve the preset prompt text library and the preset image library to obtain a retrieval business data set; in response to the user's selection operation of the target retrieval business data in the retrieval business data set, the preset image generation model library is queried based on the target retrieval business data to obtain a first target image generation model corresponding to the target retrieval business data; the present application pre-constructs a preset prompt text library, a preset image library, and a preset image generation model library in combination with each business type, so that the business fit can be improved when the user generates images for the target business type. Based on the target retrieval business data, the first target image generation model is used to generate an image to obtain a first target image. The method of the present application can improve the user's satisfaction with the generated image when generating images for the target business type application scenario.

[0047] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0049] Figure 1 is a schematic diagram of an application environment of an image generation method in an embodiment of the present application;

[0050] Figure 2 A schematic diagram of a process of an image generation method provided in an embodiment of the present application is shown;

[0051] Figure 3 A schematic diagram showing a flow chart of an image generating method provided by another embodiment of the present application is shown;

[0052] Figure 4 A structural block diagram of an image generation method provided by another embodiment of the present application is shown. DETAILED DESCRIPTION

[0053] Various aspects and features of the present application are described herein with reference to the accompanying drawings.

[0054] It should be understood that various modifications may be made to the embodiments of the present application. Therefore, the above description should not be considered as limiting, but only as an example of an embodiment. Other modifications within the scope and spirit of the present application will occur to those skilled in the art.

[0055] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the present application and, together with the general description of the present application given above and the detailed description of the embodiments given below, serve to explain the principles of the present application.

[0056] These and other characteristics of the present application will become apparent from the following description of a preferred form of embodiment given as a non-limiting example with reference to the accompanying drawings.

[0057] It should also be understood that although the present application has been described with reference to some specific examples, those skilled in the art will be able to readily implement many other equivalent forms of the present application.

[0058] The above and other aspects, features and advantages of the present application will become more apparent in view of the following detailed description when taken in conjunction with the accompanying drawings.

[0059] Specific embodiments of the present application are described hereinafter with reference to the accompanying drawings; however, it should be understood that the embodiments applied for are merely examples of the present application, which may be implemented in a variety of ways. Well-known and / or repeated functions and structures are not described in detail to avoid unnecessary or redundant details that obscure the present application. Therefore, the specific structural and functional details applied for herein are not intended to be limiting, but merely serve as a basis and representative basis for the claims to teach those skilled in the art to use the present application in a variety of ways with substantially any suitable detailed structure.

[0060] This specification may use the phrases "in one embodiment," "in another embodiment," "in yet another embodiment," or "in other embodiments," all of which may refer to one or more of the same or different embodiments according to the present application.

[0061] The image generation method provided by the embodiment of the present invention can be applied in Figure 1 In an application environment, the client communicates with the server through a network. The server can receive input business data of the target user through the client; based on the input business data, a preset retrieval method is used to retrieve the preset prompt text library and the preset image library to obtain a retrieval business data set; in response to the user's selection operation on the target retrieval business data in the retrieval business data set, a preset image generation model library is queried based on the target retrieval business data to obtain a first target image generation model corresponding to the target retrieval business data; based on the target retrieval business data, the first target image generation model is used to generate an image to obtain a first target image. The present application improves the business fit of generating pictures for each business scenario, and can improve user satisfaction with the generated images.

[0062] The present application embodiment provides an image generation method, such as Figure 2 As shown, including:

[0063] Step S101: receiving input service data of a target user;

[0064] In the specific implementation process of this step, the image generation method of the present application can be applied to the medical cloud platform. Medical cloud refers to the use of "cloud computing" to create a medical and health service cloud platform based on new technologies such as cloud computing, mobile technology, multimedia, 4G communication, big data, and the Internet of Things, combined with medical technology, to achieve the sharing of medical resources and the expansion of medical scope. Because of the combination of cloud computing technology, medical cloud improves the efficiency of medical institutions and facilitates residents to seek medical treatment. For example, the current hospital appointment registration, electronic medical records, medical insurance, etc. are all products of the combination of cloud computing and the medical field. The medical cloud also has the advantages of data security, information sharing, dynamic expansion, and overall layout. At the same time, this application can also be applied to the fields of financial securities technology, insurance technology, etc. The input business data includes data types such as image data and text data; when the user inputs image data, the image data can be of any type, such as: scenery, people, pets, objects, etc.; when the user inputs text data, this text data can be a description of the picture, an expression of a scene or emotion, etc.

[0065] Step S102: searching a preset prompt text library and a preset image library using a preset search method based on the input business data to obtain a search business data set;

[0066] In the specific implementation process of this step, a large-scale image-text pre-training model based on contrastive learning can be used to encode the input image data or text data, and calculate their similarity with each entry in the preset prompt text library and the preset image library, namely, CLIP Score. CLIP Score reflects the semantic matching degree between the input content and the content in the library. According to each similarity, the top N entries that best match the input content are retrieved from the preset prompt text library and the preset image library, where N is the preset number of recommendations.

[0067] Step S103: in response to the user's selection operation on the target retrieval business data in the retrieval business data set, querying a preset image generation model library based on the target retrieval business data to obtain a first target image generation model corresponding to the target retrieval business data;

[0068] During the specific implementation of this step, in response to the user's selection operation on the target retrieval business data in the retrieval business data set, a pre-built preset image generation model library is called to obtain a first target image generation model corresponding to the target retrieval business data.

[0069] Step S104: Based on the target retrieval business data, the first target image generation model is used to generate an image to obtain a first target image.

[0070] In the specific implementation process of this step, the first target image generation model can be a preset low-rank adaptive migration model LoRA model, and the LoRA model is used to generate the corresponding first target image according to the target retrieval business data. Since the preset low-rank adaptive migration model LoRA model has been optimized for specific business scenarios, the generated results are more in line with business needs.

[0071] The present application receives input business data from a target user; the present application can generate a final first target image based on the input image data or text data, and can realize image generation for multi-dimensional input data, thereby improving practicality. Based on the input business data, a preset retrieval method is used to retrieve a preset prompt text library and a preset image library to obtain a retrieval business data set; in response to a user's selection operation of the target retrieval business data in the retrieval business data set, a preset image generation model library is queried based on the target retrieval business data to obtain a first target image generation model corresponding to the target retrieval business data; the present application pre-constructs a preset prompt text library, a preset image library, and a preset image generation model library in combination with each business type, so that the business fit can be improved when the user generates an image for the target business type. The first target image generation model is used to generate an image based on the target retrieval business data to obtain a first target image. The method of the present application can improve the user's satisfaction with the generated image when generating an image for the target business type application scenario.

[0072] Another embodiment of the present application provides another image generation method, such as Figure 3 As shown, including:

[0073] Step S201: receiving input service data of a target user;

[0074] In the specific implementation of this step, the input business data includes data types such as image data and text data; when the user inputs image data, the image data can be of any type, such as: scenery, people, pets, objects, cases, etc.; when the user inputs text data, the text data can be a description of the picture, an expression of a scene or emotion, etc. This application can be applied to medical cloud platforms, artificial intelligence, finance and other application scenarios.

[0075] Step S202: constructing the preset prompt text library;

[0076] In the specific implementation process of this step, several initial prompt texts corresponding to the target business type are constructed; the business type can be insurance types of different types, for example: A health insurance, B pension insurance; the business type can also be a pet business type, etc. The present application can construct a preset prompt text library for any business scenario. The target business type is any one of the various business types. Based on each of the initial prompt texts, a preset multilingual translation model is used to perform data expansion to obtain a prompt text set corresponding to the target business type, so as to obtain the prompt text set corresponding to each business type; the preset multilingual translation model can be a SeamlessM4T model, an M2M-100 model and other models. The present application does not limit the preset multilingual translation model. Any multilingual translation model that can achieve the purpose of text expansion can meet the requirements of the present application for data expansion of the text, and the preset prompt text library is constructed based on each of the prompt text sets.

[0077] Step S203: constructing a preset image library;

[0078] In the specific implementation process of this step, based on each of the prompt text sets, a preset stable diffusion model is used to generate images to obtain image data sets corresponding to each of the business types; specifically, a preset stable diffusion model (SD) model can be used to generate images. During the generation process, the parameters of the denoising generation process are controlled by the guidance scale, etc. to increase diversity, and about 10,000 to 30,000 images can be generated according to the complexity of the business. Cluster analysis is performed on each image data in the image data set corresponding to the same business category to obtain each cluster cluster; each cluster cluster is scored to obtain each target cluster cluster corresponding to the same business category; specifically, a first predetermined number of image data corresponding to each cluster cluster is randomly extracted from each cluster cluster image to obtain each of the cluster clusters, and each of the image data is scored based on business requirements to obtain a first score value corresponding to each of the image data; the mean value calculation process is performed based on each of the first score values ​​corresponding to the same cluster cluster to obtain the score value corresponding to each of the cluster clusters, and the second predetermined number of cluster clusters with the highest score value are determined as the target cluster clusters corresponding to the same business category. For example: they can be divided into one category according to the similar clip score values, such as 0.1 and 0.11 are one category. Simple processing can divide the similarity clip score value range into 20-30 clusters, and adjust the clipscore range of each category based on category balance. Then, the mean calculation is performed according to the first scoring values ​​corresponding to the same cluster to obtain the scoring values ​​corresponding to each cluster, and the second predetermined number of clusters with the highest scoring values ​​are determined as the target clusters corresponding to the same business category. The second predetermined number of clusters is retained from the 20-30 clusters as the target clusters. The second preset number can be determined as 10, and the second preset number can be set according to actual needs. The preset image library is constructed based on each target cluster.

[0079] Step S204: searching a preset prompt text library and a preset image library using a preset search method based on the input business data to obtain a search business data set;

[0080] In the specific implementation process of this step, feature extraction is performed on the input business data to obtain an input feature vector; feature extraction is performed on each preset prompt text in the preset prompt text library to obtain each first feature vector; feature extraction is performed on each image data in the preset image library to obtain each second feature vector; similarity calculation is performed based on the input feature vector and each first feature vector to obtain each first similarity; similarity calculation is performed based on the input feature vector and each second feature vector to obtain each second similarity; each preset prompt text and each preset image are screened based on each first similarity, each second similarity and a first preset threshold to obtain the retrieval business data set. Specifically, each preset prompt text is sorted based on each first similarity to obtain a first retrieval business data queue; each preset image is sorted based on each second similarity to obtain a second retrieval business data queue; the first retrieval business data in the first retrieval business data queue ranked first is screened based on the first preset threshold, and the second retrieval business data in the second retrieval business data queue ranked first is screened based on the first preset threshold to obtain the retrieval business data set.

[0081] Step S205: constructing a preset image generation model library;

[0082] During the specific implementation of this step, a preset low-rank adaptive transfer model is used to perform model training based on the preset prompt text in the preset prompt text library to obtain several first image generation models; the preset low-rank adaptive transfer model can be a LoRA model. A preset low-rank adaptive transfer model is used to perform model training based on the image data in the preset image library to obtain several second image generation models; a model library is constructed based on each of the first image generation models and each of the second image generation models to obtain the preset image generation model library. The present application adopts a preset low-rank adaptive transfer model to increase the scalability of the present application. The present application only requires a small amount of business communication and a small amount of R&D investment to be copied to different business scenarios. After the system has been running for a period of time, based on the system preset image generation model library and the preset prompt text library, business personnel can expand it according to other business scenarios, which is highly practical.

[0083] Step S206: In response to the user's selection operation on the target retrieval business data in the retrieval business data set, querying a preset image generation model library based on the target retrieval business data to obtain a first target image generation model corresponding to the target retrieval business data;

[0084] During the specific implementation of this step, in response to the user's selection operation on the target retrieval business data in the retrieval business data set, a pre-built preset image generation model library is called to obtain a first target image generation model corresponding to the target retrieval business data.

[0085] Step S207: generating an image using the first target image generation model based on the target retrieval service data to obtain a first target image;

[0086] In the specific implementation process of this step, the first target image generation model can be a preset low-rank adaptive migration model LoRA model, and the LoRA model is used to generate the corresponding first target image according to the target retrieval business data. Since the preset low-rank adaptive migration model LoRA model has been optimized for specific business scenarios, the generated results are more in line with business needs. When the user is satisfied with the generation effect of the first target image, the first target image can be downloaded and used.

[0087] Step S208: In response to the optimization operation of the target user on the first target image, query the preset image generation model library based on the first target image to obtain a second image generation model corresponding to the first target image;

[0088] In the specific implementation of this step, when the user is not satisfied with the image generation effect of the first target image, personalized image generation can be performed based on the first target image. Specifically, the preset image generation model library is queried based on the first target image to obtain a second image generation model corresponding to the first target image. Because the same business category corresponds to multiple categories of target clusters, and each target cluster corresponds to multiple preset image generation models, multiple optimization operations and personalized parameter configurations can be used to generate image data that meets the needs of the user and meets the current business scenario requirements.

[0089] Step S209: Based on the first target image, the second image generation model is used to generate an image to obtain a second target image.

[0090] In the specific implementation process of this step, the second target image generation model can be a preset low-rank adaptive migration model LoRA model, and the second image generation model is used to generate a corresponding second target image according to the first target image. Since the preset low-rank adaptive migration model LoRA model has been optimized for specific business scenarios, the generated results are more in line with business needs. When the user is satisfied with the generation effect of the second target image, the second target image can be downloaded and used. Correspondingly, if the user is not satisfied with the currently generated second target image, the optimization operation can be continued until a business image that meets personalized needs is obtained.

[0091] The present application receives the input business data of the target user; constructs the preset prompt text library; constructs the preset image library; in order to solve the problem of low business compatibility of the generated image in the current image generation process, the present application pre-constructs a preset prompt text library and a preset image library including various business types, thereby greatly improving the business compatibility of the generated images in various business scenarios. Based on the input business data, the preset prompt text library and the preset image library are searched using a preset retrieval method to obtain a retrieval business data set; a preset image generation model library is constructed; by pre-constructing the image generation model library, combined with the specific business type, the scalability is strong, and when a new business category requires image generation, it can be implemented using the method of the present application without a large amount of R&D cost, and it is highly practical. In response to a user's selection operation on the target retrieval business data in the retrieval business data set, a preset image generation model library is queried based on the target retrieval business data to obtain a first target image generation model corresponding to the target retrieval business data; image generation is performed based on the target retrieval business data using the first target image generation model to obtain a first target image; in response to the target user's optimization operation on the first target image, the preset image generation model library is queried based on the first target image to obtain a second image generation model corresponding to the first target image; image generation is performed based on the first target image using the second image generation model to obtain a second target image.

[0092] Another embodiment of the present application provides a generating device, such as Figure 4 As shown:

[0093] Receiving module 1, used to receive input business data of the target user;

[0094] Retrieval module 2, used to retrieve a preset prompt text library and a preset image library based on the input business data using a preset retrieval method to obtain a retrieval business data set;

[0095] A query module 3, configured to query a preset image generation model library based on the target retrieval business data in response to a user's selection operation on the target retrieval business data in the retrieval business data set, and obtain a first target image generation model corresponding to the target retrieval business data;

[0096] The generating module 4 is used to generate an image by using the first target image generating model based on the target retrieval service data to obtain a first target image.

[0097] In the specific implementation process, the device also includes a preset prompt text library construction module, and the preset prompt text library construction module is specifically used to: construct a number of initial prompt texts corresponding to the target business type; based on each of the initial prompt texts, a preset multilingual translation model is used to perform data expansion to obtain a prompt text set corresponding to the target business type, so as to obtain the prompt text set corresponding to each business type; based on each of the prompt text sets, the preset prompt text library is constructed.

[0098] In the specific implementation process, the device also includes a preset image library construction module, which is specifically used to: generate images based on each of the prompt text sets using a preset stable diffusion model to obtain an image data set corresponding to each of the business types; perform cluster analysis on each image data in the image data set corresponding to the same business category to obtain each cluster cluster; score each of the cluster clusters to obtain each target cluster cluster corresponding to the same business category; and construct the preset image library based on each of the target cluster clusters.

[0099] In the specific implementation process, the retrieval module 2 is specifically used to: perform feature extraction on the input business data to obtain an input feature vector; perform feature extraction on each preset prompt text in the preset prompt text library to obtain each first feature vector; perform feature extraction on each image data in the preset image library to obtain each second feature vector; perform similarity calculation processing based on the input feature vector and each first feature vector to obtain each first similarity; perform similarity calculation processing based on the input feature vector and each second feature vector to obtain each second similarity; screen each preset prompt text and each preset image based on each first similarity, each second similarity and a first preset threshold to obtain the retrieved business data set.

[0100] In the specific implementation process, the retrieval module 2 is also used to: sort each of the preset prompt texts based on each of the first similarities to obtain a first retrieval business data queue; sort each of the preset images based on each of the second similarities to obtain a second retrieval business data queue; filter the first retrieval business data in the first retrieval business data queue that ranks first based on the first preset threshold and filter the second retrieval business data in the second retrieval business data queue that ranks first based on the first preset threshold to obtain the retrieval business data set.

[0101] During the specific implementation process, the device also includes a preset image generation model library construction module, and the preset image generation model library construction module is specifically used to: perform model training based on the preset prompt text in the preset prompt text library using a preset low-rank adaptive transfer model to obtain a number of first image generation models; perform model training based on the image data in the preset image library using a preset low-rank adaptive transfer model to obtain a number of second image generation models; and construct a model library based on each of the first image generation models and each of the second image generation models to obtain the preset image generation model library.

[0102] During the specific implementation process, the device also includes an optimization module, which is specifically used to: in response to the target user's optimization operation on the first target image, query the preset image generation model library based on the first target image to obtain a second image generation model corresponding to the first target image; and use the second image generation model to generate an image based on the first target image to obtain a second target image.

[0103] The present application receives input business data from a target user; the present application can generate a final first target image based on the input image data or text data, and can realize image generation for multi-dimensional input data, thereby improving practicality. Based on the input business data, a preset retrieval method is used to retrieve a preset prompt text library and a preset image library to obtain a retrieval business data set; in response to a user's selection operation of the target retrieval business data in the retrieval business data set, a preset image generation model library is queried based on the target retrieval business data to obtain a first target image generation model corresponding to the target retrieval business data; the present application pre-constructs a preset prompt text library, a preset image library, and a preset image generation model library in combination with each business type, so that the business fit can be improved when the user generates an image for the target business type. The first target image generation model is used to generate an image based on the target retrieval business data to obtain a first target image. The method of the present application can improve the user's satisfaction with the generated image when generating an image for the target business type application scenario.

[0104] Another embodiment of the present application provides a storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the following method steps are implemented:

[0105] Step 1: Receive input business data from the target user;

[0106] Step 2: Based on the input business data, a preset retrieval method is used to search the preset prompt text library and the preset image library to obtain a retrieval business data set;

[0107] Step 3: in response to a user's selection operation on target retrieval business data in the retrieval business data set, query a preset image generation model library based on the target retrieval business data to obtain a first target image generation model corresponding to the target retrieval business data;

[0108] Step 4: Based on the target retrieval business data, the first target image generation model is used to generate an image to obtain a first target image.

[0109] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0110] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0111] The specific implementation process of the above method steps can refer to the embodiments of any of the above-mentioned image generation methods, and this embodiment will not be repeated here.

[0112] The present application receives input business data from a target user; the present application can generate a final first target image based on the input image data or text data, and can realize image generation for multi-dimensional input data, thereby improving practicality. Based on the input business data, a preset retrieval method is used to retrieve a preset prompt text library and a preset image library to obtain a retrieval business data set; in response to a user's selection operation of the target retrieval business data in the retrieval business data set, a preset image generation model library is queried based on the target retrieval business data to obtain a first target image generation model corresponding to the target retrieval business data; the present application pre-constructs a preset prompt text library, a preset image library, and a preset image generation model library in combination with each business type, so that the business fit can be improved when the user generates an image for the target business type. The first target image generation model is used to generate an image based on the target retrieval business data to obtain a first target image. The method of the present application can improve the user's satisfaction with the generated image when generating an image for the target business type application scenario.

[0113] Another embodiment of the present application provides an electronic device, which may be a server, and the electronic device includes a processor, a memory, a network interface, and a database connected via a system bus. The processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile and / or volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the electronic device is used to communicate with an external client via a network connection. When the electronic device program is executed by the processor, it implements a function or step on the server side of an image generation method.

[0114] In one embodiment, an electronic device is provided, which may be a client. The electronic device includes a processor, a memory, a network interface, a display screen, and an input device connected via a system bus. The processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the electronic device is used to communicate with an external server via a network connection. When the electronic device program is executed by the processor, the functions or steps on the client side of an image generation method are implemented.

[0115] Another embodiment of the present application provides an electronic device, comprising at least a memory and a processor, wherein the memory stores a computer program, and the processor implements the following method steps when executing the computer program in the memory:

[0116] Step 1: Receive input business data from the target user;

[0117] Step 2: Based on the input business data, a preset retrieval method is used to search the preset prompt text library and the preset image library to obtain a retrieval business data set;

[0118] Step 3: in response to a user's selection operation on target retrieval business data in the retrieval business data set, query a preset image generation model library based on the target retrieval business data to obtain a first target image generation model corresponding to the target retrieval business data;

[0119] Step 4: Based on the target retrieval business data, the first target image generation model is used to generate an image to obtain a first target image.

[0120] The specific implementation process of the above method steps can refer to the embodiments of any of the above-mentioned image generation methods, and this embodiment will not be repeated here.

[0121] The present application receives input business data from a target user; the present application can generate a final first target image based on the input image data or text data, and can realize image generation for multi-dimensional input data, thereby improving practicality. Based on the input business data, a preset retrieval method is used to retrieve a preset prompt text library and a preset image library to obtain a retrieval business data set; in response to a user's selection operation of the target retrieval business data in the retrieval business data set, a preset image generation model library is queried based on the target retrieval business data to obtain a first target image generation model corresponding to the target retrieval business data; the present application pre-constructs a preset prompt text library, a preset image library, and a preset image generation model library in combination with each business type, so that the business fit can be improved when the user generates an image for the target business type. The first target image generation model is used to generate an image based on the target retrieval business data to obtain a first target image. The method of the present application can improve the user's satisfaction with the generated image when generating an image for the target business type application scenario.

[0122] The non-company software tools or components that appear in the embodiments of this application are merely examples and do not represent actual use. The above embodiments are only exemplary embodiments of this application and are not intended to limit this application. The scope of protection of this application is defined by the claims. Those skilled in the art may make various modifications or equivalent substitutions to this application within the essence and scope of protection of this application, and such modifications or equivalent substitutions shall also be deemed to fall within the scope of protection of this application.

Claims

1. An image generation method, characterized in that: include: Receive input business data from target users; Based on the input business data, a preset prompt text library and a preset image library are searched using a preset search method to obtain a search business data set; In response to a user's selection operation of target retrieval business data in the retrieval business data set, a preset image generation model library is queried based on the target retrieval business data to obtain a first target image generation model corresponding to the target retrieval business data; The first target image generation model is used to generate an image based on the target retrieval business data to obtain a first target image.

2. The method according to claim 1, characterized in that Before searching the preset prompt text library and the preset image library using a preset search method based on the input business data to obtain a search business data set, the method further includes: constructing the preset prompt text library; The step of constructing the preset prompt text library specifically includes: Constructing several initial prompt texts corresponding to the target business type; Based on each of the initial prompt texts, a preset multilingual translation model is used to perform data expansion to obtain a prompt text set corresponding to the target business type, so as to obtain the prompt text set corresponding to each business type respectively; The preset prompt text library is constructed based on each of the prompt text sets.

3. The method according to claim 2, characterized in that Before searching the preset prompt text library and the preset image library using a preset search method based on the input business data to obtain a search business data set, the method further includes: constructing the preset image library; The constructing of the preset image library specifically includes: Based on each of the prompt text sets, a preset stable diffusion model is used to generate an image to obtain an image data set corresponding to each of the business types; Performing cluster analysis on each image data in the image data set corresponding to the same business category to obtain each cluster cluster; Scoring each of the clusters to obtain target clusters corresponding to the same business category; The preset image library is constructed based on each of the target clusters.

4. The method according to claim 1, characterized in that The method of searching the preset prompt text library and the preset image library based on the input business data using a preset search method to obtain a search business data set specifically includes: Extracting features from the input service data to obtain an input feature vector; Performing feature extraction on each preset prompt text in the preset prompt text library to obtain each first feature vector; Performing feature extraction on each image data in the preset image library to obtain each second feature vector; Performing similarity calculation processing based on the input feature vector and each of the first feature vectors to obtain each first similarity; Performing similarity calculation processing based on the input feature vector and each of the second feature vectors to obtain each second similarity; The preset prompt texts and the preset images are screened based on the first similarities, the second similarities and the first preset threshold to obtain the retrieval service data set.

5. The method according to claim 4, characterized in that The screening of the preset prompt texts and the preset images based on the first similarities, the second similarities and the first preset threshold to obtain the retrieval service data set specifically includes: Sort the preset prompt texts based on the first similarities to obtain a first search service data queue; Sort the preset images based on the second similarities to obtain a second search service data queue; The first retrieval service data in the first retrieval service data queue ranked first is screened based on the first preset threshold, and the second retrieval service data in the second retrieval service data queue ranked first is screened based on the first preset threshold to obtain the retrieval service data set.

6. The method according to claim 1, characterized in that In response to a user's selection operation of the target retrieval service data in the retrieval service data set, before querying a preset image generation model library based on the target retrieval service data to obtain a first target image generation model corresponding to the target retrieval service data, the method further includes: constructing a preset image generation model library; The construction of the preset image generation model library specifically includes: Based on the preset prompt texts in the preset prompt text library, a preset low-rank adaptive transfer model is used to perform model training to obtain a plurality of first image generation models; Based on the image data in the preset image library, a preset low-rank adaptive migration model is used to perform model training to obtain a plurality of second image generation models; A model library is constructed based on each of the first image generation models and each of the second image generation models to obtain the preset image generation model library.

7. The method according to claim 1, characterized in that After the first target image generation model is used to generate an image based on the target retrieval service data to obtain the first target image, the method further includes: In response to an optimization operation performed by the target user on the first target image, querying the preset image generation model library based on the first target image to obtain a second image generation model corresponding to the first target image; The second image generation model is used to generate an image based on the first target image to obtain a second target image.

8. An image generating device, characterized in that: include: A receiving module, used to receive input business data from a target user; A retrieval module, used to search a preset prompt text library and a preset image library based on the input business data by using a preset retrieval method to obtain a retrieval business data set; A query module, configured to query a preset image generation model library based on the target retrieval business data in response to a user's selection operation on the target retrieval business data in the retrieval business data set, and obtain a first target image generation model corresponding to the target retrieval business data; A generation module is used to generate an image using the first target image generation model based on the target retrieval business data to obtain a first target image.

9. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the image generation method according to any one of claims 1 to 7 are implemented.

10. An electronic device, characterized in that: The device at least comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the image generation method according to any one of claims 1 to 7 when executing the computer program on the memory.

Images