Image generation method and device, electronic equipment and storage medium
Through semantic analysis, the embedded model identification and statement format information is obtained, the target image generation model is constructed and the target text information is generated, which solves the problem that users need to have professional knowledge to generate images, and realizes the adaptability and ease of use of image generation.
Patent Information
- Application Number
- CN202411058491.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-02
- Publication Date
- 2024-11-08
AI Technical Summary
During the image generation process, users need to have model expertise to accurately and effectively generate the required images, which makes application difficult.
By obtaining image generation prompt information, semantic analysis is performed to obtain embedded model identification and statement format information, a target image generation model is built, and the format is adjusted according to the statement format information to generate target text information so as to generate target image based on the target text information The model generates the target image.
The user can generate accurate and effective target images without the user's expertise, reducing the difficulty of user application.
Smart Images

Figure CN118918206A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of automobile technology, and in particular to an image generation method, device, electronic equipment and storage medium. Background Art
[0002] With the increasing research on intelligent technology, image generation and electronic secretary algorithms have received a lot of attention in vehicle systems. However, since vehicle systems have high requirements for the restoration of the generated products and the corresponding data sets are relatively scarce, it is necessary to train and update the parameters of the benchmark model to achieve high-precision image generation. In this field, the most common method is to introduce an embedded model for data in a specific field into the network. The embedded model can update the style of the generated image without changing the parameters of the benchmark network, and is easy to transplant, train and use. Simply integrate its parameters into the network and use the preset command to activate the corresponding style. However, the existing embedded models are usually only for specific styles or fields, which requires users to know the embedded model and triggering method that can achieve the style before generating images of a specific style. That is, in specific applications, users need to have model expertise to accurately and effectively generate the required images, resulting in high difficulty for users to apply. Summary of the invention
[0003] The present application provides an image generation method, device, electronic device and storage medium to solve the problem that in the image generation process, users are required to have model expertise in order to accurately and effectively generate the required image, which leads to high difficulty in user application.
[0004] In a first aspect, the present application provides an image generation method, comprising:
[0005] Get image generation prompt information;
[0006] Performing semantic analysis based on the image generation prompt information to obtain generated sentence information, wherein the generated sentence information includes an embedded model identifier and sentence format information;
[0007] When the generated sentence information meets the preset sentence condition, a target embedded model corresponding to the embedded model identifier is used to construct a target image generation model, and the image generation prompt information is formatted according to the sentence format information to obtain target text information;
[0008] Based on the target text information, a target image corresponding to the image generation prompt information is generated through the target image generation model.
[0009] Optionally, when the generated sentence information meets the preset sentence condition, using the target embedded model corresponding to the embedded model identifier to construct a target image generation model includes:
[0010] When the generated sentence information meets the preset sentence condition, determining the preset hot start mode as the image generation mode;
[0011] According to the image generation mode, obtaining a target embedded model corresponding to the embedded model identifier;
[0012] The target embedded model is combined with a preset image generation model to obtain the target image generation model.
[0013] Optionally, the formatting of the image generation prompt information is adjusted according to the sentence format information to obtain target text information, including:
[0014] Performing word segmentation on the image generation prompt information to obtain at least two word segmentation information;
[0015] Using the sentence format information, adjusting the format of each word segmentation information to obtain target word segmentation information;
[0016] The target word segmentation information is used to generate the target text information.
[0017] Optionally, the generating a target image corresponding to the image generation prompt information based on the target text information by using the target image generation model includes:
[0018] Inputting the target text information into the target image generation model to obtain model output information, wherein the model output information represents an image generated by the target image generation model based on the target text information;
[0019] The target image is generated using the model output information.
[0020] Optionally, after performing semantic analysis based on the image-generated prompt information to obtain generated sentence information, the method further includes:
[0021] Determining a sentence format of the generated sentence information;
[0022] When the statement format conforms to a preset statement format, extracting the embedded model identifier and the statement format information from the generated statement information;
[0023] When the embedded model identifier meets the preset identifier condition and the statement format information meets the preset statement format condition, it is determined that the generated statement information meets the preset statement condition.
[0024] Optionally, after extracting the embedded model identifier from the generated statement information, the method further includes:
[0025] Get the preset identification set corresponding to the preset identification condition;
[0026] When the embedded model identifier belongs to a preset identifier in the preset identifier set, it is determined that the embedded model identifier meets the preset identifier condition.
[0027] Optionally, after performing semantic analysis based on the image-generated prompt information to obtain generated sentence information, the method further includes:
[0028] When the generated sentence information does not meet the preset sentence condition, determining the preset cold start mode as the image generation mode;
[0029] According to the image generation mode, performing vector conversion on the image generation prompt information to obtain a prompt vector;
[0030] Determining a target embedded model corresponding to the prompt vector;
[0031] Combining the target embedded model with a preset image generation model to obtain the target image generation model;
[0032] The format of the image generation prompt information is adjusted to obtain the target text information.
[0033] Optionally, determining a target embedded model corresponding to the prompt vector includes:
[0034] Determining at least two model description vectors that match the prompt vector in a preset model description vector database;
[0035] Determine an embedded model corresponding to each of the model description vectors;
[0036] Determining a degree of match between the image generation prompt information and each of the embedded models;
[0037] When the matching degree reaches a preset matching degree, the embedded model is determined as the target embedded model.
[0038] Optionally, the step of adjusting the format of the image generation prompt information to obtain the target text information includes:
[0039] Determining statement format information corresponding to the target embedded model;
[0040] The image generation prompt information is formatted using the sentence format information to obtain the target text information.
[0041] In a second aspect, the present application provides an image generating device, comprising:
[0042] An acquisition module is used to obtain image generation prompt information;
[0043] An analysis module, configured to perform semantic analysis based on the image generation prompt information to obtain generated sentence information, wherein the generated sentence information includes an embedded model identifier and sentence format information;
[0044] A processing module, configured to construct a target image generation model by using a target embedded model corresponding to the embedded model identifier when the generated sentence information meets a preset sentence condition, and to adjust the format of the image generation prompt information according to the sentence format information to obtain target text information;
[0045] A generation module is used to generate a target image corresponding to the image generation prompt information based on the target text information and through the target image generation model.
[0046] In a third aspect, an electronic device is provided, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;
[0047] Memory, used to store computer programs;
[0048] The processor is used to implement the image generation method described in any one of the first aspects when executing the program stored in the memory.
[0049] According to a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the image generation method as described in any one of the first aspects is implemented.
[0050] Beneficial effects of the present invention:
[0051] The embodiment of the present application obtains image generation prompt information, performs semantic analysis based on the image generation prompt information, obtains generation sentence information, the generation sentence information includes an embedded model identifier and sentence format information, and when the generation sentence information meets the preset sentence conditions, uses the target embedded model corresponding to the embedded model identifier to construct a target image generation model, and adjusts the format of the prompt information according to the sentence format information to obtain target text information, and generates a target image corresponding to the prompt information based on the target text information through the target image generation model, so that the target image generation model can be adaptively constructed for the image generation prompt information, and thus accurate and effective target images can be generated without the user having model expertise, thereby solving the problem in the existing related technology that users need to have model expertise to accurately and effectively generate the required images, and can effectively reduce the difficulty of user application. BRIEF DESCRIPTION OF THE DRAWINGS
[0052] Figure 1 A schematic diagram of a flow chart of an image generation method provided in an embodiment of the present application;
[0053] Figure 2 A schematic diagram of the structure of an image generating device provided in an embodiment of the present application;
[0054] Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0055] The following will describe the embodiments of the present invention with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for illustrating the present invention, not for limiting the scope of protection of the present invention.
[0056] In the process of image generation, since different types of images require different model parameters, in general, the corresponding image generation models are adaptively trained for different types of image generation. For example, in a scenario where type A images and type B images need to be generated frequently, it is necessary to respectively build and train image generation model A suitable for generating type A images and image generation model B suitable for generating type B images. This method will require frequent construction and training of multiple models, resulting in high application costs and difficulties.
[0057] In the existing related technologies, the model parameters of the image generation model can be changed by loading an embedded model in the image generation model, so that the image generation model can rely on different model parameters to generate different types of images. Although this method can improve the diversity of the image generation model, the types of image generation requirements of users are various. If different embedded models corresponding to many different types are loaded into the image generation model, the image generation model will be bloated, resulting in the problem of reduced efficiency and accuracy of image generation. In addition, when the user is performing a specific image generation process, the embedded model corresponding to the current image generation requirement needs to be activated in order to generate the image. For example, the image generation model is pre-loaded with 26 embedded models, A, B, C...Z. When the user uses the image generation model, it is necessary to determine the corresponding embedded model according to the category of the generated image, such as embedded model D. At this time, it is necessary to use feature statement instructions to activate the embedded model D. In this process, the user needs to master the functions of each embedded model in the target image generation model and the feature statement instructions corresponding to each embedded model in order to generate the corresponding image. For general users, this method requires significantly more professional knowledge, that is, the user needs to have model professional knowledge in order to accurately and effectively generate the required image, which leads to the problem of high difficulty in user application.
[0058] In order to solve the problem that in the image generation process, users need to have model expertise to accurately and effectively generate the required image, which leads to high difficulty in user application, the present application provides an image generation method, device, electronic device and storage medium, by obtaining image generation prompt information, performing semantic analysis based on the image generation prompt information, obtaining generation sentence information, the generation sentence information includes an embedded model identifier and sentence format information, and when the generation sentence information meets the preset sentence conditions, using the target embedded model corresponding to the embedded model identifier to construct a target image generation model, and adjusting the format of the prompt information according to the sentence format information to obtain target text information, so as to generate a target image corresponding to the prompt information based on the target text information through the target image generation model, so that the target image generation model can be adaptively constructed according to the image generation prompt information, and accurate and effective target images can be generated without the user having model expertise, thereby solving the problem caused by the need for users to have model expertise to accurately and effectively generate the required images in the existing related technology, and effectively reducing the difficulty of user application.
[0059] Figure 1A flow chart of an image generation method provided for an embodiment of the present application. This method can be applied to one or more electronic devices such as vehicles, clients, and servers. In addition, the execution subject of this method can be hardware or software. When the above-mentioned execution subject is hardware, the execution subject can be one or more of the above-mentioned electronic devices. For example, a single electronic device can execute this method, or multiple electronic devices can cooperate with each other to execute this method. When the above-mentioned execution subject is software, this method can be implemented as multiple software or software modules, or as a single software or software module. No specific limitation is made here.
[0060] like Figure 1 As shown, an image generation method provided in an embodiment of the present application may specifically include the following steps:
[0061] Step S110: Obtain image generation prompt information.
[0062] Specifically, the image generation prompt information in this embodiment can represent the prompt information corresponding to the image generation requirement. It can be the prompt text input by the user according to the image generation requirement, or it can be the user inputting the requirement voice according to the image generation requirement, and the image generation prompt information is obtained by performing voice recognition on the requirement voice. Of course, other methods can also be used to obtain the image generation prompt information, and this embodiment does not make specific limitations on this.
[0063] In one example, when a user needs to generate a panoramic image of a vehicle, the user enters a prompt text "Please generate a panoramic image of this vehicle". At this time, the prompt text "Please generate a panoramic image of this vehicle" can be used as image generation prompt information.
[0064] Step S120: Perform semantic analysis based on the image generation prompt information to obtain generated sentence information, where the generated sentence information includes an embedded model identifier and sentence format information.
[0065] Specifically, after obtaining the image generation prompt information, semantic analysis can be performed based on the image generation prompt information to obtain generated sentence information, wherein the generated sentence information can include an embedded model identifier and sentence format information. The embedded model identifier represents an identifier used to indicate the embedded model, that is, the embedded model identifier and the embedded model can have a preset one-to-one correspondence. Since different embedded models require different sentence input formats, the sentence format information can represent the sentence input format corresponding to the embedded model.
[0066] It should be noted that the semantic analysis method in this embodiment may be to adopt a preconfigured semantic analysis algorithm or a semantic analysis neural network model. In the specific process of performing semantic analysis on the image generation prompt information, the text in the image generation prompt information may be extracted, and the text may be segmented to obtain one or more segmented words. For each segmented word, it is judged whether there is a preset model identifier that matches the segmented word. When there is a preset model identifier that matches the segmented word, the preset model identifier may be determined as an embedded model identifier. It can be seen that the generated sentence information in this embodiment may include one or more embedded model identifiers; then the sentence format information corresponding to the embedded model identifier may be determined, so that the generated sentence information may be obtained based on the embedded model identifier and the sentence format information.
[0067] In one example, the generated statement information may be as follows:
[0068] Model TD:nexa i
[0069] Model usage: add style\add entity\inference acceleration
[0070] Training set: Asian face dataset\Chinese provincial capital city landmark buildings\Mid-Autumn Festival style
[0071] Applicable scope: face\architecture\painting style
[0072] Trigger words: Asian face\The national stadiumMid autumn festival style
[0073] Trigger word processing method: add trigger word\replace trigger word\no processing
[0074] The model ID may be an embedded model identifier, the trigger word and the trigger word processing method may be sentence format information, and other information may be used as embedded model description information.
[0075] In one example, the process of performing semantic analysis on the image generation prompt information may include two steps. The first step may be to use a tokenizer to convert the text in the image generation prompt information into multiple tokens according to a preset vocabulary and the principle of maximum matching. Then the second step may be to use a semantic analysis text model to perform semantic extraction on each token in the form of a vector, so as to obtain a matching preset model identifier as a vehicle image model identifier. The vehicle image model identifier may be determined as an embedded model identifier, and then the sentence format information corresponding to the embedded model identifier may be determined, so that the generated sentence information may be obtained based on the embedded model identifier and the sentence format information. Of course, the above is only an example for illustration, and this embodiment does not make any specific limitation on this.
[0076] In addition, the word segmenter can be combined to perform semantic analysis on the image generation prompt information to determine the embedded model identifier. Specifically, n+1 special tokens can be added to the word segmenter:<nexa_0> ~<nexa_n> As an embedded model identifier,<nexa_0> Indicates that there is no corresponding embedded model. After semantic analysis of the image generation prompt information, the corresponding token can be generated, such as<nexa_3> , then you can determine according to the pre-configured correspondence<nexa_3> The corresponding embedded model identifier is the vehicle image model identifier, that is, the vehicle image model identifier can be determined as the embedded model identifier; of course, the above is only an example for illustration, and this embodiment does not make specific limitations on this; in addition, this embodiment can be trained in advance for different target image generation models, and the specific training samples can be as follows:
[0077] Define the role: As follows, you need to play the embedding model routing robot of the image generation model.
[0078] You are required to select an embedding model that can achieve the corresponding function based on the user's needs and the given embedding model description. The embedding model is a small model trained from a certain dataset, which is integrated into the generated model to achieve parameter update.
[0079] User needs:{query}
[0080] Embedded model description statement: model describes}
[0081] The output format is as follows:
[0082] For embedding models that require adding trigger words
[0083] <nexa i> ( <add>:Specific trigger words)<nexa end>
[0084] For embedding models that need to replace trigger words
[0085] <nexa i> 〈replace>: entity in the requirement->corresponding entity in the description statement)<nexa end>
[0086] For embedded models without any processing
[0087] (nexa i>( <null>)<nexa end>
[0088] When you need to use multiple embedding models
[0089] <nexa i> ( <mul1>);nexa i>( <add>:Specific trigger words)<nexa end>
[0090] When the user needs cannot be met according to the given embedding model description
[0091] (nexa nul>( <refuse> ) <nexaend>
[0092] Response:
[0093] Step S130: When the generated sentence information meets the preset sentence conditions, the target embedded model corresponding to the embedded model identifier is used to construct a target image generation model, and the format of the image generation prompt information is adjusted according to the sentence format information to obtain the target text information.
[0094] Specifically, after obtaining the generated sentence information, it can be determined whether the generated sentence information meets the preset sentence conditions. The preset sentence conditions represent the pre-configured conditions for the normal generated sentence information. That is, when the generated sentence information does not meet the preset sentence conditions, it can be determined that the current generated sentence information is abnormal, wherein the embedded model identifier may be abnormal and / or the sentence format information may be abnormal; and when the generated sentence information meets the preset sentence conditions, it can be determined that the current generated sentence information is not abnormal. At this time, the target embedded model corresponding to the embedded model identifier can be determined, and the target embedded model corresponding to the embedded model identifier can be used to construct a target image generation model, and the image generation prompt information is formatted according to the sentence format information to obtain the target text information, wherein the target text information can represent text that meets the target embedded model input format corresponding to the embedded model identifier. A specific example of the target text information can be as follows:
[0095] Define the role: As follows, you need to play the role of an embedding model routing robot for an image generation model.
[0096] You are required to select an embedding model that can achieve the corresponding function based on the user's needs and the given embedding model description. The embedding model is a small model trained from a certain dataset, which is integrated into the generated model to achieve parameter update.
[0097] User needs:{query}
[0098] The embedded model description statement provided: iodeldescribes}
[0099] The output should include the selected plugin id and the corresponding trigger word processing method.
[0100] Please list <ison>Output format. An example that can be implemented with a single model
[0101] [{model_id:1,actiom:add,param:trigger}]
[0102] [{model_id:2, action:replace, param:entity in prompt -> entity provided by embedding model description}]
[0103] [{model id:3,action:null,param:null}]
[0104] For examples that require multiple models to implement.
[0105] [{model id:1,action:add,param:trigger},{model id:2,action:replace,param:entity in prompt -> entity provided by embedding model description}]
[0106] When this cannot be achieved based on the input embedding model description.
[0107] [{model id:null,action:refuse,param:null}]
[0108] Response:
[0109] Next, in the process of constructing the target image generation model, the target embedded model corresponding to the embedded model identifier can be loaded into the preset image generation model to construct the target image generation model. The target image generation model can represent a model that can generate the required image corresponding to the image generation prompt information.
[0110] It should be noted that in determining whether the generated statement information meets the preset statement conditions, it is possible to determine whether the format of the generated statement information matches the regular expression
[0111] <nexa_[0-9]*>\(<[refuse,add,replace,null]>:?.*<nexa_end> \), whether the embedded model identifier in the generated sentence information is a preset embedded model identifier, whether the sentence format information is information for formatting the image generation prompt information, and whether it does not contain a rejection identifier; wherein the rejection identifier indicates that there is no preset embedded model identifier that meets the current image generation prompt information, for example, an embedded model identifier A corresponding to an embedded model for generating a vehicle image, and an embedded model identifier B corresponding to an embedded model for generating a person image are pre-configured. At this time, the image generation prompt information indicates that a sea image needs to be generated. At this time, there is no embedded model for indicating the generation of a sea image in the preset embedded model identifier, then a generated sentence information containing a rejection identifier can be generated, so that in the subsequent step of judging whether the generated sentence information meets the preset sentence conditions, it is determined that the generated sentence information does not meet the preset sentence conditions, thereby achieving the effect of improving the recognition efficiency of the image generation prompt information. Of course, the above is only an example for illustration. In specific implementation, adaptive adjustments can be made according to the scenario and needs. This embodiment does not make specific limitations on this.
[0112] Step S140: Based on the target text information, a target image corresponding to the image generation prompt information is generated through a target image generation model.
[0113] Specifically, after constructing a target image generation model and determining the target text information, a target image corresponding to the image generation prompt information can be generated based on the target text information through the target image generation model, wherein the target image can represent an image for output, and the specific process of generating the target image corresponding to the image generation prompt information can be to input the target text information into the target image generation model, so that the target image generation model generates an image based on the target text information, and outputs a model-generated image, and the model-generated image can be used as the target image corresponding to the image generation prompt information.
[0114] The embodiment of the present application obtains image generation prompt information, performs semantic analysis based on the image generation prompt information, and obtains generation sentence information, wherein the generation sentence information includes an embedded model identifier and sentence format information, so that in the semantic analysis stage, the generated image indicated by the image generation prompt information can be determined by the generated sentence information, and the required embedded model identifier and sentence format information make it possible for the user to input the image generation prompt information without having to understand the professional knowledge related to the embedded model during the image generation process; then, when the generated sentence information meets the preset sentence conditions, the target embedded model corresponding to the embedded model identifier can be used to construct a target image generation model, and the prompt information can be formatted according to the sentence format information. The target text information is obtained, so that the target image generation model can be adaptively constructed according to the target embedded model corresponding to the embedded model identifier, avoiding the step that the user needs to activate the target embedded model corresponding to the current image generation demand, which can effectively reduce the user's application difficulty; based on the target text information, the target image corresponding to the prompt information is generated through the target image generation model, so that the target image generation model can be adaptively constructed according to the image generation prompt information, and then accurate and effective target images can be generated without the user having model expertise, which solves the problem caused by the existing related technology that the user needs to have model expertise to accurately and effectively generate the required image, which can effectively reduce the user's application difficulty.
[0115] In order to avoid the problem that users need to have professional knowledge of embedded models in advance during the image generation process, in an optional embodiment of the present application, when the generated sentence information meets the preset sentence conditions in step S130, the target embedded model corresponding to the embedded model identifier is used to construct the target image generation model, which may specifically include the following sub-steps: when the generated sentence information meets the preset sentence conditions, the preset hot start mode is determined as the image generation mode; based on the image generation mode, the target embedded model corresponding to the embedded model identifier is obtained; the target embedded model is combined with the preset image generation model to obtain the target image generation model.
[0116] In this embodiment, when it is determined that the generated statement information meets the preset statement conditions, the preset hot start mode can be determined as the image generation mode, wherein the image generation mode represents a mode for performing image generation in response to the image generation prompt information, and the hot start mode can represent a mode pre-configured for representing the execution of steps S130-S140 to perform image generation; thus, when the image generation mode is the hot start mode, the embedded model identifier can be extracted from the generated statement information, and then the target embedded model corresponding to the embedded model identifier can be obtained, and the target embedded model can be combined with the preset image generation model to obtain the target image generation model. The specific combination method can be to load the target embedded model into the image generation model and activate it, thereby obtaining the target image generation model. image generation model; the target embedded model in this embodiment represents a model for adjusting the model parameters in the image generation model, and the image generation model can generate different types of images according to different model parameters, that is, by combining the target embedded model with the preset image generation model, a target image generation model is obtained, which can enable the target image generation model to have the ability to generate the required image corresponding to the generated image prompt information; in the process of generating images, the user does not need to combine his own embedded model expertise, but can adaptively determine the target embedded model through the embedded model identifier in this embodiment, and construct a target image generation model that can generate the required image corresponding to the generated image prompt information based on the target embedded model, which can effectively reduce the difficulty of user application.
[0117] It should be noted that in this embodiment, the corresponding target embedded model can be directly determined through the embedded model identifier. Compared with the existing related technologies that require cumbersome steps of embedded model retrieval, sorting, generation, etc. to determine the target embedded model, it can effectively improve the efficiency and accuracy of determining the target embedded model.
[0118] In one example, in the scenario where the user needs to generate an image of a vehicle image, the process of using the existing related technology for generation requires the user to determine whether the current image generation model contains an embedded model corresponding to the vehicle image according to the generation requirements. If it does, the user needs to determine the command corresponding to activating the embedded model according to the embedded model expertise, so as to activate the embedded model in the image generation model, and then generate the vehicle image; while in this embodiment, the user only needs to input the image generation prompt information, and then the semantic analysis can be performed according to the image generation prompt information to obtain the generation sentence information, and the embedded model identifier can be extracted from the generation sentence information, and then the target embedded model corresponding to the embedded model identifier can be obtained, and the target embedded model can be combined with the preset image generation model to obtain the target image generation model. In this process, the user does not need to perform cumbersome operations according to the embedded model expertise, which can effectively reduce the user's application difficulty and improve the image generation efficiency. Of course, the above is only an example for illustration. In the specific implementation, it can be adaptively adjusted according to the scene and needs, and this embodiment does not make specific limitations on this.
[0119] In order to improve the generation accuracy of the target image generation model, in an optional embodiment of the present application, the format of the image generation prompt information is adjusted according to the sentence format information to obtain the target text information, which may specifically include the following sub-steps: segmenting the image generation prompt information to obtain at least two segmentation information; using the sentence format information, formatting each segmentation information to obtain the target segmentation information; using the target segmentation information to generate the target text information.
[0120] In this embodiment, after obtaining the generated sentence information, sentence format information can be extracted from the generated sentence information, and the image generation prompt information can be segmented to obtain at least two segmentation information, the segmentation information indicates that the image generation prompt information contains preset words, so that the sentence format information can be used to adjust the format of each segmentation information to obtain target segmentation information, wherein the format adjustment can represent the maintenance, replacement, addition and other adjustment processing of the words represented by the segmentation information, and the target segmentation information can represent the words that meet the input specifications of the target embedded model; and then the target segmentation information can be used to generate target text information, and the target text information represents the text used to input the target image generation model.
[0121] This embodiment generates target text information for input into a target image generation model by performing operations such as word segmentation and format adjustment on image generation prompt information, and then inputs the target text information into the target image generation model for image generation. This avoids the problem of low accuracy of image generation by the target image generation model due to the input text not meeting the specifications of the target image generation model, and can effectively improve the generation accuracy of the target image generation model.
[0122] In one example, a word segmenter can be used to adjust the format of the image generation prompt information according to the sentence format information to obtain the target text information. First, in the word segmenter, <replace> 、 <add> 、 <null> 、 <refuse>Four special tokens are used to indicate the type of formatting that needs to be adjusted. <replace>Indicates that the format adjustment needs to be replaced. <add>Indicates that the format of adding trigger words needs to be adjusted. <null>Indicates format adjustment without any processing. <refuse>Indicates the format adjustment of rejection; after the word segmentation is performed on the image generated prompt information by the word segmenter, at least two word segmentation information and the token corresponding to each word segmentation information can be obtained. For example, after the word segmentation is performed on the image generated prompt information by the word segmenter, the word segmentation information A and the token corresponding to the word segmentation information A are obtained. <replace>, word segmentation information B and word segmentation information B corresponding to <add>, word segmentation information C and word segmentation information C corresponding to <null>, word segmentation information D and word segmentation information D corresponding to <null>, so that we can base <replace>The words represented by the word segmentation information A are replaced with preset corresponding words to obtain the target word segmentation information A. <add>Add the preset corresponding words to the words represented by the word segmentation information B to obtain the target word segmentation information B. <null>The words represented by the word segmentation information C do not need to be processed to obtain the target word segmentation information C. <null>The words represented by the word segmentation information D do not need to be processed to obtain the target word segmentation information C, and then the set of the target word segmentation information A, the target word segmentation information B, the target word segmentation information C, and the target word segmentation information D can be determined as the target text information. Of course, the above is only an example for illustration, and in specific implementation, adaptive adjustments can be made according to scenarios and requirements, and this embodiment does not make specific limitations on this.
[0123] In an optional embodiment of the present application, based on the target text information, a target image corresponding to the image generation prompt information is generated through a target image generation model, which may specifically include the following sub-steps: inputting the target text information into the target image generation model to obtain model output information, wherein the model output information represents an image generated by the target image generation model based on the target text information; and using the model output information to generate the target image.
[0124] In this embodiment, in the process of generating a target image, the target text information can be input into the target image generation model, so that the image generation model whose model parameters have been modified by the target embedded model is generated, and because the image generation prompt information is formatted by using the sentence format information, the target text information that meets the input format specification of the target image generation model is obtained, so that the target image generation model can accurately and efficiently generate images based on the target text information, and obtain model output information. The model output information can represent the image generated by the target image generation model based on the target text information. Then, the model output information can be used to generate the target image, thereby obtaining the target image corresponding to the image generation prompt information, so that the user can generate an accurate and effective target image without having model expertise, thereby solving the problem caused by the existing related technology that the user needs to have model expertise to accurately and effectively generate the required image, and can effectively reduce the difficulty of user application.
[0125] In order to avoid the problem of reducing the accuracy of generating a target image due to abnormal generated sentence information, in an optional embodiment of the present application, after performing semantic analysis based on the image generation prompt information to obtain the generated sentence information, the following sub-steps may be specifically included: determining the sentence format of the generated sentence information; when the sentence format meets the preset sentence format, extracting the embedded model identifier and sentence format information from the generated sentence information; when the embedded model identifier meets the preset identifier condition and the sentence format information meets the preset sentence format condition, determining that the generated sentence information meets the preset sentence condition.
[0126] In this embodiment, after obtaining the generated statement information, it can be determined whether the generated statement information meets the preset statement conditions. In the specific judgment process, first, the sentence format of the generated statement information can be determined, and the sentence format indicates the format of the statement contained in the generated statement information, such as a regular expression format, an abnormal format, etc.; then it can be determined whether the statement format meets the preset statement format, and the preset statement format indicates the format displayed by the generated statement information that is pre-configured to meet the specifications. That is, when the statement format does not meet the preset statement format, it can be determined that the current generated statement information has an abnormality and subsequent steps cannot be executed; when the statement format meets the preset statement format, it can be determined that the current generated statement information is in a normal format. At this time, the embedded model identifier and statement format information can be extracted from the generated statement information, and it can be determined whether the embedded model identifier meets the preset identifier condition. The preset identifier condition indicates a pre-configured identifier that can be identified. When the embedded model identifier meets the preset identifier condition Next, it can be determined whether the statement format information meets the preset statement format conditions. The preset statement format conditions can represent a pre-configured format that can be recognized. For example, the preset statement format conditions include pre-configured statement format conditions such as addition, replacement, and no processing. At this time, if the statement format information indicates deletion, it can be determined that the statement format information does not meet the preset statement format conditions, and there is no need to perform subsequent steps. When the statement format information indicates one of addition, replacement, and no processing, it can be determined that the statement format information meets the preset statement format conditions; combined with the two judgment results of the embedded model identifier and the statement format information, when the embedded model identifier meets the preset identification conditions and the statement format information meets the preset statement format conditions, it can be determined that the generated statement information meets the preset statement conditions, that is, when the embedded model identifier does not meet the preset identification conditions, and / or the statement format information does not meet the preset statement format conditions, it is determined that the generated statement information does not meet the preset statement conditions.
[0127] In this embodiment, by judging whether the sentence format of the generated sentence information conforms to the preset sentence format, judging whether the embedded model identifier conforms to the preset identifier condition and judging whether the sentence format information conforms to the preset sentence format condition, it is determined whether the generated sentence information conforms to the preset sentence condition, thereby only filtering out the generated sentence information whose sentence format conforms to the preset sentence format, whose embedded model identifier conforms to the preset identifier condition and whose judgment sentence format information conforms to the preset sentence format condition, so that the target image generation model constructed according to the embedded model identifier and the sentence format information in the generated sentence information and the target text information obtained by formatting the image generation prompt information are highly accurate, thereby improving the accuracy of the generated target image and avoiding the problem of reducing the accuracy of the generated target image due to abnormal generated sentence information.
[0128] In an optional embodiment of the present application, after extracting the embedded model identifier from the generated sentence information, the following sub-steps may be specifically included: obtaining a preset identifier set corresponding to the preset identifier condition; when the embedded model identifier belongs to a preset identifier in the preset identifier set, determining that the embedded model identifier meets the preset identifier condition.
[0129] In this embodiment, in the process of specifically determining whether the embedded model identifier meets the preset identifier condition, a preset identifier set corresponding to the preset identifier adjustment can be obtained, the preset identifier set includes one or more preset identifiers, each preset identifier is used to indicate an embedded model, and then the embedded model identifier can be compared with each preset identifier in the preset identifier set to determine whether there is a preset identifier consistent with the embedded model identifier. If there is no preset identifier consistent with the embedded model identifier, it can be determined that the embedded model identifier does not belong to the preset identifier in the preset identifier set, so that it can be determined that the embedded model identifier does not meet the preset identifier condition; if there is a preset identifier consistent with the embedded model identifier, it can be determined that the embedded model identifier belongs to the preset identifier in the preset identifier set, so that it can be determined that the embedded model identifier meets the preset identifier condition. In addition, the preset identifier set in this embodiment can be generated according to the identifier corresponding to each embedded model in the pre-configured embedded model database.
[0130] It can be seen that although the retrieval enhancement generation based on a large language model in the existing related technology is an effective and easy-to-implement solution, it involves document recall, precise sorting, and embedded model selection based on a large-scale language model, which will undoubtedly seriously prolong the time required for image generation. Therefore, the present application uses the target embedded model corresponding to the embedded model identifier contained in the generated sentence information to construct a target image generation model, and adjusts the format of the image generation prompt information according to the sentence format information contained in the generated sentence information to obtain the target text information, so that the target image corresponding to the image generation prompt information can be generated based on the target text information through the target image generation model; it plays a role in directly constructing the target image generation model to save the time of embedded model selection, and this process can become a hot start process.
[0131] In an optional embodiment of the present application, after performing semantic analysis based on the image generation prompt information to obtain the generated sentence information, the following sub-steps may be specifically included: when the generated sentence information does not meet the preset sentence conditions, determining the preset cold start mode as the image generation mode; performing vector conversion on the image generation prompt information based on the image generation mode to obtain a prompt vector; determining a target embedded model corresponding to the prompt vector; combining the target embedded model with the preset image generation model to obtain a target image generation model; and formatting the image generation prompt information to obtain target text information.
[0132] In this embodiment, when it is determined that the generated sentence information does not meet the preset sentence conditions, the preset cold start mode can be determined as the image generation mode, wherein the image generation mode represents a mode of performing image generation in response to the image generation prompt information. When the image generation mode is the cold start mode, the image generation prompt information can be vectorized to obtain a prompt vector, wherein the prompt vector represents a semantic vector corresponding to the text in the generated prompt information. The specific vector conversion method can be to use a preset algorithm or a neural network model for conversion, such as the Chinese-English semantic vector model (BAAIGeneral Embedding, BGE), etc., and the image generation prompt information may include multiple image generation requirements or multiple word texts with different functions, so that the image generation prompt information is vectorized and one or more prompt vectors can be obtained; then the target embedded model corresponding to the prompt vector can be determined, and the specific determination method can be determined according to a preset corresponding relationship, or can be matched and determined according to a preset rule strategy, which is not specifically limited in this embodiment; thereby, the target embedded model can be combined with the preset image generation model to obtain a target image generation model, and the image generation prompt information can be formatted to obtain target text information, wherein the format adjustment method can be adjusted by adopting a preset format adjustment strategy, or by adopting an adjustment strategy corresponding to the target embedded model, which is not specifically limited in this embodiment; then step S140 can be executed to generate a target image corresponding to the image generation prompt information based on the target text information and through the target image generation model. Therefore, when the generated sentence information does not meet the preset sentence conditions, the prompt vector corresponding to the image generation prompt information can be used to determine the target embedded model, so as to obtain the target image generation model and target text information. Regardless of whether the generated sentence information meets the preset sentence conditions, the purpose of generating the target image corresponding to the image generation prompt information can be achieved.
[0133] In this embodiment, when the generated statement information does not meet the preset statement conditions, the cold start mode is determined as the image generation mode. Combined with the above-mentioned embodiment, when the generated statement information meets the preset statement conditions, the hot start mode is determined as the image generation mode. This allows the present application to adopt different image generation strategies according to whether the image generation mode is a hot start mode or a cold start mode when responding to image generation prompt information. That is, regardless of whether the generated statement information meets the preset statement conditions, the purpose of generating a target image corresponding to the image generation prompt information can be achieved; and when the image generation mode is a hot start, the target embedded model can be quickly and accurately determined according to steps S130-S140, thereby improving the efficiency of image generation, and when the image generation mode is a cold start, the image generation prompt information can be vectorized to determine the target embedded model, thereby achieving the purpose of improving the effective responsiveness of the image generation prompt information.
[0134] In order to improve the accuracy of determining the target embedded model, in an optional embodiment of the present application, determining the target embedded model corresponding to the prompt vector can specifically include the following sub-steps: determining at least two model description vectors that match the prompt vector in a preset model description vector database; determining the embedded model corresponding to each model description vector; determining the degree of match between the image generation prompt information and each embedded model; when the degree of match reaches a preset degree of match, determining the embedded model as the target embedded model.
[0135] In this embodiment, after determining the prompt vector, at least two model description vectors that match the prompt vector can be determined in a preset model description vector database, wherein the preset model description vector database contains one or more model description vectors, and the model description vector represents a semantic vector obtained after vector conversion of the embedded model description text. The specific vector conversion method can be converted using the BGE model, that is, the model description vector and the embedded model can be in a one-to-one correspondence. At this time, for each prompt vector, at least two model description vectors that match the prompt vector can be determined in the model description vector database. The specific matching method can be vector matching, sine matching, etc.; thereby, each model description vector can be determined. The embedded model corresponding to the quantity is obtained, and the matching degree between the image generation prompt information and each embedded model is determined. Specifically, the image generation prompt information and the algorithm statement of the embedded model are combined with the test statement, and the test statement is input into the preset matching detection model, and the matching detection model outputs the matching degree for the test statement. Of course, other methods can also be used to determine the matching degree, which is not specifically limited in this embodiment. Then, it can be judged whether the matching degree reaches the preset matching degree. The preset matching degree can be a specific matching degree value or a matching degree ranking sequence. When the matching degree reaches the preset matching degree, the embedded model can be determined as the target embedded model. The accuracy of determining the target embedded model can be effectively improved.
[0136] In one example, a model description vector database includes model description vector A, model description vector B, model description vector C, and model description vector D. In a preset model description vector database, at least two model description vectors that match a prompt vector are determined to be model description vector B, model description vector C, and model description vector D. An embedded model B corresponding to model description vector B, an embedded model C corresponding to model description vector C, and an embedded model D corresponding to model description vector D are determined. Image generation prompt information is combined with algorithm statement B of embedded model B to be tested statement B, image generation prompt information is combined with algorithm statement C of embedded model C to be tested statement C, and image generation prompt information is combined with algorithm statement D of embedded model D to be tested statement D. The tested statement B, tested statement C, and tested statement D are input into a preset matching detection model. The matching detection model outputs the matching degree for the test sentence B, the test sentence C, and the test sentence D, and obtains a matching degree of 5 for B, a matching degree of 6 for C, and a matching degree of 7 for D. Then, it can be determined whether the matching degree reaches the preset matching degree. If the preset matching degree is the top 2 in the matching degree ranking, it can be determined that the matching degrees C and D reach the preset matching degrees, thereby the matching degree C embedded model C and the matching degree D embedded model D can be determined. In addition, if the preset matching degree is a specific matching degree value of 5, it can be determined that the matching degrees B, C, and D reach the preset matching degrees, thereby the embedded model B corresponding to the matching degree B, the matching degree C embedded model C, and the matching degree D embedded model D can be determined as the target embedded model. Of course, the above is only an example for illustration. In the specific implementation, it can be adaptively adjusted according to the scenario and needs. This embodiment does not make specific limitations on this.
[0137] In another example, a corresponding embedded model description statement can be generated for each style and each entity of the pre-collected embedded model, and regarded as a text information, and converted into multiple vector information using the embedding model. Then, the vector database FAISS is used to record the vector information and the corresponding text information to form a model description vector database. By vectorizing each entity and style, the reliability of the system can be improved while improving the performance of the system. Then, the retrieval stage is similar to a conventional document retrieval system. In the preset model description vector database, at least two model description vectors that match the prompt vector are determined. By utilizing the embedding model and algorithms such as BM25, the first K embedded model description statements related to the indication vector, that is, the model description vector, can be recalled from the model description vector database. Then, a more reliable sorting result can be obtained by using the sorting model, that is, the matching degree between the image generation prompt information and each embedded model is determined by the sorting model. When the matching degree reaches the preset matching degree, the embedded model is determined as the target embedded model.
[0138] In an optional embodiment of the present application, the format of the image generation prompt information is adjusted to obtain the target text information, which may specifically include the following sub-steps: determining the sentence format information corresponding to the target embedded model; using the sentence format information to adjust the format of the image generation prompt information to obtain the target text information.
[0139] In this embodiment, in the process of formatting the image generation prompt information to obtain the target text information, the sentence format information corresponding to the target embedded model can be determined. The sentence format information represents the format of the target embedded model matching sentence. The sentence format information corresponding to different target embedded models can be the same or different. Then, the sentence format information can be used to format the image generation prompt information. The specific format adjustment method can be to replace, add, or not process words, so as to obtain the target text information. For specific examples of the target text information, please refer to Figure 3 shown.
[0140] It can be seen that the target image generation model will receive the target text information after the format adjustment of the selected target embedded model and generate the final image. In addition, each target embedded model and sentence format information that successfully generates a target image can be collected into a database and used as a sample database for the hot start process in the aforementioned embodiment. The target image generation model can be trained through the samples in the database.
[0141] In one example, a word segmenter can be used to adjust the format of the image generation prompt information according to the sentence format information to obtain the target text information. First, in the word segmenter, <replace> 、 <add> 、 <null> 、 <refuse>Four special tokens are used to indicate the type of formatting that needs to be adjusted. <replace>Indicates that the format adjustment needs to be replaced. <add>Indicates that the format of adding trigger words needs to be adjusted. <null>Indicates format adjustment without any processing. <refuse>Indicates the format adjustment of rejection; after the word segmentation is performed on the image generated prompt information by the word segmenter, at least two word segmentation information and the token corresponding to each word segmentation information can be obtained. For example, after the word segmentation is performed on the image generated prompt information by the word segmenter, the word segmentation information A and the token corresponding to the word segmentation information A are obtained. <replace>, word segmentation information B and word segmentation information B corresponding to <add>, word segmentation information C and word segmentation information C corresponding to <null>, word segmentation information D and word segmentation information D corresponding to <null>, so that we can base <replace>The words represented by the word segmentation information A are replaced with preset corresponding words to obtain the target word segmentation information A. <add>Add the preset corresponding words to the words represented by the word segmentation information B to obtain the target word segmentation information B. <null>The words represented by the word segmentation information C do not need to be processed to obtain the target word segmentation information C. <null>The words represented by the word segmentation information D do not need to be processed to obtain the target word segmentation information C, and then the set of the target word segmentation information A, the target word segmentation information B, the target word segmentation information C, and the target word segmentation information D can be determined as the target text information. Of course, the above is only an example for illustration, and in specific implementation, adaptive adjustments can be made according to scenarios and requirements, and this embodiment does not make specific limitations on this.
[0142] like Figure 2 As shown, the present application also discloses an embodiment, providing an image generating device, including:
[0143] An acquisition module 210 is used to acquire image generation prompt information;
[0144] The analysis module 220 is used to perform semantic analysis based on the image generation prompt information to obtain generated sentence information, where the generated sentence information includes an embedded model identifier and sentence format information;
[0145] The processing module 230 is used to construct a target image generation model by using the target embedded model corresponding to the embedded model identifier when the generated sentence information meets the preset sentence condition, and to adjust the format of the image generation prompt information according to the sentence format information to obtain the target text information;
[0146] The generating module 240 is used to generate a target image corresponding to the image generation prompt information based on the target text information and through a target image generation model.
[0147] In an optional embodiment of the present application, the processing module 230 may include:
[0148] A first determining unit, configured to determine a preset hot start mode as an image generation mode when the generated sentence information meets a preset sentence condition;
[0149] A first acquisition unit, configured to acquire a target embedded model corresponding to the embedded model identifier according to the image generation mode;
[0150] The first combining unit is used to combine the target embedded model with the preset image generation model to obtain the target image generation model.
[0151] In an optional embodiment of the present application, the processing module 230 may include:
[0152] A first word segmentation unit is used to segment the image generation prompt information to obtain at least two word segmentation information;
[0153] The first format adjustment unit is used to adjust the format of each word segmentation information by using the sentence format information to obtain the target word segmentation information;
[0154] The first generating unit is used to generate target text information using target word segmentation information.
[0155] In an optional embodiment of the present application, the generating module 240 may include:
[0156] A model output unit, used to input the target text information into the target image generation model to obtain model output information, wherein the model output information represents an image generated by the target image generation model based on the target text information;
[0157] The second generating unit is used to generate a target image using the model output information.
[0158] In an optional embodiment of the present application, the image generating device may further include:
[0159] A first determination module, used to determine a sentence format for generating sentence information;
[0160] A first extraction module, used for extracting the embedded model identifier and the sentence format information from the generated sentence information when the sentence format conforms to the preset sentence format;
[0161] The second determination module is used to determine that the generated sentence information meets the preset sentence condition when the embedded model identifier meets the preset identifier condition and the sentence format information meets the preset sentence format condition.
[0162] In an optional embodiment of the present application, the image generating device may further include:
[0163] An acquisition set module is used to acquire a preset identification set corresponding to a preset identification condition;
[0164] The third determination module is used to determine that the embedded model identifier meets the preset identifier condition when the embedded model identifier belongs to a preset identifier in the preset identifier set.
[0165] In an optional embodiment of the present application, the image generating device may further include:
[0166] A fourth determination module, configured to determine a preset cold start mode as an image generation mode when the generated sentence information does not meet a preset sentence condition;
[0167] A vector conversion module, used for performing vector conversion on the image generation prompt information according to the image generation mode to obtain a prompt vector;
[0168] A fifth determination module, used to determine a target embedded model corresponding to the prompt vector;
[0169] A combining module, used to combine the target embedded model with the preset image generation model to obtain the target image generation model;
[0170] The format adjustment module is used to adjust the format of the image generation prompt information to obtain the target text information.
[0171] In an optional embodiment of the present application, the fifth determining module may include:
[0172] A second determining unit, configured to determine at least two model description vectors that match the prompt vector in a preset model description vector database;
[0173] A third determining unit, used to determine the embedded model corresponding to each model description vector;
[0174] a fourth determination unit, configured to determine a matching degree between the image generation prompt information and each embedded model;
[0175] The fifth determination unit is used to determine the embedded model as the target embedded model when the matching degree reaches a preset matching degree.
[0176] In an optional embodiment of the present application, the format adjustment module may include:
[0177] A sixth determining unit, used to determine the statement format information corresponding to the target embedded model;
[0178] The second format adjustment unit is used to adjust the format of the image generation prompt information by using the sentence format information to obtain the target text information.
[0179] The implementation process of the functions and effects of each module in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, which will not be repeated here.
[0180] like Figure 3 As shown, the embodiment of the present application provides an air conditioner control device, including a processor 310, a communication interface 320, a memory 330 and a communication bus 340, wherein the processor 310, the communication interface 320, and the memory 330 communicate with each other through the communication bus 340.
[0181] Memory 330, for storing computer programs;
[0182] In one embodiment of the present application, the processor 310 is used to execute the program stored in the memory 330 to implement the control method for image generation provided by any of the aforementioned method embodiments, by obtaining image generation prompt information, performing semantic analysis based on the image generation prompt information, and obtaining generation sentence information, the generation sentence information includes an embedded model identifier and sentence format information, so that in the semantic analysis stage, the generated sentence information can be used to determine the generated image indicated by the image generation prompt information, the required embedded model identifier and sentence format information, so that the user does not need to understand the professional knowledge related to the embedded model during the image generation process, and only needs to input the image generation prompt information; then, when the generated sentence information meets the preset sentence conditions, the target embedded model corresponding to the embedded model identifier can be used to construct the target An image generation model is provided, and the prompt information is formatted according to the sentence format information to obtain the target text information, so that the target image generation model can be adaptively constructed according to the target embedded model corresponding to the embedded model identifier, avoiding the step that the user needs to activate the target embedded model corresponding to the current image generation demand, which can effectively reduce the user's application difficulty; based on the target text information, the target image corresponding to the prompt information is generated through the target image generation model, so that the target image generation model can be adaptively constructed according to the image generation prompt information, and accurate and effective target images can be generated without the user having model expertise, solving the problem that the existing related technology requires users to have model expertise to accurately and effectively generate the required images, which can effectively reduce the user's application difficulty.
[0183] The embodiment of the present application also provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the steps of the image generation method provided in any of the aforementioned method embodiments are implemented, by obtaining image generation prompt information, performing semantic analysis based on the image generation prompt information, and obtaining generation sentence information, wherein the generation sentence information includes an embedded model identifier and sentence format information, so that in the semantic analysis stage, the generated image indicated by the image generation prompt information, the required embedded model identifier and sentence format information can be determined by the generated sentence information, so that the user does not need to understand the professional knowledge related to the embedded model during the image generation process, and only needs to input the image generation prompt information; then, when the generated sentence information meets the preset sentence conditions, the target embedded model corresponding to the embedded model identifier can be used. , construct a target image generation model, and adjust the format of the prompt information according to the sentence format information to obtain the target text information, so that the target image generation model can be adaptively constructed according to the target embedded model corresponding to the embedded model identifier, avoiding the step of requiring the user to activate the target embedded model corresponding to the current image generation requirement, which can effectively reduce the user's application difficulty; based on the target text information, through the target image generation model, the target image corresponding to the prompt information is generated, so that the target image generation model can be adaptively constructed according to the image generation prompt information, and then the user can generate accurate and effective target images without having model expertise, which solves the problem caused by the existing related technology that the user needs to have model expertise to accurately and effectively generate the required image, which can effectively reduce the user's application difficulty.
[0184] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0185] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a general hardware platform, and of course, by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.
[0186] It should be understood that the terms used herein are only for the purpose of describing specific example embodiments and are not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms "one", "an" and "said" as used herein may also be meant to include plural forms. The terms "include", "comprise", "contain", and "have" are inclusive and therefore specify the existence of stated features, steps, operations, elements and / or parts, but do not exclude the existence or addition of one or more other features, steps, operations, elements, parts, and / or combinations thereof. The method steps, processes, and operations described herein are not interpreted as necessarily requiring them to be performed in the specific order described or illustrated, unless the execution order is clearly indicated. It should also be understood that additional or alternative steps may be used.
[0187] The foregoing is merely a specific embodiment of the present invention, which enables those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features claimed herein.< / null> < / null> < / add> < / replace> < / null> < / null> < / add> < / replace> < / refuse> < / null> < / add> < / replace> < / refuse> < / null> < / add> < / replace> < / null> < / null> < / add> < / replace> < / null> < / null> < / add> < / replace> < / refuse> < / null> < / add> < / replace> < / refuse> < / null> < / add> < / replace> < / ison> < / nexaend> < / refuse> < / add> < / null> < / add>
Claims
1. An image generation method, characterized in that: include: Get image generation prompt information; Performing semantic analysis based on the image generation prompt information to obtain generated sentence information, wherein the generated sentence information includes an embedded model identifier and sentence format information; When the generated sentence information meets the preset sentence condition, a target embedded model corresponding to the embedded model identifier is used to construct a target image generation model, and the image generation prompt information is formatted according to the sentence format information to obtain target text information; Based on the target text information, a target image corresponding to the image generation prompt information is generated through the target image generation model.
2. The image generation method according to claim 1, characterized in that: When the generated sentence information meets the preset sentence condition, using the target embedded model corresponding to the embedded model identifier to construct a target image generation model, including: When the generated sentence information meets the preset sentence condition, determining the preset hot start mode as the image generation mode; According to the image generation mode, obtaining a target embedded model corresponding to the embedded model identifier; The target embedded model is combined with a preset image generation model to obtain the target image generation model.
3. The image generation method according to claim 1, characterized in that: The step of adjusting the format of the image generation prompt information according to the sentence format information to obtain target text information includes: Performing word segmentation on the image generation prompt information to obtain at least two word segmentation information; Using the sentence format information, adjusting the format of each word segmentation information to obtain target word segmentation information; The target word segmentation information is used to generate the target text information.
4. The image generation method according to claim 1, characterized in that: The step of generating a target image corresponding to the image generation prompt information based on the target text information and using the target image generation model includes: Inputting the target text information into the target image generation model to obtain model output information, wherein the model output information represents an image generated by the target image generation model based on the target text information; The target image is generated using the model output information.
5. The image generation method according to claim 1, characterized in that: After the semantic analysis is performed based on the image-generated prompt information to obtain the generated sentence information, the method further includes: Determining a sentence format of the generated sentence information; When the statement format conforms to a preset statement format, extracting the embedded model identifier and the statement format information from the generated statement information; When the embedded model identifier meets the preset identifier condition and the statement format information meets the preset statement format condition, it is determined that the generated statement information meets the preset statement condition.
6. The image generation method according to claim 5, characterized in that: After extracting the embedded model identifier from the generated sentence information, the method further includes: Get the preset identification set corresponding to the preset identification condition; When the embedded model identifier belongs to a preset identifier in the preset identifier set, it is determined that the embedded model identifier meets the preset identifier condition.
7. The image generation method according to claim 1, characterized in that: After the semantic analysis is performed based on the image-generated prompt information to obtain the generated sentence information, the method further includes: When the generated sentence information does not meet the preset sentence condition, determining the preset cold start mode as the image generation mode; According to the image generation mode, performing vector conversion on the image generation prompt information to obtain a prompt vector; Determining a target embedded model corresponding to the prompt vector; Combining the target embedded model with a preset image generation model to obtain the target image generation model; The format of the image generation prompt information is adjusted to obtain the target text information.
8. The image generation method according to claim 7, characterized in that: The determining the target embedded model corresponding to the prompt vector includes: Determining at least two model description vectors that match the prompt vector in a preset model description vector database; Determine an embedded model corresponding to each of the model description vectors; Determining a degree of match between the image generation prompt information and each of the embedded models; When the matching degree reaches a preset matching degree, the embedded model is determined as the target embedded model.
9. The image generation method according to claim 7, characterized in that: The step of formatting the image generation prompt information to obtain the target text information includes: Determining statement format information corresponding to the target embedded model; The image generation prompt information is formatted using the sentence format information to obtain the target text information.
10. An image generating device, characterized in that: include: An acquisition module is used to obtain image generation prompt information; An analysis module, configured to perform semantic analysis based on the image generation prompt information to obtain generated sentence information, wherein the generated sentence information includes an embedded model identifier and sentence format information; A processing module, configured to construct a target image generation model by using a target embedded model corresponding to the embedded model identifier when the generated sentence information meets a preset sentence condition, and to adjust the format of the image generation prompt information according to the sentence format information to obtain target text information; A generation module is used to generate a target image corresponding to the image generation prompt information based on the target text information and through the target image generation model.
11. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor is used to implement the image generation method described in any one of claims 1 to 9 when executing a program stored in a memory.
12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the image generating method according to any one of claims 1 to 9 is implemented.
