Image generation method and device, storage medium and program product

Abstract

The invention relates to an image generation method and device, a storage medium and a program product, and relates to the technical field of computers. The image generation method comprises the steps that in response to receiving of input content of a user in a current dialogue round, first prompt information for a first image is obtained, and the first image is generated in a historical dialogue round; determining second prompt information corresponding to the current dialogue round according to the input content and the first prompt information; and generating a second image corresponding to the current dialogue round according to the second prompt information.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to an image generation method and device, a storage medium, and a program product. Background Art

[0002] With the development of artificial intelligence technology, related applications have penetrated into every aspect of our lives, such as intelligent question answering, AI (Artificial Intelligence) raw images, etc. AI raw images are a method of generating images using artificial intelligence technology.

[0003] In the related art, a user can have multiple rounds of conversations with an intelligent agent in a conversation interface, and the image returned by the intelligent agent in each conversation round is determined based on the user's input content in that conversation round. Summary of the invention

[0004] This summary is provided to introduce concepts in a brief form that will be described in detail in the detailed description below. This summary is not intended to identify key features or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0005] According to a first aspect of some embodiments of the present disclosure, there is provided an image generating method, comprising:

[0006] In response to receiving input content from a user in a current conversation round, obtaining first prompt information for a first image, wherein the first image is generated in a historical conversation round;

[0007] Determining second prompt information corresponding to the current dialogue turn according to the input content and the first prompt information;

[0008] A second image corresponding to the current dialogue turn is generated according to the second prompt information.

[0009] According to a second aspect of some embodiments of the present disclosure, there is provided an image generating device, including:

[0010] an acquisition module, configured to acquire first prompt information for a first image in response to receiving input content from a user in a current conversation round, wherein the first image is generated in a historical conversation round;

[0011] a determination module, configured to determine second prompt information corresponding to the current dialogue turn according to the input content and the first prompt information;

[0012] A generating module is configured to generate a second image corresponding to the current dialogue turn according to the second prompt information.

[0013] According to a third aspect of some embodiments of the present disclosure, there is provided an image generating device, comprising: a memory; and a processor coupled to the memory, wherein the processor is configured to execute the image generating method of any embodiment described in the present disclosure based on instructions stored in the memory.

[0014] According to a fourth aspect of some embodiments of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the image generating method of any embodiment described in the present disclosure is performed.

[0015] According to a fifth aspect of some embodiments of the present disclosure, a computer program product is provided. When the computer program product is run on a computer, the computer implements the image generation method of any of the embodiments.

[0016] Other features, aspects and advantages of the present disclosure will become apparent from the following detailed description of exemplary embodiments of the present disclosure with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The preferred embodiments of the present disclosure are described below with reference to the accompanying drawings. The drawings described herein are used to provide a further understanding of the present disclosure. Each of the drawings together with the following detailed description is included in this specification and forms a part of the specification, and is used to explain the present disclosure. It should be understood that the drawings described below only relate to some embodiments of the present disclosure and do not constitute a limitation on the present disclosure. In the drawings:

[0018] Figure 1 is a flowchart illustrating an image generation method according to some embodiments of the present disclosure;

[0019] Figure 2 is a flowchart illustrating an image generating method according to other embodiments of the present disclosure;

[0020] Figure 3A is a flowchart illustrating an image generating method according to yet other embodiments of the present disclosure;

[0021] Figure 3B is a flowchart illustrating an image generating method according to yet other embodiments of the present disclosure;

[0022] Figure 4A is a flowchart illustrating an image generating method according to yet other embodiments of the present disclosure;

[0023] Figure 4B is a flowchart illustrating an image generating method according to yet other embodiments of the present disclosure;

[0024] Figure 5Ais a flowchart illustrating an image generating method according to yet other embodiments of the present disclosure;

[0025] Figure 5B is a flowchart illustrating an image generating method according to yet other embodiments of the present disclosure;

[0026] Figure 5C is a flowchart illustrating an image generating method according to yet other embodiments of the present disclosure;

[0027] Figure 6 is a block diagram illustrating an image generating apparatus according to some embodiments of the present disclosure;

[0028] Figure 7 is a block diagram showing an image generating device according to some other embodiments of the present disclosure;

[0029] Figure 8 A block diagram of an electronic device according to some embodiments of the present disclosure is shown.

[0030] It should be understood that, for ease of description, the sizes of the various parts shown in the drawings are not necessarily drawn according to the actual proportional relationship. The same or similar reference numerals are used in the various drawings to represent the same or similar parts. Therefore, once an item is defined in one drawing, it may not be further discussed in subsequent drawings. DETAILED DESCRIPTION

[0031] The following will be combined with the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure, but it is obvious that the described embodiments are only part of the embodiments of the present disclosure, rather than all the embodiments. The following description of the embodiments is actually only illustrative and is by no means intended to limit the present disclosure and its application or use. It should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein.

[0032] It should be understood that the various steps recorded in the method embodiments of the present disclosure can be performed in different orders, and / or performed in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown in the execution. The scope of the present disclosure is not limited in this respect. Unless otherwise specifically stated, the relative arrangement, numerical expressions and numerical values ​​of the parts and steps set forth in these embodiments should be interpreted as being merely exemplary and do not limit the scope of the present disclosure.

[0033] The term "include" and its variations used in the present disclosure mean an open term that includes at least the following elements / features but does not exclude other elements / features, that is, "including but not limited to". In addition, the term "include" and its variations used in the present disclosure mean an open term that includes at least the following elements / features but does not exclude other elements / features, that is, "including but not limited to". Therefore, include is synonymous with include. The term "based on" means "based at least in part on".

[0034] References throughout this specification to "one embodiment," "some embodiments," or "an embodiment" mean that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. For example, the term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; and the term "some embodiments" means "at least some embodiments." Furthermore, the appearances of the phrases "in one embodiment," "in some embodiments," or "in an embodiment" in various places throughout this specification do not necessarily all refer to the same embodiment, but may do so.

[0035] It should be noted that the concepts of "first", "second", etc. mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units. Unless otherwise specified, the concepts of "first", "second", etc. are not intended to imply that the objects described in this way must be in a given order in time, space, ranking, or any other manner.

[0036] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0037] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0038] The embodiments of the present disclosure are described in detail below in conjunction with the accompanying drawings, but the present disclosure is not limited to these specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. In addition, in one or more embodiments, specific features, structures or characteristics can be combined in any suitable manner that will be clear from the present disclosure by a person of ordinary skill in the art.

[0039] In the related art, there may be contextual associations between multiple rounds of conversations between a user and an intelligent agent, and the multiple rounds of conversations are independent of each other when generating images, resulting in poor consistency between images generated by multiple rounds of conversations and poor user experience.

[0040] The present disclosure provides a technical solution that can improve the consistency of image generation in multi-round dialogue scenarios and enhance user experience.

[0041] Figure 1 is a flowchart illustrating an image generating method according to some embodiments of the present disclosure.

[0042] like Figure 1 As shown, the image generation method includes: step S10, in response to receiving input content from a user in a current dialogue round, obtaining first prompt information for a first image, wherein the first image is generated in a historical dialogue round; step S20, determining second prompt information corresponding to the current dialogue round according to the input content and the first prompt information; and step S30, generating a second image corresponding to the current dialogue round according to the second prompt information. For example, the image generation method is performed by an image generation device.

[0043] A conversation round refers to the process in which a user sends input content and receives an image corresponding to the input content. The prompt is the text that guides the generation of the image desired by the user. The historical conversation round occurs before the current conversation round.

[0044] Taking the generation of an image including a duckling as an example, in a historical dialogue round, the image generation device receives the user's input content "draw a duckling" in the historical dialogue round, and generates a first image including a yellow duckling standing by the river for the user. In the current dialogue round, the image generation device receives the user's input content "put on sunglasses" in the current dialogue round, and obtains the first prompt information "a yellow duckling standing by the river" for the first image. The image generation device determines the second prompt information "a yellow duckling standing by the river wearing sunglasses" based on the first prompt information "a yellow duckling standing by the river" and the user's input content "put on sunglasses" in the current dialogue round. The image generation device generates a second image including a yellow duckling standing by the river wearing sunglasses based on the second prompt information "a yellow duckling standing by the river wearing sunglasses".

[0045] In the above embodiment, in the process of generating an image for the user in the current conversation round, based on the user's input content, the second prompt information is generated in combination with the first prompt information of the first image generated in the historical conversation round to guide the generation of the second image, so that the image generated in the current conversation round incorporates the user's contextual intention, thereby improving the consistency of image generation in multi-round conversation scenarios and improving the user experience.

[0046] The following will be combined Figures 2 to 5C The image generating method in other embodiments of the present disclosure is described in detail.

[0047] Figure 2 is a flowchart illustrating an image generating method according to some other embodiments of the present disclosure. Figure 2 and Figure 1 The difference is that Figure 2 Steps S21, S23 and S25 in Figure 1 An implementation of step S21 in the following description will only be described Figure 2 and Figure 1 The differences are shown below, and the similarities will not be repeated here.

[0048] like Figure 2 As shown, in step S21, the image feature of the first image is extracted to obtain the first image feature;

[0049] In step S23, text description information of the first image is generated according to the first image feature.

[0050] In step S25, the second prompt information is determined according to the input content, the first prompt information and the text description information of the first image.

[0051] For example, a feature extraction model may be used to extract image features of the first image to obtain the first image features. The feature extraction model may be a convolutional neural network or a recurrent neural network, which is not limited in the present disclosure.

[0052] Taking the first image feature as a feature vector as an example, a text generator can be used to map the feature vector of the first image to a text sequence, thereby obtaining text description information of the first image. The text generator can be implemented by a recursive neural network, a long short-term memory network, or based on attention.

[0053] Taking the generation of an image including a duckling as an example, the image generation device extracts features of a first image including a yellow duckling standing by a river, obtains first image features, and then generates text description information of the first image based on the first image features: "There is a duckling with yellow and brown feathers by the river, the duckling is standing on the grass by the river, and the river water is clear." Further, the image generation device determines the second prompt information "a duckling wearing sunglasses is standing on the grass by the river, the duckling has yellow and brown feathers, and the river water is clear" based on the input content "wearing sunglasses" in the current dialogue round, the first prompt information "a yellow duckling is standing by the river", and the text description information of the first image "a duckling with yellow and brown feathers is standing on the grass by the river, and the river water is clear." The image generation device generates a second image including a duckling with yellow and brown feathers wearing sunglasses and standing on the grass by the river based on the second prompt information "a yellow duckling wearing sunglasses is standing by the river."

[0054] In this embodiment, in the process of determining the second prompt information for generating the second image, based on the input content, not only the first prompt information of the first image generated in the historical dialogue round is incorporated, but also the text description information of the first image generated based on the image features of the first image is incorporated, thereby providing more contextual intention information for reference for the generation of the second image in the current dialogue round, thereby further improving the consistency of image generation in multi-round dialogue scenarios and further improving the user experience.

[0055] Figure 3A is a flowchart illustrating an image generating method according to some further embodiments of the present disclosure. Figure 3A and Figure 1 The difference is that Figure 3A Step S22, step S24 and step S26 are Figure 1 An implementation of step S20 in the following will only be described Figure 3A and Figure 1 The differences are shown below, and the similarities will not be repeated here.

[0056] like Figure 3A As shown, taking the input content including content in multiple modes as an example, in step S22, the content in multiple modes is multimodally fused to obtain multimodal fusion information. In step S24, text format content corresponding to the content in multiple modes is generated according to the multimodal fusion information. In step S26, the second prompt information is determined according to the text format content corresponding to the content in multiple modes and the first prompt information.

[0057] The content of multiple modes includes but is not limited to images, audio, text, etc., which is not limited in the present disclosure. For example, the user can input at least one of images, audio and text in the current dialogue turn to more fully express the user's intention.

[0058] Taking the generation of an image including a duck as an example, in the historical dialogue round, the image generation device receives the user's input content in the current dialogue round, including the input text "put on sunglasses" and the input image including a duck. The image generation device performs multimodal fusion on the input text "put on sunglasses" in the text mode and the input image in the image mode to obtain multimodal fusion information, and then generates text format content corresponding to the input content based on the multimodal fusion information.

[0059] For example, in an input image including a duckling, there is a duckling with yellow and brown feathers. The text format content corresponding to the input content may be "a duckling with yellow and brown feathers wearing sunglasses". The image generation device may further determine the second prompt information "a duckling with yellow and brown feathers wearing sunglasses standing by the river" based on the text format content corresponding to the input content "a duckling with yellow and brown feathers wearing sunglasses" and the first prompt information "a yellow duckling standing by the river".

[0060] The image generating device generates a second image including a duckling with yellow and brown feathers wearing sunglasses standing by the river according to the second prompt information “a duckling with yellow and brown feathers wearing sunglasses standing by the river”.

[0061] In this embodiment, in response to the user input content including content in multiple modalities, multimodal fusion can be used to better learn the user's intention and improve the ability to understand the user's intention, thereby improving the accuracy of generating the second prompt information, thereby further improving the consistency of image generation in multi-round dialogue scenarios, and further improving the user experience.

[0062] Multimodal fusion is, for example, feature-level fusion based on multimodality, that is, semantic features are extracted from content of different modalities respectively, and then the semantic features corresponding to the information of different modalities are fused to obtain fused features as multimodal fusion information. Extracting semantic features may, for example, include representing the content of each modality as a feature vector, and fusing semantic features includes merging feature vectors of content of multiple modalities in a feature space.

[0063] Multimodal fusion can also be, for example, model-level fusion based on multimodality, that is, the content of each modality is processed by an independent model, and then the outputs of the models corresponding to the contents of multiple modalities are combined to obtain multimodal fusion information.

[0064] Multimodal fusion may also be other types of fusion, for example, and the present disclosure does not limit this.

[0065] Figure 3B is a flowchart illustrating an image generating method according to some further embodiments of the present disclosure. Figure 3B and Figure 1 The difference is that Figure 3B Step S22' and step S24' are Figure 1 An implementation of step S20 in the following will only be described Figure 3B and Figure 1 The differences are shown below, and the similarities will not be repeated here.

[0066] like Figure 3B As shown, taking the input content including non-text format content as an example, in step S22', the non-text format content in the input content is converted into text format content.

[0067] In step S24', the second prompt information is determined according to the content in the text format obtained by the conversion and the first prompt information.

[0068] In the case where the input content also includes input text, the second prompt information is determined according to the content in the text format obtained by the conversion, the input text included in the input content, and the first prompt information.

[0069] For example, the input content may include an input text similar to the aforementioned embodiment "put on sunglasses" and an input image including a duckling, and the image generation device may convert the input image in image format into content in text format. For example, in an input image including a duckling, there is a duckling with yellow and brown feathers, and the content in text format corresponding to the input image may be "a duckling with yellow and brown feathers". The image generation device may determine the second prompt information "a duckling with yellow and brown feathers wearing sunglasses standing by the river" based on the content in text format corresponding to the input image "a duckling with yellow and brown feathers", the input text "put on sunglasses" and the first prompt information "a yellow duckling standing by the river".

[0070] In this embodiment, the non-text format content can be directly converted into a text format content, thereby improving the efficiency of image generation while ensuring the consistency of image generation in multi-round dialogue scenarios.

[0071] In some embodiments, the non-text format includes an image format, and converting the non-text format content in the input content into text format content includes the following steps.

[0072] First, feature extraction is performed on content in image format in the input content to obtain a second image feature.

[0073] Then, text description information corresponding to the content in the image format in the target content is generated according to the second image feature.

[0074] Taking the input content including the input image including a duckling in the aforementioned embodiment as an example, the image generation device extracts features from the input image to obtain a second image feature, and can generate text description information corresponding to the input image according to the second image feature. Taking the input image including a duckling with yellow and brown feathers as an example, the text description information corresponding to the input image can be "a duckling with yellow and brown feathers wearing sunglasses".

[0075] In some embodiments, the non-text format includes content in audio format, and the conversion of the non-text format content in the target content into text format content can be achieved in the following manner.

[0076] Speech recognition is performed on the content in the audio format of the target content to obtain text description information corresponding to the content in the audio format.

[0077] The input content includes input audio, and the content of the input audio is "this duck is standing on the grass by the river". Speech recognition can be directly performed on the input audio to obtain text description information corresponding to the input audio "this duck is standing on the grass by the river".

[0078] Figure 4A FIG4 is a flow chart showing an image generation method according to some other embodiments of the present disclosure. Figure 1 The difference is that Figure 4A The other steps S40 to S60 of the image generation method in some embodiments of the present disclosure are shown. Figure 4A and Figure 1 The differences are shown below, and the similarities will not be repeated here.

[0079] like Figure 4A As shown, in step S40, the second prompt information is sent to the user.

[0080] In step S50, in response to receiving adjustment information for the second prompt information from the user, the second prompt information is adjusted according to the adjustment information for the second prompt information to obtain adjusted second prompt information.

[0081] In step S60, an adjusted second image is generated according to the adjusted second prompt information.

[0082] Taking the second prompt information as "a yellow duckling wearing sunglasses standing by the river" in the aforementioned embodiment as an example, the image generation device sends the second prompt information "a yellow duckling wearing sunglasses standing by the river" to the user, and the user's adjustment information for the second prompt information is "change the yellow duckling to a brown duckling". After receiving the adjustment information for the second prompt information, the image generation device adjusts the second prompt information to obtain the adjusted second prompt information "a brown duckling wearing sunglasses standing by the river", and then generates the adjusted second image according to the adjusted second prompt information "a brown duckling wearing sunglasses standing by the river". The duckling in the adjusted second image is brown.

[0083] In this embodiment, by sending the second prompt information to the user, it is convenient for the user to provide adjustment information according to the user's intention, so that the second prompt information can be adjusted according to the user's adjustment information, thereby improving the flexibility and convenience of image generation while ensuring the consistency of image generation.

[0084] Figure 4B is a flowchart illustrating an image generating method according to some further embodiments of the present disclosure. Figure 4B and Figure 1 The difference is that Figure 4B The other steps S70 to S90 of the image generation method in some embodiments of the present disclosure are shown. Figure 4B and Figure 1 The differences are shown below, and the similarities will not be repeated here.

[0085] like Figure 4B As shown, in step S70, adjustment information for one or more elements in the second image is received from the user.

[0086] In step S80, the second prompt information is adjusted according to the adjustment information for one or more elements in the second image and the second image to obtain adjusted second prompt information.

[0087] In step S90, the one or more elements in the second image are adjusted according to the adjusted second prompt information to obtain an adjusted second image.

[0088] The adjustment information for the second image may be adjustment information for one or more elements in the second image. Taking the second image as an image of a yellow duckling wearing sunglasses standing by the river generated in the aforementioned embodiment as an example, after receiving the second image, the user sends adjustment information for the duckling element "turn the duckling feathers in the image brown" to the image generation device. The image generation device adjusts the second prompt information "a yellow duckling wearing sunglasses standing by the river" according to the adjustment information "turn the duckling feathers in the image brown" and the second image, and obtains the adjusted second prompt information "a brown duckling wearing sunglasses standing by the river", and then adjusts the duckling feathers in the second image to brown according to the adjusted second prompt information, and obtains the adjusted second image.

[0089] The adjustment information for the second image can also be adjustment information for the image style of the second image. Taking the second image of a yellow duckling wearing sunglasses and standing by a river generated in the aforementioned embodiment as an example, after receiving the second image, the user sends adjustment information for the duckling element, "turn the duckling into an anime-style duck", to the image generating device. The image generating device adjusts the second prompt information "a yellow duckling wearing sunglasses is standing by the river" based on the adjustment information "turn the duckling into an anime-style duck" and the second image, and obtains the adjusted second prompt information "a yellow duckling wearing sunglasses is standing by the river, and the duckling is anime-style". Then, based on the adjusted second prompt information, the duckling in the second image is turned into anime-style, to obtain the adjusted second image.

[0090] The adjustment information for the second image may also be adjustment information for other aspects of the second image, including but not limited to expanding the image, adjusting the background, etc. This disclosure does not limit this.

[0091] In this embodiment, based on the adjustment information for the second image input by the user, the second prompt information can be adjusted, so that based on the adjusted second prompt information, the user can make adjustments to the second image, thereby improving the flexibility and convenience of image generation while ensuring the consistency of image generation.

[0092] Figure 5A is a flowchart illustrating an image generating method according to some further embodiments of the present disclosure. Figure 5A and Figure 1 The difference is that Figure 5A Step S31 is Figure 1 An implementation of step S30 in the following will only be described Figure 5A and Figure 1 The differences are shown below, and the similarities will not be repeated here.

[0093] like Figure 5A As shown, in step S31, the second image is generated by using an image-to-image model according to the second prompt information and the first image.

[0094] For example, the second prompt information and the first image may be input into a graph-generated graph model to obtain the second image. The graph-generated graph model may process the second prompt information and the first image to generate the second image.

[0095] In this embodiment, the second prompt information and the first image are processed together by the graph-to-graph model, which can integrate more feature information of the first image, thereby further improving the consistency of image generation in multi-round dialogue scenarios.

[0096] Figure 5B is a flowchart illustrating an image generating method according to some further embodiments of the present disclosure. Figure 5B and Figure 1 The difference is that Figure 5B Step S31' is Figure 1 An implementation of step S30 in the following will only be described Figure 5B and Figure 1 The differences are shown below, and the similarities will not be repeated here.

[0097] like Figure 5B As shown, in step S31', the second image is generated according to the second prompt information using a text graph model.

[0098] In this embodiment, the second prompt information is processed using a text graph model to generate a second image, which can improve the image generation efficiency while ensuring the consistency of image generation in multi-round dialogue scenarios.

[0099] Figure 5C is a flowchart illustrating an image generating method according to some further embodiments of the present disclosure. Figure 5C and Figure 1 The difference is that Figure 5C Step S11 in Figure 1 An implementation of step S10 in the following will only be described Figure 5C and Figure 1 The differences are shown below, and the similarities will not be repeated here.

[0100] like Figure 5C As shown, in step S11, in response to receiving input content from the user in the current dialogue round, the first prompt information for generating the first image targeted by the input content is obtained, wherein the first image is generated in a historical dialogue round.

[0101] In this embodiment, the first image is an image corresponding to the input content of the user in the current dialogue round, so that the second image generated in the current dialogue round is more targeted, and the consistency and accuracy of image generation are improved.

[0102] In some embodiments, the historical conversation rounds in any of the foregoing embodiments include at least one historical conversation round adjacent to the current conversation round. For example, the historical conversation rounds include the conversation round before the current conversation round. The number of historical conversation rounds can be determined according to actual needs.

[0103] In some embodiments, the second prompt information in any of the foregoing embodiments is obtained through natural language processing. For example, the input content and the first prompt information are subjected to natural language processing to obtain the user's intention information, and then the model is used to rewrite and / or expand the input content and the first prompt information according to the user's intention information to obtain the second prompt information.

[0104] In some embodiments, the generation of the second prompt information and the generation of the second image in any of the above embodiments can be achieved with the help of a machine learning model. The machine learning model may include, for example, a large language model (LLM), a natural language processing model, and the like.

[0105] In some embodiments, the image generation method may be performed by an image generation device, which may include an intelligent agent or may not include an intelligent agent. In the case where the image generation device does not include an intelligent agent, the user may interact with the image generation device through the intelligent agent.

[0106] The above is an image generation method provided by some embodiments of the present disclosure. Figure 6 Describe the content publishing device in some embodiments of the present disclosure.

[0107] Figure 6 is a block diagram illustrating an image generating apparatus according to some embodiments of the present disclosure.

[0108] like Figure 6 As shown, the image generating device 6 includes an acquiring module 61 , a determining module 62 and a generating module 63 .

[0109] The acquisition module 61 is configured to acquire first prompt information for a first image in response to receiving input content from a user in a current conversation round, wherein the first image is generated in a historical conversation round; the determination module 62 is configured to determine second prompt information corresponding to the current conversation round based on the input content and the first prompt information; the generation module 63 is configured to generate a second image corresponding to the current conversation round based on the second prompt information.

[0110] The image generating device 6 can be used to perform Figure 1 In some embodiments, the image generating device 6 may also perform the following steps: Figures 2 to 5C Any of the steps shown.

[0111] It should be noted that the above modules are only logical modules divided according to the specific functions implemented by them, and are not used to limit the specific implementation methods. For example, they can be implemented in software, hardware, or a combination of software and hardware. In actual implementation, the above modules can be implemented as independent physical entities, or can also be implemented by a single entity (for example, a processor (CPU or DSP, etc.), an integrated circuit, etc.). In addition, the above modules are shown with dotted lines in the drawings to indicate that these modules may not actually exist, and the operations / functions implemented by them can be implemented by the processing circuit itself.

[0112] Figure 7 is a block diagram showing an image generating apparatus according to some other embodiments of the present disclosure.

[0113] like Figure 7 As shown, the image generating device 7 includes: a memory 71; and a processor 72 coupled to the memory 71, wherein the processor 72 is configured to execute the image generating method described in any of the aforementioned embodiments based on instructions stored in the memory 71.

[0114] The memory 71 is used to store one or more computer-readable instructions. The memory 71 may include any combination of various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory, including but not limited to random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), read-only memory (ROM), flash memory. The memory 71 may store, for example, an operating system, an application, a boot loader (BootLoader), a database, and other programs, and may also store various applications and various data.

[0115] The processor 72 is used to run computer-readable instructions to implement the image generation method described in any of the above embodiments. The specific implementation of each step of the image generation method can be found in the above embodiments, and the repeated parts are not repeated here.

[0116] The processor 72 and the memory 71 may communicate with each other directly or indirectly. For example, the processor 72 and the memory 71 may communicate with each other via a network. The network may include a wireless network, a wired network, and / or any combination of a wireless network and a wired network. The processor 72 and the memory 71 may also communicate with each other via a system bus, which is not limited in the present disclosure.

[0117] It should be noted that Figure 7 The components of the image generating device 7 shown are only exemplary and non-restrictive. The image generating device 7 may also have other components according to actual application requirements. The processor 72 may control other components in the image generating device 7 to perform desired functions.

[0118] The image generating device may be implemented by software, firmware and / or hardware, and may be integrated into an electronic device installed with relevant application programs.

[0119] The above is an image generating device in some embodiments of the present disclosure.

[0120] Figure 8 A block diagram of an electronic device according to some embodiments of the present disclosure is shown.

[0121] Figure 8 The electronic device 8 shown may be a computer system with a dedicated hardware structure, which can execute corresponding functions when a relevant application program is installed.

[0122] Electronic devices include, but are not limited to, mobile terminals such as smart phones, laptops, personal digital assistants (PDA), tablet computers (Tablet Personal Computer, Tablet PC), PMP (portable multimedia player), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), wearable devices, etc., as well as fixed terminals such as digital televisions, desktop computers, etc.

[0123] like Figure 8 As shown, the central processing unit (CPU) 81 performs various processes according to the program stored in the read-only memory (ROM) 82 or the program loaded from the storage part 88 to the random access memory (RAM) 83. In the RAM 83, data required when the CPU 81 performs various processes, etc. is stored as needed. The central processing unit is only exemplary, and it can also be other types of processors, such as the various processors described above. The ROM 82, RAM 83 and the storage part 88 can be various forms of computer-readable storage media. It should be noted that although Figure 8 ROM 82, RAM 83 and storage section 88 are shown separately in FIG. 8 , but one or more of them may be combined or located in the same or different memory or storage modules.

[0124] The CPU 81, the ROM 82, and the RAM 83 are connected to one another via a bus 84. To the bus 84, an input / output interface 85 is also connected.

[0125] The following components are connected to the input / output interface 85: an input section 86, such as a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output section 87, including a display, such as a cathode ray tube (CRT), a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage section 88, including a hard disk, a magnetic tape, etc.; and a communication section 89, including a network interface card such as a LAN card, a modem, etc. The communication section 89 allows communication processing to be performed via a network such as the Internet. It is easy to understand that although Figure 8 The various devices or modules in the electronic device 8 are shown to communicate via a bus 84, but they may also communicate via a network or other means, wherein the network may include a wireless network, a wired network, and / or any combination of a wireless network and a wired network.

[0126] A drive 810 is also connected to the input / output interface 85 as needed. A removable medium 811 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is mounted on the drive 810 as needed so that a computer program read therefrom is installed into the storage section 88 as needed.

[0127] When the above-described series of processing is implemented by software, a program constituting the software can be installed from a network such as the Internet or a storage medium such as the removable medium 811 .

[0128] According to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which, when running on a computer, enables the computer to implement the image generation method described in any of the aforementioned embodiments. The computer program product includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 89, or installed from the storage part 88, or installed from the ROM 82. When the computer program is executed by the CPU 81, the image generation method of the embodiment of the present disclosure is executed.

[0129] It should be noted that, in the context of the present disclosure, a computer-readable medium may be a tangible medium that may contain or store a program for use by an instruction execution system, apparatus, or device or for use in conjunction with an instruction execution system, apparatus, or device.

[0130] The computer readable medium may be a computer readable storage medium, or a computer readable signal medium, or any combination of the two.

[0131] Computer-readable storage media include, but are not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or components, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, electrical connections with one or more wires, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, device, or device. A computer program is stored on a computer-readable storage medium, and when the program is executed by a processor, the image generation method described in any of the foregoing embodiments is implemented.

[0132] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer readable program codes. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer readable signal media may also be any computer readable medium other than a computer readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, device, or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0133] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0134] In some embodiments, a computer program product is also provided. When the computer program product is run on a computer, the computer is enabled to implement the image generation method described in any of the above embodiments.

[0135] In some embodiments, a computer program is further provided, comprising: instructions, which, when executed by a processor, cause the processor to execute the image generation method of any of the above embodiments. For example, the instructions may be embodied as computer program codes.

[0136] In embodiments of the present disclosure, computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or combinations thereof, including but not limited to object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code may be executed entirely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or entirely on a remote computer or server. In situations involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0137] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0138] The functions described above may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0139] Although some specific embodiments of the present disclosure have been described in detail by way of example, it should be understood by those skilled in the art that the above examples are for illustration only and are not intended to limit the scope of the present disclosure. It should be understood by those skilled in the art that the above embodiments may be modified without departing from the scope and spirit of the present disclosure. The scope of the present disclosure is defined by the appended claims.

Claims

1. A method for generating an image, comprising: In response to receiving input content from a user in a current conversation round, obtaining first prompt information for a first image, wherein the first image is generated in a historical conversation round; Determining second prompt information corresponding to the current dialogue turn according to the input content and the first prompt information; A second image corresponding to the current dialogue turn is generated according to the second prompt information.

2. The image generation method according to claim 1, wherein: Determining, according to the input content and the first prompt information, second prompt information corresponding to the current dialogue turn includes: Extracting image features of the first image to obtain first image features; Generating text description information of the first image according to the first image feature; The second prompt information is determined according to the input content, the first prompt information and the text description information of the first image.

3. The image generation method according to claim 1 or 2, wherein: The input content includes content in multiple modes, and determining, according to the input content and the first prompt information, the second prompt information corresponding to the current dialogue turn includes: Performing multimodal fusion on the contents of the multiple modalities to obtain multimodal fusion information; Generating content in a text format corresponding to the content of the multiple modalities according to the multimodal fusion information; The second prompt information is determined according to the content in text format corresponding to the content of the multiple modalities and the first prompt information.

4. The image generation method according to claim 1 or 2, further comprising: Sending the second prompt information to the user; In response to receiving adjustment information for the second prompt information from the user, adjusting the second prompt information according to the adjustment information for the second prompt information to obtain adjusted second prompt information; An adjusted second image is generated according to the adjusted second prompt information.

5. The image generation method according to claim 1 or 2, wherein: The input content includes content in a non-text format, and determining, according to the input content and the first prompt information, the second prompt information corresponding to the current dialogue turn includes: Converting non-text content in the input content into text content; The second prompt information is determined according to the content in the text format obtained by the conversion and the first prompt information.

6. The image generation method according to claim 5, wherein: The non-text format includes an image format, and converting the non-text format content in the input content into text format content includes: Extracting features of content in image format in the input content to obtain second image features; According to the second image feature, text description information corresponding to the content in the image format in the target content is generated.

7. The image generation method according to claim 5, wherein: The non-text format includes content in an audio format, and converting the non-text format content in the target content into content in a text format includes: Speech recognition is performed on the content in the audio format of the target content to obtain text description information corresponding to the content in the audio format.

8. The image generation method according to claim 1 or 2, wherein: Generating, according to the second prompt information, a second image corresponding to the current dialogue turn includes: Generate the second image using an image-to-image model according to the second prompt information and the first image; or The second image is generated according to the second prompt information using a text graph model.

9. The image generation method according to claim 1 or 2, further comprising: receiving adjustment information for the second image from the user; adjusting the second prompt information according to the adjustment information for the second image and the second image to obtain adjusted second prompt information; The second image is adjusted according to the adjusted second prompt information to obtain an adjusted second image.

10. The image generation method according to claim 1 or 2, wherein: Acquiring first prompt information for the first image includes: The first prompt information for generating the first image targeted by the input content is obtained.

11. The image generation method according to claim 1 or 2, wherein: The historical conversation turns include at least one historical conversation turn adjacent to the current conversation turn.

12. The image generation method according to claim 1 or 2, wherein: The second prompt information is obtained through natural language processing.

13. An image generating device, comprising: an acquisition module, configured to, in response to receiving input content from a user in a current dialogue round, acquire first prompt information for a first image, wherein the first image is generated in a historical dialogue round; a determination module, configured to determine second prompt information corresponding to the current dialogue turn according to the input content and the first prompt information; A generating module is configured to generate a second image corresponding to the current dialogue turn according to the second prompt information.

14. An image generating device, comprising: Memory; as well as A processor coupled to the memory, wherein the processor is configured to execute the image generation method according to any one of claims 1 to 12 based on instructions stored in the memory.

15. A computer-readable storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the image generating method according to any one of claims 1 to 12 is implemented. 16 . A computer program product, when the computer program product is run on a computer, enables the computer to implement the image generation method according to any one of claims 1 to 12.

Images