Image generation method, device, equipment, storage medium and program product

Abstract

The present disclosure provides an image generation method, apparatus, device, storage medium and program product, the method comprising: obtaining label information of an intelligent agent from an intelligent agent image and / or an intelligent agent persona respectively through a label extraction model; obtaining subject information; generating a scene description according to the label information and the subject information through a language model; generating a frame of artificial intelligence image according to the scene description through a text graph model, wherein the artificial intelligence image contains the intelligent agent. The method provided by the present disclosure can generate AI graphics with good quality.

Description

Technical Field

[0001] The present invention relates to the field of image generation technology, and in particular to an image generation method, device, equipment, storage medium and program product. Background Art

[0002] Generative AI (Artificial Intelligence Generated Content, AIGC) refers to a technology based on artificial intelligence technical methods such as generative adversarial networks and large pre-trained models, which generates relevant content with appropriate generalization capabilities through learning and identifying existing data. The core idea of ​​AIGC technology is to use artificial intelligence algorithms to generate content with certain creativity and quality. Through training models and learning from a large amount of data, AIGC can generate relevant content based on the text description entered by the user. However, the text description entered by the user is usually relatively thin, resulting in poor quality of the generated AI images. Summary of the invention

[0003] The embodiments of this specification provide an image generation method, apparatus, device, computer-readable storage medium, and computer program product that can generate AI graphics with good quality.

[0004] In a first aspect, one or more embodiments of the present specification provide an image generation method, the method comprising: obtaining label information of the agent from the agent image and / or the agent persona respectively through a label extraction model; obtaining subject information; generating a scene description based on the label information and the subject information through a language model; generating a frame of artificial intelligence image based on the scene description through a text graph model, wherein the artificial intelligence image includes the agent.

[0005] According to the image generation method provided by some embodiments of the present specification, the acquiring of subject information includes: acquiring pre-set subject information; or, randomly generating the subject information; or, acquiring the subject information according to label information of the agent.

[0006] According to the image generation method provided in some embodiments of the present specification, the subject information includes at least one of the following: a daily scene theme, a holiday theme, an occupation theme, and an interest theme.

[0007] According to the image generation method provided by some embodiments of the present specification, the subject information includes at least two types of subject information, and different types of subject information have different levels.

[0008] The image generation method provided according to some embodiments of the present specification also includes: generating different scene descriptions according to the label information and the different types of subject information through the language model; generating multiple frames of AI images corresponding to each type of subject information according to the different scene descriptions through the text graph model; wherein the number of AI images corresponding to subject information of different levels is different, and the higher the level of the subject type, the fewer the number of AI images corresponding to the corresponding subject type.

[0009] According to the image generation method provided by some embodiments of the present specification, the label information includes at least one of the following label information of the intelligent agent: age, occupation, personality, interest, figure, clothing, action, and expression.

[0010] According to the image generation method provided in some embodiments of the present specification, the generating of the scene description according to the label information and the subject information through the language model includes: generating at least one of the following supplementary information according to the label information and the subject information: scene information, composition information, posture information of the intelligent body, clothing accessories information, and the scene description includes the supplementary information.

[0011] According to some embodiments of the present specification, the image generation method further includes: obtaining image style information from the intelligent agent image through a style extraction model; generating a frame of artificial intelligence image according to the scene description through the Vincent graph model, including: generating a frame of artificial intelligence image according to the scene description and the image style information through the Vincent graph model.

[0012] According to the image generation method provided by some embodiments of the present specification, the style information includes at least one of the following: color matching information, local texture information, material information, brushstroke information, lighting information, and composition information.

[0013] According to some embodiments of the present specification, the image generation method further includes: acquiring facial features of the intelligent agent from the intelligent agent image, and generating a frame of artificial intelligence image according to the scene description through the Vincent graph model, including: generating a frame of artificial intelligence image according to the scene description and the facial features through the Vincent graph model.

[0014] According to the image generation method provided in some embodiments of the present specification, the method of obtaining the label information of the intelligent agent from the intelligent agent image and / or the intelligent agent personality respectively also includes: extracting the facial features of the target character from the reference image through a facial extraction model; obtaining a text description input by a user; and generating the intelligent agent image according to the facial features of the target character and the text description.

[0015] In a second aspect, one or more embodiments of the present specification provide an image generation device, including: a label extraction model, used to obtain label information of an intelligent agent from an intelligent agent image and / or an intelligent agent personality, respectively; a first acquisition module, used to obtain subject information; a language model, used to generate a scene description based on the label information and the subject information; a text graph model, used to generate a frame of artificial intelligence image based on the scene description, wherein the artificial intelligence image contains the intelligent agent.

[0016] In a third aspect, one or more embodiments of the present specification provide an image generating device, comprising: a processor and a memory, wherein the memory stores executable code, and when the executable code is processed by the processor, the processor can execute any one of the methods described.

[0017] In a fourth aspect, one or more embodiments of the present specification provide a computer-readable storage medium having executable code stored thereon, which, when executed by a processor of an electronic device, enables the electronic device to execute any one of the methods described.

[0018] In a fifth aspect, one or more embodiments of the present specification provide a computer program product, including a computer program or computer executable instructions, which, when executed by a processor, implements any of the methods described.

[0019] The embodiments of this specification provide an image generation method and related devices, equipment, computer-readable storage media, and computer program products, which obtain the label information of the agent from the agent image and / or the agent persona, input the label information and subject information into the language model to expand the scene description, and then use the text graph model to generate an artificial intelligence image with the subject and the agent based on the scene description. Compared with the long writing time and unstable writing quality caused by the user directly writing the scene description for the agent image and then generating the artificial intelligence image, the embodiments of this specification can efficiently and quickly generate high-quality artificial intelligence images. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] This specification will be further described in the form of exemplary embodiments, which will be described in detail by way of the accompanying drawings. These embodiments are not restrictive, and in these embodiments, the same numbers represent the same structures or steps.

[0021] Figure 1 It is a flowchart of an embodiment of the image generation method of this specification.

[0022] Figure 2 It is a flowchart of another embodiment of the image generating method of the present specification.

[0023] Figure 3 It is a flowchart of another embodiment of the image generating method of the present specification.

[0024] Figure 4 It is a schematic block diagram of an embodiment of an image generating device of the present specification.

[0025] Figure 5 It is a structural schematic diagram of an embodiment of the image generating device of this specification. DETAILED DESCRIPTION

[0026] In order to more clearly illustrate the technical solutions of the embodiments of this specification, the embodiments will be described in detail below with reference to the accompanying drawings. Obviously, the contents described below are some examples or embodiments of this specification. For ordinary technicians in this field, without paying creative work, the technical solutions or means disclosed in this specification can also be applied to other scenarios based on these technical contents.

[0027] It should be understood that the "system", "device", "unit" and / or "module" used in this specification is a method for distinguishing different components, elements, parts, portions or assemblies at different levels. However, if other words can achieve the same purpose, the words can be replaced by other expressions.

[0028] Unless otherwise specified, technical terms used in this specification to describe components, elements, etc. do not refer to the singular, but may also include the plural. Generally speaking, terms such as "include", "comprise", etc. only indicate that the steps, elements, or components that have been clearly identified are included, and these steps, elements, and components do not constitute an exclusive list, such as the method or device described may also include other steps or components.

[0029] It should also be understood that the term "and / or" used in this specification refers to any or all possible combinations of one or more associated listed items. Although the terms "first", "second", "third", etc. may be used in this specification to describe various information, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this specification, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of this specification, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0030] Flowcharts are used in this specification to illustrate the operation steps performed by the device or system of the relevant embodiments, but unless otherwise specified, the order used to describe these steps should not be understood as a limitation on the order in which the steps are performed. A person of ordinary skill in the art can adjust the order in which these steps are performed based on the knowledge and information conveyed by the embodiments of this specification, and the adjustment includes but is not limited to swapping the order, merging multiple steps, and splitting a certain step.

[0031] Figure 1 FIG. 1 is a flow chart of an embodiment of the image generation method of the present specification. Figure 1 As shown, the image generating method includes the following steps.

[0032] Step S101, obtaining the label information of the agent from the agent image and / or the agent persona respectively through a label extraction model.

[0033] The intelligent agent can be a virtual image. The image generation method in this specification can be applied to virtual social interaction. Based on AI technology, users create an intelligent agent that can communicate and interact with users, giving users an immersive social experience. Optionally, the intelligent agent can be a person or an animal. For example, the intelligent agent can be a character created by the user based on a person or animal in a novel or film or television work, or a character created based on a person or animal encountered in daily life, and so on. There are many ways to create the intelligent agent, for example, obtaining a text description output by the user, the text description including a description of the intelligent agent, such as gender, occupation, appearance, clothing, etc.; generating an AI image based on the text description through a text graph model. The intelligent agent image can be the AI ​​image.

[0034] The intelligent agent persona can be determined and stored by obtaining the text description input by the user; or it can be determined and stored by generating multiple text description templates and determining the intelligent agent persona according to the text description template selected by the user. There are also multiple ways to generate text description templates, for example, different text description templates can be stored in advance, or the content can be expanded according to the keywords input by the user to generate the corresponding text description template.

[0035] After obtaining the intelligent agent image and / or intelligent agent personality, the information related to the intelligent agent can be extracted from it as label information through a trained label extraction model. The label information may include the intelligent agent's age, occupation, personality, interests, body shape, clothing, movements, expressions and other information.

[0036] Step S102, obtaining subject information.

[0037] Optionally, the subject information may include at least one of the following subjects: daily scene subject, holiday subject, occupation subject, interest subject. The daily scene subject may include any scene in daily life, such as class, sports, cooking, etc. The holiday subject may include any holiday, such as Christmas, Spring Festival, etc. The occupation subject may include any occupation, such as chef, teacher, etc. The interest subject may include any hobby, such as skiing, dancing, etc.

[0038] Optionally, the subject information may be preset, randomly generated, or generated based on the tag information of the agent. For example, subject information more relevant to the agent is generated based on the agent's gender, occupation, or hobbies. For example, if the agent is a student who likes outdoor sports, the subject information may include class themes and outdoor sports themes. For another example, if the agent is a dancer who likes cooking, the subject information may include performance themes and cooking themes.

[0039] Step S103: Generate a scene description according to the tag information and the topic information through a language model.

[0040] The language model uses the content in the tag information and the subject information as keywords to supplement the composition information and scene content element information related to the tag information and the subject information. In some examples, the language model generates at least one of the following supplementary information based on the tag information and the subject information: scene information, composition information, action and expression information of the intelligent agent, and clothing and accessories information, and the scene description includes the supplementary information.

[0041] The scene information may include some conventional scene contents, such as whether it is indoors or outdoors, and some corresponding conventional scene elements. For example, the scene information includes that the agent is in a classroom, and some common objects in the classroom, such as a blackboard, a desk, etc. Optionally, the scene information also includes scene elements that are common sense related to the theme. For example, for the Christmas theme, the supplementary information generated by the language model includes at least one element of a Christmas market, snow, elk, Christmas tree, and Santa Claus. The composition information may include whether the generated image is a distant view, a mid-range view, or a close-up view, the position of the agent in the image, etc. Optionally, the composition information may be supplemented according to the composition rules and scene information corresponding to different scenes.

[0042] For example, if the label information indicates that the agent is a 16-year-old girl and the theme information is Christmas, the language model will expand the common sense scene details that are consistent with the girl and Christmas based on the agent's information, such as other Christmas-related elements that appear in the scene, the girl's clothing accessories and actions related to Christmas, etc., to form a more detailed scene description.

[0043] Step S104, generating a frame of artificial intelligence image according to the scene description through a text graph model, wherein the artificial intelligence image includes the intelligent agent.

[0044] In some embodiments, the label information of the agent extracted from the agent image also includes facial features of the agent so that the text graph model can generate an artificial intelligence image containing the agent.

[0045] In the embodiment of this specification, the label information of the intelligent agent is obtained from the intelligent agent image and / or the intelligent agent persona, and the label information and the subject information are first input into the language model to expand the scene description, and then the text graph model is used to generate an artificial intelligence image with the subject and the intelligent agent according to the scene description. Compared with the user directly writing the scene description for the intelligent agent image and then generating the artificial intelligence image, which results in a long writing time and unstable writing quality, resulting in unstable quality of the generated artificial intelligence image, the embodiment of this specification can efficiently and quickly generate high-quality artificial intelligence images.

[0046] In the above embodiment, the label information of the agent obtained from the agent image by the label extraction model also includes the facial features of the agent. In some embodiments, facial features of the agent image may be extracted by a facial extraction model trained separately for facial feature extraction instead of a label extraction model, and the extracted facial features may be input into the text graph model so that the text graph model can more accurately generate an artificial intelligence image with the agent.

[0047] Figure 2 FIG. 1 is a flow chart of another embodiment of the image generation method of the present specification. Figure 2 As shown, the image generating method includes the following steps.

[0048] Step S201, acquiring facial features of the agent from the agent image using a facial extraction model.

[0049] In some embodiments, the facial extraction model can be a mapping (embedding) space model trained for face extraction. After the intelligent agent image is input into the facial extraction model, the facial extraction model generates a corresponding mapping space based on the facial features of the intelligent agent, and the mapping space consists of thousands of dimensional vectors. The scene description and the mapping space are input into the Wensheng graph model together to generate a frame of AI image with the intelligent agent as the protagonist. Optionally, the mapping space model can be trained using the existing model architecture BIT. The mapping space model has good generalization and robustness, and can enable the mapping space model to quickly and effectively extract facial features with only one intelligent agent image input, rather than requiring multiple pictures to extract facial features as in traditional models.

[0050] Step S202: Obtain the label information of the agent from the agent image and / or the agent persona respectively through a label extraction model.

[0051] Step S203, obtaining subject information.

[0052] Step S204: Generate a scene description according to the tag information and the topic information using a language model.

[0053] For the description of steps S202 - 204 , please refer to the description of steps S101 - 103 , which will not be repeated here.

[0054] Step S205: Generate an artificial intelligence image frame according to the scene description and the facial features through the text image model.

[0055] There are many ways to create an intelligent agent. In some embodiments, the intelligent agent image in step S101 is an AI image generated by the text graph model based on the text description input by the user. Optionally, before step S101, when generating the intelligent agent image, in addition to obtaining the text description output by the user, a reference image specified by the user can also be obtained, and the facial features of the character in the reference image can be extracted; the text graph model generates the intelligent agent image based on the text description output by the user and the acquired facial features. In some embodiments, the face of the intelligent agent in the intelligent agent image can be the same as the face in the reference image, or it can be similar. Among them, it can be understood that the face of the intelligent agent in the intelligent agent image is the same as the face in the reference image, which can mean that the expression and demeanor are exactly the same, or it can be the face of the same person or animal at different angles, different demeanors or different expressions.

[0056] In some embodiments, when there are at least two characters in the reference image, the user can select one of the characters, and when extracting the facial features of the character in the reference image, it specifically includes extracting the facial features of the character selected by the user; or, when the user selects two or more characters, the facial features of the at least two selected characters can also be extracted. The Wensheng graph model generates an intelligent body image according to the text description output by the user and the facial features of the at least two characters. In this example, the facial features of the intelligent body in the generated intelligent body image are integrated with the facial features of the at least two characters. Optionally, the user can also specify the weights of different characters, and the Wensheng graph model can also integrate the facial features of the at least two characters according to the weights of different characters when generating the intelligent body image, wherein the greater the weight of a certain character, the more the Wensheng graph model refers to the facial features of the character to generate the intelligent body. In some examples, when the reference image has at least two characters, the user may not select the character or specify the weight of the character, but the facial features of all characters in the reference image are integrated to generate the intelligent body image by default.

[0057] In some embodiments, the user may also want the style of the generated artificial intelligence image to be consistent with the style of the agent image. Figure 3 FIG. 1 is a flow chart of another embodiment of the image generation method of the present specification. Figure 3 As shown, the method includes the following steps.

[0058] Step S301, obtaining image style information from the agent image through a style extraction model.

[0059] In some embodiments, the image style information may include at least one of the following information: color information, local texture information, material information, brushstroke information, lighting information, and composition information. In some embodiments, the style extraction model is an embedding space model trained for style extraction. The mapping space model can adopt the existing model architecture BIT and be trained through images of different styles, such as different types of illustrations, paintings, photographs, and other works.

[0060] Step S302: Obtain the label information of the agent from the agent image and / or the agent persona respectively through a label extraction model.

[0061] Step S303, obtaining subject information.

[0062] Step S304: Generate a scene description according to the tag information and the topic information using a language model.

[0063] For the description of steps S302 - 304 , please refer to the description of steps S101 - 103 , which will not be repeated here.

[0064] Step S305: Generate an artificial intelligence image frame according to the scene description and the image style information through the text-based graph model.

[0065] In some embodiments, the style extraction model is obtained by training a mapping (embedding) space model for the style. The mapping space model can adopt the existing model architecture BIT. The training data of the mapping space model can include image works of different styles, such as illustrations, paintings, fine arts or photographs of various styles. The mapping space model has good style learning ability, generalization ability and robustness, and can quickly learn style information from one or a few reference images. When the reference image is input into the style extraction model, the style extraction model outputs a mapping space of vectors with multiple dimensions (for example, thousands of dimensions) based on the reference image to reflect the style information of the reference image.

[0066] In this embodiment, the style information extracted in step S301 is input into the text-generated graph model together with the scene description, so that the AI ​​image generated by the text-generated graph model can continue the style of the intelligent agent image.

[0067] In some embodiments, in addition to extracting style information, the extraction model also extracts facial features of the agent. By maintaining the style and facial features, the language model is used to generate text descriptions to change the clothing and scenes of the agent, and different AI images can be generated. These AI images can interact with users as different skins of the agent to give users a stronger sense of immersion.

[0068] In some embodiments, the acquired subject information may include at least two types, and the levels of different types of subject information are different. For each type of subject information in the at least two types of acquired subject information, different scene descriptions are generated according to the label information and the type of subject information through the language model, and multiple frames of AI images corresponding to each type of subject information are generated according to the different scene descriptions through the text graph model. Moreover, the specific number of AI images generated corresponding to each type of subject information is related to the level of the subject information. The number of AI images corresponding to subject information of different levels is different, and the number of AI images corresponding to the higher the level of the subject type is. For example, holiday themes are more scarce than daily scene themes. When generating AI images, the number of AI images corresponding to holiday themes is controlled to be less than that of AI images corresponding to daily scene themes. Optionally, the ratio of the number of AI images corresponding to different types of subject information can be preset.

[0069] Figure 4 FIG. 1 is a schematic block diagram of an embodiment of an image generating device of the present specification. Figure 4As shown, the image generation device 400 includes a label extraction model 401, a first acquisition module 402, a language model 403, and a text image model 404.

[0070] The label extraction model 401 is used to obtain the label information of the agent from the agent image and / or the agent persona respectively.

[0071] The first acquisition module 402 is used to acquire subject information.

[0072] The language model 403 is used to generate a scene description according to the tag information and the topic information.

[0073] The Wensheng graph model 404 is used to generate a frame of artificial intelligence image according to the scene description, and the artificial intelligence image includes the intelligent agent.

[0074] In some embodiments, when acquiring the topic information, the first acquisition module 402 is specifically used to: acquire pre-set topic information; or, randomly generate the topic information; or, acquire the topic information according to the tag information of the agent.

[0075] Optionally, the subject information includes at least one of the following themes: daily scene theme, festival theme, occupation theme, interest theme.

[0076] Optionally, the subject information may include at least two categories of subject information, and different categories of subject information have different levels. The language model 403 is also used to generate different scene descriptions according to the label information and the different categories of subject information. The text graph model 404 is also used to generate multiple frames of AI images corresponding to each category of subject information according to the different scene descriptions. Among them, the number of AI images corresponding to subject information of different levels is different, and the number of AI images corresponding to the higher level of the subject type is smaller.

[0077] Optionally, the label information includes at least one of the following label information of the agent: age, occupation, personality, interests, figure, clothes, actions, and expressions.

[0078] In some embodiments, when the language model 403 generates a scene description based on the tag information and the subject information, it is specifically used to: generate at least one of the following supplementary information based on the tag information and the subject information: scene information, composition information, posture information of the intelligent body, clothing accessories information, and the scene description includes the supplementary information.

[0079] In some embodiments, the apparatus 400 further comprises: a style extraction model for acquiring image style information from the agent image. When the text graph model 404 generates a frame of artificial intelligence image according to the scene description, it is specifically used to: generate a frame of artificial intelligence image according to the scene description and the image style information through the text graph model.

[0080] Optionally, the style information includes at least one of the following: color information, local texture information, material information, brushstroke information, lighting information, and composition information.

[0081] In some embodiments, the apparatus 400 further comprises: a face extraction model for acquiring facial features of the agent from the agent image. When the viz-image model 404 generates a frame of artificial intelligence image according to the scene description, it is specifically used to: generate a frame of artificial intelligence image according to the scene description and the facial features.

[0082] In some embodiments, the apparatus 400 further includes: a facial extraction model for extracting facial features of the target character from the reference image before acquiring the label information of the agent from the agent image and / or the agent persona; and a second acquisition module for acquiring a text description input by a user. The text graph model 204 is used to generate the agent image according to the facial features of the target character and the text description.

[0083] Some embodiments of the present specification may also be implemented as a computer-readable storage medium (or a non-transitory machine-readable storage medium or a machine-readable storage medium) on which an executable code (or a computer program or a computer instruction code) is stored. When the executable code (or a computer program or a computer instruction code) is executed by a processor of an electronic device (or a server, etc.), the processor executes part or all of the steps of the above method shown in the embodiments of the present specification.

[0084] Some embodiments of the present specification may also be implemented as a computer program product, including a computer program or computer executable instructions. When the computer program or computer executable instructions are executed by a processor, part or all of the steps of the above method of the present specification are implemented.

[0085] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this specification. Figure 5As shown, the electronic device 500 includes a processor 501 and a memory 502. The memory 502 is used to store a computer program. The processor 501 is used to execute the above-mentioned method for editing text when calling the computer program. Further, the electronic device may also include a bus, a microphone, a speaker, a display and a camera. Among them, the processor 501, the memory 502, the microphone, the speaker, the display and the camera communicate through the bus, and can also communicate through other means such as wireless transmission.

[0086] It is understood that the processor in the embodiments of this specification may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. A general-purpose processor may be a microprocessor or any conventional processor.

[0087] The method steps in the embodiments of this specification can be implemented by hardware or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, and the software modules can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC.

[0088] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of this specification is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state disk (SSD)), etc.

[0089] It should be understood that the various numerical numbers involved in the embodiments of this specification are only used for the convenience of description and are not used to limit the scope of the embodiments of this specification.

[0090] The embodiments of the present specification have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. An image generation method, characterized in that: include: Acquire a text description input by a user and a reference image specified by the user, wherein the reference image includes at least two characters, and extract facial features of the at least two characters in the reference image; The text-generated graph model generates an intelligent agent image according to the text description input by the user and the facial features of the at least two characters; When generating the intelligent agent image, fusing the facial features of the at least two characters according to the weights respectively specified by the user for the different characters; Or, fusing facial features of all characters in the reference image to generate an intelligent agent image; The intelligent agent is a virtual image; Acquire label information of the agent from the agent image and / or the agent persona respectively through a label extraction model; Get subject information; Generate a scene description including at least one item of supplementary information by using a language model with the tag information and content in the subject information as keywords; The at least one item of supplementary information includes: scene information, composition information, action and expression information of the agent, and clothing and accessories information; Generate an artificial intelligence image according to the scene description by using a text graph model, wherein the artificial intelligence image includes the intelligent agent; The step of generating an artificial intelligence image according to the scene description by using a text-generated graph model includes: Generate an artificial intelligence image according to the scene description and the facial features of the agent through a text graph model; the facial features of the agent are a fusion of the facial features of the at least two characters; the facial features of the agent are obtained from the agent image through a face extraction model or a label extraction model; or, Through the text-generated graph model, a frame of artificial intelligence image is generated according to the scene description and image style information; the image style information is obtained from the intelligent agent image through a style extraction model, and the style extraction model is a mapping space model. When training the style extraction model, after the reference image is input into the style extraction model, the style extraction model outputs a mapping space of a vector with multiple dimensions based on the reference image to reflect the style information of the reference image.

2. The image generation method according to claim 1, characterized in that: The acquisition of subject information includes: Get information about a pre-set topic; or randomly generate the subject information; or The subject information is obtained according to the tag information of the agent.

3. The image generation method according to claim 1, characterized in that: The subject information includes at least one of the following: daily scene theme, festival theme, occupation theme, and interest theme.

4. The image generation method according to claim 1, characterized in that: The subject information includes at least two categories of subject information, and different categories of subject information have different levels; The method further comprises: Generate different scene descriptions respectively according to the tag information and the different types of topic information by using the language model; Generate multiple frames of AI images corresponding to each type of subject information according to the different scene descriptions through the text-generated graph model; Among them, the number of AI images corresponding to different levels of subject information is different, and the higher the level of the subject type, the fewer the number of AI images corresponding to it.

5. The image generation method according to claim 1, characterized in that: The label information includes at least one of the following label information of the agent: age, occupation, personality, interest, figure, clothing, action, and expression.

6. The image generation method according to claim 1, characterized in that: The image style information includes at least one of the following: Color information, local texture information, material information, brushstroke information, lighting information, and composition information.

7. An image generating device, characterized in that: include: A face extraction model for extracting facial features of at least two characters from a user-specified reference image; The reference image includes at least two characters; The second acquisition module is used to acquire the text description input by the user; A text-generated graph model, used to generate an intelligent agent image according to a text description input by a user and facial features of the at least two characters; when generating the intelligent agent image, the facial features of the at least two characters are merged according to weights respectively specified by the user for different characters; or, the facial features of all characters in the reference image are merged to generate the intelligent agent image; the intelligent agent is a virtual image; A label extraction model, used to obtain label information of the agent from the agent image and / or the agent persona respectively; A first acquisition module is used to acquire subject information; A language model, used to generate a scene description including at least one item of supplementary information by using the tag information and content in the subject information as keywords; The at least one item of supplementary information includes: scene information, composition information, action and expression information of the agent, and clothing and accessories information; The Vincent graph model is also used to generate a frame of artificial intelligence image according to the scene description and the facial features of the agent; the facial features of the agent fuse the facial features of the at least two characters; the facial features of the agent are obtained from the agent image through a facial extraction model or a label extraction model; or, it is also used to generate a frame of artificial intelligence image according to the scene description and image style information through the Vincent graph model; the image style information is obtained from the agent image through a style extraction model, and the style extraction model is a mapping space model. When training the style extraction model, after a reference image is input into the style extraction model, the style extraction model outputs a mapping space of a vector with multiple dimensions based on the reference image to reflect the style information of the reference image; the artificial intelligence image contains the agent.

8. An image generating device, characterized in that: include: A processor and a memory, wherein the memory stores executable code, and when the executable code is processed by the processor, the processor can execute the method as claimed in any one of claims 1 to 6.

9. A computer-readable storage medium having executable codes stored thereon, which, when executed by a processor of an electronic device, causes the electronic device to execute the method according to any one of claims 1 to 6.

10. A computer program product comprising a computer program or computer executable instructions, characterized in that: When the computer program or computer executable instructions are executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images