Collar picture generation method and device, equipment and storage medium

Abstract

The invention discloses a comic generation method and device, equipment and a storage medium, and relates to the technical field of artificial intelligence. The comic picture generation method comprises the following steps: generating a multi-view picture associated with a scene article based on a single-view picture of the scene article in a comic picture; determining a model weight and a model cue word associated with the role image based on the role image in the comic picture; based on the model cue word, the number of the sub-graphs of the comic picture and the story description text, determining segmented text cue words associated with each sub-graph in the comic picture by a natural language processing model; and based on the multi-view picture, the model weight and the segmented text cue word associated with each sub-picture, generating a comic picture by a picture generation model. According to the technical scheme provided by the embodiment of the invention, the consistency of the character image and the article appearance can be kept in the plurality of continuously generated pictures.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a comic strip generation method, device, equipment and storage medium. Background Art

[0002] With the development of artificial intelligence, AI image generation tools have made remarkable progress in creating visual content. The common application scenario of these tools is to generate a single image based on a text description.

[0003] However, AI image generation tools face consistency challenges when generating multiple images in succession, especially in maintaining the consistency of character images and object appearance. The above limitations seriously hinder the application of these AI image generation tools to comic strip generation scenarios that require storyline coherence. Summary of the invention

[0004] The present invention provides a comic strip generation method, device, equipment and storage medium to solve the problem that the existing artificial intelligence picture generation tool cannot maintain the consistency of character image and object appearance when continuously generating multiple pictures.

[0005] According to one aspect of the present invention, there is provided a comic strip generation method, comprising:

[0006] Based on a single-view image of a scene object in the comic strip, generating a multi-view image associated with the scene object;

[0007] Based on the character image in the comic strip, determining a model weight and a model prompt word associated with the character image;

[0008] Based on the model prompt words, the number of sub-pictures in the comic strip, and the story description text, the natural language processing model determines the segmented text prompt words associated with each sub-picture in the comic strip;

[0009] Based on the multi-view images, model weights, and segmented text prompt words associated with each sub-image, a comic strip is generated by the image generation model.

[0010] According to another aspect of the present invention, there is provided a comic strip generating device, comprising:

[0011] A multi-view picture generation module, used for generating multi-view pictures associated with the scene objects based on the single-view pictures of the scene objects in the comic strip;

[0012] A model weight determination module, used to determine the model weight and model prompt words associated with the character image based on the character image in the comic strip;

[0013] A segmented text determination module, for determining the segmented text prompt words associated with each sub-picture in the comic strip by a natural language processing model based on the model prompt words, the number of sub-pictures in the comic strip, and the story description text;

[0014] The comic strip generation module is used to generate a comic strip by using the picture generation model based on the multi-view pictures, model weights and segmented text prompt words associated with each sub-picture.

[0015] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0016] at least one processor; and

[0017] a memory communicatively connected to the at least one processor; wherein,

[0018] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the comic strip generation method described in any embodiment of the present invention.

[0019] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the comic strip generation method described in any embodiment of the present invention when executed.

[0020] According to another aspect of the present invention, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the method for generating a comic strip according to any embodiment of the present disclosure is implemented.

[0021] The technical solution of the embodiment of the present invention generates a multi-view image associated with the scene object based on a single-view image of the scene object in the comic strip, and determines the model weight and model prompt words associated with the character image based on the character image in the comic strip. Furthermore, based on the model prompt words, the number of sub-images in the comic strip and the story description text, a natural language processing model determines the segmented text prompt words associated with each sub-image in the comic strip. Finally, based on the multi-view images, the model weights and the segmented text prompt words associated with each sub-image, a comic strip is generated by a picture generation model, which can ensure the appearance consistency of the scene objects and the character image in multiple consecutively generated pictures.

[0022] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0024] Figure 1 is a flow chart of a comic strip generation method provided according to the first embodiment of the present invention;

[0025] Figure 2a is a flow chart of a comic strip generation method provided according to Embodiment 2 of the present invention;

[0026] Figure 2b This is a flowchart of comic strip generation provided according to Embodiment 2 of the present invention;

[0027] Figure 3 is a structural schematic diagram of a comic strip generating device provided according to Embodiment 3 of the present invention;

[0028] Figure 4 It is a structural schematic diagram of an electronic device for implementing the comic strip generation method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0029] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0030] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0031] Embodiment 1

[0032] Figure 1A flowchart of a comic strip generation method is provided for the first embodiment of the present invention. This embodiment is applicable to the case of generating comic strips based on a natural language processing model. The method can be executed by a comic strip generation device. The comic strip generation device can be implemented in the form of hardware and / or software. The comic strip generation device can be configured in various general-purpose computing devices. Figure 1 As shown, the method includes:

[0033] S110 . Generate multi-view images associated with scene objects based on single-view images of scene objects in the comic strip.

[0034] The scene objects are objects whose shapes do not change in the scene described in the comic strip. For example, the scene objects are the table in the home of character A or the vase on the table in the comic strip.

[0035] A single-view image is a picture of a scene object captured in a single view, for example, a picture of a table captured in a top view. Correspondingly, a multi-view image is a collection of images of the same scene object taken from multiple angles or perspectives.

[0036] In an embodiment of the present invention, based on a single-view image of a scene object in a comic strip provided by a user, a 3D image generation model can be used to generate a multi-view image associated with the scene object. Specifically, the single-view image can be input into the 3D image generation model, and the 3D image generation model generates a 3D model of the scene object based on the depth information in the single-view image. Finally, multi-view images of the 3D model at different viewing angles are collected.

[0037] In a specific example, a front view image of a table in the home of character A in a comic strip is obtained, and the front view image is input into a 3D image generation model to obtain a 3D model of the table output by the 3D image generation model. Based on the 3D model of the table, multiple view images from different perspectives are collected.

[0038] S120: Based on the character image in the comic strip, determine the model weight and model prompt words associated with the character image.

[0039] The character image is an image whose form changes as described in the comic strip. For example, the character image is a person image A or an animal image B described in the comic strip.

[0040] In an embodiment of the present invention, after obtaining the character image in the comic strip input by the user, the model weight associated with the current character image is determined according to the mapping relationship between the character image and the model weight. The model weight can be obtained by fine-tuning the pre-trained picture generation model using multiple pictures associated with the character image, and the model weight is specifically a low-rank adaptation (Lora) weight. Each set of Lora weights corresponds to a model prompt. Exemplarily, the model prompt corresponding to character image A is the Chinese name, English name or other character representation of character image A.

[0041] S130, based on the model prompt words, the number of sub-pictures in the comic strip, and the story description text, the natural language processing model determines the segmented text prompt words associated with each sub-picture in the comic strip.

[0042] In an embodiment of the present invention, after obtaining the model prompt words associated with the character image, the natural language processing model determines the segmented text prompt words associated with each sub-image in the comic strip based on the model prompt words, the number of sub-images of the comic strip input by the user, and the story description text. Specifically, the natural language processing model can first segment the story description text according to the number of sub-images in the comic strip to obtain the story description paragraphs corresponding to each character. Further, for each story description paragraph and model prompt words associated with each sub-image, the segmented text prompt words associated with each sub-image in the comic strip are determined. Among them, the natural language processing model can be a large language model (Large Language Model, LLM for short), which is hereinafter referred to as the "large model".

[0043] In a specific example, the model prompt associated with the character image is WW. The number of sub-pictures in the comic strip is 4, and the story description text is "WW was lying on the bed at home and suddenly felt a stomachache. So he went out to the hospital and saw a big tree on the way, which was full of pink flowers. After arriving at the hospital, he described the symptoms to the doctor wearing glasses and followed the doctor to the examination room for examination." First, according to the number of sub-pictures in the comic strip, the story description text can be divided into 4 parts to obtain the story description paragraphs corresponding to each character. For example, it is divided into "WW was lying on the bed at home and suddenly felt a stomachache", "so he went out to the hospital and saw a big tree on the way, which was full of pink flowers", "after arriving at the hospital, he described the symptoms to the doctor wearing glasses" and "followed the doctor to the examination room for examination". Further, for the story description paragraphs and model prompts associated with each sub-picture, the segmented text prompts associated with each sub-picture in the comic strip are determined. For example, each story description paragraph can be segmented in the end, and the model prompts are added to the multiple words obtained by segmentation to form segmented text prompts.

[0044] S140, generating a comic strip using the picture generation model based on the multi-view pictures, the model weights, and the segmented text prompt words associated with each sub-picture.

[0045] In the embodiment of the present invention, the multi-view image, the model weight, and the segmented text prompt words associated with each sub-image can be input into the image generation model. The image generation model generates each sub-image according to the multi-view image, the model weight, and the segmented text prompt words associated with each sub-image. Finally, the sub-images are arranged in order to obtain a comic strip.

[0046] The technical solution of the embodiment of the present invention generates a multi-view image associated with the scene object based on a single-view image of the scene object in the comic strip, and determines the model weight and model prompt words associated with the character image based on the character image in the comic strip. Furthermore, based on the model prompt words, the number of sub-images in the comic strip and the story description text, a natural language processing model determines the segmented text prompt words associated with each sub-image in the comic strip. Finally, based on the multi-view images, the model weights and the segmented text prompt words associated with each sub-image, a comic strip is generated by a picture generation model, which can ensure the appearance consistency of the scene objects and the character image in multiple consecutively generated pictures.

[0047] Embodiment 2

[0048] Figure 2a A flowchart of a comic strip generation method provided in the second embodiment of the present invention. This embodiment further refines the above embodiments and provides specific steps for generating multi-view images associated with scene objects based on single-view images of scene objects in the comic strip, specific steps for determining model weights and model prompt words associated with character images based on character images in the comic strip, specific steps for determining segmented text prompt words associated with each sub-image in the comic strip by a natural language processing model based on model prompt words, the number of sub-images in the comic strip, and story description text, and specific steps for generating comic strips by a picture generation model based on multi-view images, model weights, and segmented text prompt words associated with each sub-image. Figure 2a As shown, the method includes:

[0049] S210: Input a single-view image of a scene object in the comic strip into a three-dimensional image generation model to obtain a three-dimensional model of the scene object input into the three-dimensional image generation model.

[0050] In the embodiment of the present invention, since scenes with different perspectives often appear in comic strips, multi-view images of scene objects are required. In order to obtain multi-view images of scene objects in the comic strip, single-view images of scene objects in the comic strip are input into a 3D image generation model. The 3D image generation model generates a 3D model of the scene object based on depth information in the single-view image.

[0051] S220: Generate multi-view images associated with scene objects based on the three-dimensional model.

[0052] In the embodiment of the present invention, after generating the 3D model of the scene object, the pictures of the 3D model in different views are collected to obtain pictures of the same scene object in multiple views. In the subsequent processing, the appropriate view pictures can be quickly selected according to the direction and perspective information described in the story description text of the comic book.

[0053] Optionally, before generating the multi-view images associated with the scene objects, the method further includes:

[0054] For each character image in the comic strip, multiple pictures associated with the character image and model prompt words associated with the character image are used as training samples to train the picture generation model, determine the model weight associated with the character image, and the text prompt words associated with the character image;

[0055] The character image, the model weight associated with the character image, and the text prompt words associated with the character image are stored in a character weight list accordingly.

[0056] In this optional embodiment, specific steps are provided before generating a multi-view picture associated with a scene object: first, for each character image in the comic strip, multiple pictures associated with the character image and model prompt words associated with the character image are input into the picture generation model as training samples, the picture generation model is trained, and the model weight associated with the character image and the text prompt words associated with the character image are determined.

[0057] Furthermore, the character image, the model weight associated with the character image, and the text prompt word associated with the character image are stored in a character weight list accordingly, so that the model weight and text prompt word associated with the character image can be quickly determined according to the character image input by the user, thereby improving the efficiency of comic strip generation.

[0058] S230: Based on the character image in the comic strip, determine the model weight and model prompt word associated with the character image in the character weight list.

[0059] The character weight list stores the character image, the model weight associated with the character image, and the model prompt word associated with the character image.

[0060] In the embodiment of the present invention, the specific process of the comic strip generation method is as follows: Figure 2bAs shown, after obtaining the character image in the comic strip input by the user, the character image can be searched in the character weight table, and the model weight and model prompt word associated with the character image can be obtained in the character weight table. By pre-establishing a character weight table, the character image and the model weight and model prompt word corresponding to the character image are associated and stored, and the model weight and model prompt word corresponding to the character image in the comic strip can be quickly located.

[0061] S240, the natural language processing model segments the story description text according to the number of sub-pictures in the comic strip, and obtains the story description paragraphs associated with each sub-picture in the comic strip.

[0062] In an embodiment of the present invention, the number of comic strip sub-pictures, the story description text, and the model prompt words associated with the character image input by the user can be input into the natural language processing model together. The natural language processing model segments the story description text according to the number of comic strip sub-pictures to obtain the story description paragraphs associated with each sub-picture in the comic strip. Exemplarily, if the number of comic strip sub-pictures is 4, the story description text is divided into 4 story description paragraphs. Among them, the segmentation method can be segmentation according to punctuation marks in the story description text, or segmentation according to the number of words.

[0063] S250: Determine the segmented text prompt words associated with each sub-picture in the comic strip based on the story description paragraphs and the model prompt words.

[0064] In the embodiment of the present invention, the story description paragraph is segmented to obtain a word set consisting of multiple words contained in the story description paragraph, and the model prompt words are added to the word set. Finally, the word set with the model prompt words added is used as the segment text prompt words associated with the current subgraph.

[0065] S260, the picture generation model sequentially obtains the segmented text prompt words associated with each sub-picture in the comic strip, and determines the character text prompt words and the object text prompt words in the segmented text prompt words.

[0066] In the embodiment of the present invention, each sub-image contains two parts: an object and a character image. The picture generation model sequentially obtains the segmented text prompt words associated with each sub-image in the comic strip as the current segmented text prompt words. For the current segmented text prompt words, word recognition is performed to determine the character text prompt words and the object text prompt words contained therein. Among them, the character text prompt words are prompt words associated with the character image, and the object text prompt words are prompt words associated with the scene objects.

[0067] S270: Determine object feature data associated with scene objects based on the object text prompt words and the multi-view images.

[0068] In the embodiment of the present invention, the item feature data associated with the scene items are determined based on the item text prompt words and the multi-view pictures. Specifically, the target view picture that matches the current display requirements can be selected from the multi-view pictures based on the item text prompt words. Then, the target view picture is used as a reference picture to determine the item feature data associated with the scene items. The picture generation model uses the multi-view pictures as reference pictures and combines the item text prompt words to generate item feature data, which can ensure the appearance consistency of the scene items in multiple sub-pictures.

[0069] S280: Determine the character feature data associated with the character image based on the model weight and the character text prompt word.

[0070] In the embodiment of the present invention, character feature data associated with the character image is generated based on the model weight and the character text prompt word. The image generation model is pre-trained with multiple images associated with the character image to obtain the model weight matching the character image, and finally the character feature data is determined based on the model weight and the character text prompt word, which can ensure the consistency of the task image in multiple sub-images.

[0071] S290, based on the object feature data and the character feature data, generate sub-graphs in the comic strip, and return to execute the operation of sequentially obtaining the segmented text prompt words associated with each sub-graph in the comic strip, until the traversal of each sub-graph in the comic strip is completed and the comic strip is composed of the sub-graphs.

[0072] In the embodiment of the present invention, the image generation model generates a sub-graph associated with the current segmented text prompt word based on the object feature data and the character feature data, and returns to execute the operation of sequentially obtaining the segmented text prompt word associated with each sub-graph in the comic strip until the traversal of each sub-graph in the comic strip is completed. Finally, the comic strip is formed by sequentially arranging the sub-graphs to ensure the appearance consistency of the scene objects and character images in each sub-graph.

[0073] The technical solution of the embodiment of the present invention generates a multi-view picture associated with the scene object based on a single-view picture of the scene object in the comic strip, and determines the model weight and model prompt words associated with the character image based on the character image in the comic strip. Based on the model prompt words, the number of sub-pictures in the comic strip and the story description text, a natural language processing model determines the segmented text prompt words associated with each sub-picture in the comic strip. Finally, based on the multi-view pictures, the model weights and the segmented text prompt words associated with each sub-picture, a comic strip is generated by a picture generation model, which can ensure the appearance consistency of the scene objects and the character images in multiple consecutively generated pictures.

[0074] Embodiment 3

[0075] Figure 3 This is a schematic diagram of the structure of a comic strip generation device provided in Embodiment 3 of the present invention. Figure 3 As shown, the device comprises:

[0076] A multi-view picture generation module 310, for generating a multi-view picture associated with a scene object in a comic strip based on a single-view picture of the scene object;

[0077] A model weight determination module 320, for determining a model weight and a model prompt word associated with the character image based on the character image in the comic strip;

[0078] The segmented text determination module 330 is used to determine the segmented text prompt word associated with each sub-picture in the comic strip by a natural language processing model based on the model prompt word, the number of sub-pictures in the comic strip, and the story description text;

[0079] The comic strip generation module 340 is used to generate a comic strip by using the picture generation model based on the multi-view pictures, model weights and segmented text prompt words associated with each sub-picture.

[0080] The technical solution of the embodiment of the present invention generates a multi-view image associated with the scene object based on a single-view image of the scene object in the comic strip, and determines the model weight and model prompt words associated with the character image based on the character image in the comic strip. Furthermore, based on the model prompt words, the number of sub-images in the comic strip and the story description text, a natural language processing model determines the segmented text prompt words associated with each sub-image in the comic strip. Finally, based on the multi-view images, the model weights and the segmented text prompt words associated with each sub-image, a comic strip is generated by a picture generation model, which can ensure the appearance consistency of the scene objects and the character image in multiple consecutively generated pictures.

[0081] Optionally, the segment text determination module 330 is specifically configured to:

[0082] The natural language processing model segments the story description text according to the number of sub-pictures in the comic strip to obtain the story description paragraphs associated with each sub-picture in the comic strip;

[0083] Based on the story description paragraphs and the model prompt words, the segmented text prompt words associated with each sub-picture in the comic strip are determined.

[0084] Optionally, the model weight determination module 320 is specifically configured to:

[0085] Based on the character image in the comic strip, determining the model weight and model prompt word associated with the character image in the character weight list;

[0086] The character weight list correspondingly stores the character image, the model weight associated with the character image, and the model prompt word associated with the character image.

[0087] Optionally, the comic strip generation module 340 is specifically used for:

[0088] The picture generation model sequentially obtains the segmented text prompt words associated with each sub-picture in the comic strip, and determines the character text prompt words and the object text prompt words in the segmented text prompt words;

[0089] Determining object feature data associated with scene objects based on the object text prompt word and the multi-view image;

[0090] Determining character feature data associated with the character image based on the model weight and the character text prompt word;

[0091] Based on the item feature data and the character feature data, a sub-image in the comic strip is generated, and the operation of sequentially obtaining the segmented text prompt words associated with each sub-image in the comic strip is returned to execute until each sub-image in the comic strip is traversed and the comic strip is formed by the sub-images.

[0092] Optionally, the comic strip generating device further includes:

[0093] A model training module is used for, before generating a multi-view picture associated with the scene object, for each character image in the comic strip, using a plurality of pictures associated with the character image and a model prompt word associated with the character image as training samples, training a picture generation model, and determining a model weight associated with the character image and a text prompt word associated with the character image;

[0094] The role weight list generating module is used to store the role image, the model weight associated with the role image and the text prompt word associated with the role image in the role weight list in a corresponding manner.

[0095] Optionally, the multi-view image generation module 310 is specifically configured to:

[0096] Inputting a single-view image of a scene object in the comic strip into a three-dimensional image generation model to obtain a three-dimensional model of the scene object input by the three-dimensional image generation model;

[0097] Based on the three-dimensional model, a multi-view picture associated with the scene object is generated.

[0098] The comic strip generating device provided in the embodiment of the present invention can execute the comic strip generating method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0099] In the technical solution of the present invention, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0100] Embodiment 4

[0101] According to an embodiment of the present invention, the present invention also provides an electronic device, a readable storage medium and a computer program product.

[0102] Figure 4 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, applicators, blade applicators, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0103] like Figure 4 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0104] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0105] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a comic strip generation method.

[0106] In some embodiments, the comic strip generation method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the comic strip generation method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute the comic strip generation method in any other appropriate manner (e.g., by means of firmware).

[0107] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0108] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or application.

[0109] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, device, or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0110] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0111] The systems and techniques described herein may be implemented in a computing system that includes a backend component (e.g., as a data applier), or a computing system that includes a middleware component (e.g., an application applier), or a computing system that includes a frontend component (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0112] A computing system may include a client and an application server. The client and the application server are generally remote from each other and usually interact through a communication network. The relationship between the client and the application server is generated by computer programs running on corresponding computers and having a client-application server relationship with each other. The application server may be a cloud application server, also known as a cloud computing application server or a cloud host, which is a host product in a cloud computing application system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS applications.

[0113] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0114] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A comic strip generation method, characterized in that: include: Based on a single-view image of a scene object in the comic strip, generating a multi-view image associated with the scene object; Based on the character image in the comic strip, determining the model weight and model prompt words associated with the character image; Based on the model prompt words, the number of sub-pictures in the comic strip, and the story description text, the natural language processing model determines the segmented text prompt words associated with each sub-picture in the comic strip; Based on the multi-view images, model weights, and segmented text prompt words associated with each sub-image, a comic strip is generated by the image generation model.

2. The method according to claim 1, characterized in that Based on the model prompt words, the number of sub-pictures in the comic strip, and the story description text, the natural language processing model determines the segmented text prompt words associated with each sub-picture in the comic strip, including: The natural language processing model segments the story description text according to the number of sub-pictures in the comic strip to obtain the story description paragraphs associated with each sub-picture in the comic strip; Based on the story description paragraphs and the model prompt words, the segmented text prompt words associated with each sub-picture in the comic strip are determined.

3. The method according to claim 1, characterized in that Based on the character image in the comic strip, a model weight and a model prompt word associated with the character image are determined, including: Based on the character image in the comic strip, determining the model weight and model prompt word associated with the character image in the character weight list; The character weight list correspondingly stores the character image, the model weight associated with the character image, and the model prompt word associated with the character image.

4. The method according to claim 1, characterized in that: Based on the multi-view images, the model weights, and the segmented text prompt words associated with each sub-image, the image generation model generates a comic strip, including: The picture generation model sequentially obtains the segmented text prompt words associated with each sub-picture in the comic strip, and determines the character text prompt words and the object text prompt words in the segmented text prompt words; Determining object feature data associated with scene objects based on the object text prompt word and the multi-view image; Determining character feature data associated with the character image based on the model weight and the character text prompt word; Based on the item feature data and the character feature data, a sub-image in the comic strip is generated, and the operation of sequentially obtaining the segmented text prompt words associated with each sub-image in the comic strip is returned to execute until each sub-image in the comic strip is traversed and the comic strip is formed by the sub-images.

5. The method according to claim 3, characterized in that: Before generating the multi-view pictures associated with the scene objects, the method further includes: For each character image in the comic strip, multiple pictures associated with the character image and model prompt words associated with the character image are used as training samples to train the picture generation model, and determine the model weight associated with the character image and the text prompt words associated with the character image; The character image, the model weight associated with the character image, and the text prompt word associated with the character image are stored in the character weight list in a corresponding manner.

6. The method according to claim 1, characterized in that Based on a single-view image of a scene object in a comic strip, a multi-view image associated with the scene object is generated, comprising: Inputting a single-view image of a scene object in the comic strip into a three-dimensional image generation model to obtain a three-dimensional model of the scene object input by the three-dimensional image generation model; Based on the three-dimensional model, a multi-view picture associated with the scene object is generated.

7. A comic strip generating device, characterized in that: include: A multi-view picture generation module, used for generating multi-view pictures associated with the scene objects based on the single-view pictures of the scene objects in the comic strip; A model weight determination module, used to determine the model weight and model prompt words associated with the character image based on the character image in the comic strip; A segmented text determination module, for determining the segmented text prompt words associated with each sub-picture in the comic strip by a natural language processing model based on the model prompt words, the number of sub-pictures in the comic strip, and the story description text; The comic strip generation module is used to generate a comic strip by using the picture generation model based on the multi-view pictures, model weights and segmented text prompt words associated with each sub-picture.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the comic strip generation method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the comic strip generation method according to any one of claims 1 to 6 when executed.

10. A computer program product, characterized in that The computer program product comprises a computer program, which, when executed by a processor, implements the comic strip generation method according to any one of claims 1 to 6.

Images