Image generation method, apparatus, device, medium, and computer program

Abstract

This disclosure provides an image generation method, apparatus, device, storage medium, and program product. The method includes determining prompt text and image style information, generating an initial image having the image style based on semantic information of the prompt text, and performing a transparency process on non-target regions of the initial image to obtain a target image having transparency channel information.

Description

Technical Field

[0001] [Cross - Reference to Related Applications] This application claims the priority of a Chinese patent application with the application number 202310921622.7 and the invention title "Image Generation Method, Apparatus, Device, Medium and Program Product", which was filed with the Chinese Patent Office on July 25, 2023, and all of its contents are incorporated herein by reference.

[0002] [Technical Field] The present disclosure relates to the field of computer technology, and in particular to an image generation method, apparatus, device, medium and program product.

Background Art

[0003] In some application programs, users always use sticker materials to process image data to enrich the expression forms of image data such as pictures and videos. Current sticker images need to be created manually, with high creation costs, long cycles, and it is not easy to meet aesthetic standards, and users cannot create the sticker images they expect.

Summary of the Invention

Problems to be Solved by the Invention

[0004] The present disclosure proposes an image generation method, apparatus, device, storage medium and program product to solve, to a certain extent, the technical problems of poor creation efficiency and unsatisfactory effects of sticker images.

Means for Solving the Problems

[0005] According to a first aspect of the present disclosure, an image generation method is provided, and this method includes: determining prompt text and image style information; generating an initial image having the image style based on the semantic information of the prompt text; This includes performing a transparency process on the non-target region of the initial image to obtain a target image having transparency channel information.

[0006] According to a second aspect of this disclosure, an image generating apparatus is provided, which apparatus An information determination module for determining prompt text and image style information, An image generation module for generating an initial image having the image style based on the semantic information of the prompt text, The system includes a transparency processing module for performing transparency processing on non-target regions of the initial image to obtain a target image having transparency channel information.

[0007] A third aspect of the present disclosure provides an electronic device comprising one or more processors, memory, and one or more programs, wherein the one or more programs are stored in the memory and executed by the one or more processors, and the programs include instructions for performing the method of the first or second aspect.

[0008] A fourth aspect of the present disclosure provides a non-volatile computer-readable storage medium containing a computer program, wherein, when the computer program is executed by one or more processors, the processors are instructed to perform the method according to the first or second aspect.

[0009] According to a fifth aspect of this disclosure, a computer program product is provided, which includes computer program instructions, and which, when executed on a computer, causes the computer to perform the method described in the first aspect. [Effects of the Invention]

[0010] As can be seen from the above, the image generation method, apparatus, device, medium, and program product provided in this disclosure generates an initial image from the semantic information of the prompt text and the image style information, then performs a transparency process to obtain a target image having transparency channel information. This enables the automatic generation of sticker images without manual creation, according to the user's prompt text and expected image style, thereby improving the effectiveness and efficiency of sticker image creation. To more clearly illustrate the technical concepts in this disclosure or related technologies, the following briefly introduces the drawings that may be used in the descriptions of the embodiments or related technologies. Obviously, the drawings in the following descriptions are merely embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without expending any creative effort. [Brief explanation of the drawing]

[0011] [Figure 1] This is a schematic diagram of the image generation architecture of an embodiment of the present disclosure. [Figure 2] This is a schematic diagram of the hardware structure of an exemplary electronic device of an embodiment of the present disclosure. [Figure 3] This is a schematic flowchart of the image generation method of the embodiment of this disclosure. [Figure 4] This is a schematic diagram of the target image of an embodiment of the present disclosure. [Figure 5] This is a schematic diagram of an image generation apparatus according to the embodiments of the present disclosure. [Modes for carrying out the invention]

[0012] To make the purpose, technical proposal, and advantages of this disclosure clearer, the following will be a more detailed explanation of this disclosure, with reference to the drawings and accompanied by specific examples.

[0013] It should be noted that, unless otherwise defined, technical or scientific terms used in the embodiments of this disclosure should have the general meaning understood by a person of general skill in the art to which this disclosure belongs. The terms “first,” “second,” and similar terms used in the embodiments of this disclosure do not indicate any order, number, or importance, but are used solely to distinguish different components. Similar terms such as “include” or “incorporate” mean that the element or object appearing before the term covers the enumerated element or object and its equivalents appearing after the term, and do not exclude other elements or objects. Similar terms such as “connected” or “linked” are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. Terms such as “up,” “down,” “left,” and “right” are used solely to describe relative positional relationships, and these relative positional relationships may change accordingly after the absolute position of the described object changes.

[0014] To ensure understanding, before using any of the technical proposals disclosed in each embodiment of this disclosure, the user should be informed in an appropriate manner in accordance with applicable laws and regulations about the type of personal information related to this disclosure, the scope of use, and the circumstances under which it may be used, and the user's permission should be obtained.

[0015] For example, when responding to a user's voluntary request, an operation that sends prompt information to the user to clearly prompt and request the user to perform an action requires acquiring and using the user's personal information. This allows the user to autonomously choose whether or not to provide personal information to software or hardware such as electronic devices, application programs, servers, or storage media that perform the operation of the proposed technical method of this disclosure in response to the prompt information.

[0016] As a selective but non-limiting implementation method, the method of sending prompt information to the user in response to receiving the user's spontaneous request may be, for example, in the form of a pop-up window, and the prompt information may be presented in text form in the pop-up window. In addition, a selection control for the user to select "agree" or "disagree" to provide personal information to the electronic device may be placed on the pop-up window.

[0017] As can be understood, the above notification and user permission acquisition process are only schematic and do not constitute a limitation to the implementation method of the present disclosure. Other methods that meet the relevant laws and regulations may also be used in the implementation method of the present disclosure.

[0018] FIG. 1 shows a schematic diagram of an image generation architecture according to an embodiment of the present disclosure. Referring to FIG. 1, this image generation architecture 100 may include a server 110, a terminal 120, and a network 130 that provides a communication link. The server 110 and the terminal 120 can be connected via a wired or wireless network 130. Here, the server 110 may be an independent physical server, or may be a server cluster or a distributed system composed of multiple physical servers, or may be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, security services, and CDN.

[0019] The terminal 120 may be implemented by hardware or software. For example, when the terminal 120 is implemented by hardware, it may be various electronic devices having a display screen and supporting page display, including but not limited to smartphones, tablet computers, e-book readers, laptop computers, and desktop computers. When the terminal 120 device is implemented by software, it may be installed on the electronic devices listed above, and it may be implemented as multiple software or software modules (for example, software or software modules for providing distributed services), or it may be implemented as a single software or software module, without specific limitation thereto.

[0020] It should be noted that the image generation method provided in the embodiments of the present application may be executed by the terminal 120 or may be executed by the server 110. It should be understood that the numbers of the terminal, network, and server in FIG. 1 are merely schematic and are not intended to limit them. Depending on the needs of implementation, any number of terminals, networks, and servers may be provided.

[0021] FIG. 2 shows a schematic hardware structure diagram of an exemplary electronic device 200 provided in an embodiment of the present disclosure. As shown in FIG. 2, the electronic device 200 may include a processor 202, a memory 204, a network module 206, a peripheral interface 208, and a bus 210. Here, the processor 202, the memory 204, the network module 206, and the peripheral interface 208 realize mutual communication connections inside the electronic device 200 via the bus 210.

[0022] The processor 202 may be a central processing unit (CPU), an image generator, a neural network processor (NPU), a microcontroller (MCU), a programmable logic device, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), or one or more integrated circuits. The processor 202 may be used to perform functions related to the technology described herein. In some embodiments, the processor 202 may further include multiple processors integrated into a single logic assembly. For example, as shown in Figure 2, the processor 202 may include multiple processors 202a, 202b, and 202c.

[0023] Memory 204 may be configured to store data (e.g., instructions, computer code, etc.). As shown in Figure 2, the data stored in memory 204 may include program instructions (e.g., program instructions for implementing the image generation method of the embodiment of this disclosure) and data that needs to be processed (e.g., the memory may store configuration files for other modules, etc.). The processor 202 can access the program instructions and data stored in memory 204 and can also manipulate the data that needs to be processed by executing the program instructions. Memory 204 may include volatile or non-volatile storage devices. In some embodiments, memory 204 may include random access memory (RAM), read-only memory (ROM), optical disks, magnetic disks, hard disks, solid-state drives (SSDs), flash memory, memory sticks, etc.

[0024] The network module 206 may be configured to provide the electronic device 200 with communication with other external devices via a network. This network may be any wired or wireless network capable of transmitting and receiving data. For example, this network may be a wired network, a local wireless network (e.g., Bluetooth®, Wi-Fi, Near Field Communication (NFC), etc.), a cellular network, the Internet, or a combination of the above. To make it clear, the type of network is not limited to the specific examples above. In some embodiments, the network module 306 may include any number of network interface controllers (NICs), radio frequency modules, transceivers, modems, routers, gateways, adapters, cellular network chips, and any combination thereof.

[0025] The peripheral interface 208 may be configured to connect the electronic device 200 to one or more peripheral devices to enable information input and output. For example, the peripheral devices may include input devices such as keyboards, mice, touchpads, touchscreens, microphones, and various sensors, and output devices such as displays, speakers, vibrators, and indicator lights.

[0026] Bus 210 may be configured to transmit information between each assembly of the electronic device 200 (e.g., processor 202, memory 204, network module 206, and peripheral interface 208), such as an internal bus (e.g., processor-memory bus) or an external bus (USB port, PCI-E bus).

[0027] It should be noted that the architecture of the electronic device 200 described above shows only the processor 202, memory 204, network module 206, peripheral interface 208, and bus 210, but in a specific implementation, the architecture of this electronic device 200 may further include other assemblies necessary to achieve normal operation. It should also be understood by those skilled in the art that the architecture of the electronic device 200 includes only the assemblies necessary to implement the embodiment of this disclosure and does not necessarily include all assemblies shown in the figures.

[0028] In some application programs, users constantly use sticker materials to process image data and enrich the representation of image data such as pictures and videos. For example, sticker materials can be directly applied to pictures or videos. Currently, sticker images generally rely on designers to create them one by one, which is costly, time-consuming, and requires designers to possess a certain level of artistic skill to create sticker images that meet aesthetic standards. On the other hand, ordinary users often lack these abilities of professional designers and are unable to create the sticker images they expect. Therefore, how to provide efficient and effective sticker image creation has become an urgent technical issue that needs to be resolved.

[0029] In view of this, embodiments of the present disclosure provide an image generation method, apparatus, device, storage medium, and program product. An initial image is generated from the semantic information of the prompt text and the image style information, and then a transparency process is performed to obtain a target image having transparency channel information. Automatic generation of sticker images can be achieved without manual creation according to the user's prompt text and expected image style, thereby improving the effectiveness and efficiency of sticker image creation.

[0030] Referring to Figure 3, Figure 3 shows a schematic flowchart of an image generation method according to an embodiment of the present disclosure. The image generation method according to an embodiment of the present disclosure may be deployed to a client. In Figure 3, the image generation method 300 may further include the following steps.

[0031] In step S310, the prompt text and image style information are determined.

[0032] Here, the prompt text is used to indicate the content of the image and contains semantic information. Specifically, the prompt text may include an object, and may further include the object's behavior, such as "A boy with a bright smile." The image style may refer to the style of the image's representation, such as an oil painting style, a pixelated style, a childlike style, a stick-drawing style, etc. It should be understood that the above image style attributes are merely examples and not intended to limit them; the image style attributes may include many more styles and are not limited thereto.

[0033] Specifically, the user may determine prompt text and image style information based on the interaction interface. For example, the user may input prompt text based on the input assembly in the interaction interface. The interaction interface may present different image styles, including at least one pre-configured image style, such as image style A, image style B, image style C, etc. Here, one image style may be set as the default value, and when the user has not selected an image style, an image may be generated based on the image style corresponding to this default value. The user may also determine the image style to generate an image based on an image style selection operation (e.g., a click) in the interaction interface.

[0034] In step S320, an initial image having the image style is generated based on the semantic information of the prompt text.

[0035] Here, the initial image has semantic information corresponding to the prompt text. For example, if the prompt text is "A boy with a bright smile", the content of the initial image may include a boy with a bright smile. The initial image may also have the image style determined in step S310. For example, if the image style attribute determined in step S310 is an oil painting style, the boy with a bright smile in the initial image may be presented in an oil painting style, and if the image style determined in step S310 is a pixelated style, the boy with a bright smile in the initial image may be presented in a pixelated style.

[0036] In some embodiments, generating an initial image having the image style based on the semantic information of the prompt text is: The process involves performing feature extraction on the aforementioned prompt text to obtain text features, Based on the aforementioned semantic information, determine image features that have a semantic relationship corresponding to the text features, Selecting the corresponding image style attribute based on the aforementioned image style information, This includes obtaining an initial image having the image style based on the image style attributes and the image features.

[0037] Specifically, feature extraction may be performed on the prompt text text to obtain a text feature F_text. Since the prompt text has semantic information, the text feature F_text may be used to determine the corresponding image feature F_image based on this semantic information. For example, matching may be performed on the image feature set based on the text feature F_text, and the image feature with the highest semantic similarity to the text feature F_text may be determined as the image feature F_image that has a corresponding semantic relationship with the text feature F_text. Here, the image feature set may be a pre-configured database, or it may be obtained by performing feature extraction based on image data. The image feature set may include at least one image feature that has semantic information. Then, the corresponding image style attribute type is determined based on the determined image style information. Based on this image style attribute type and the image feature F_image, the corresponding initial image image_s_type having this image style attribute type may be generated. For example, feature extraction may be performed on the prompt text based on an image generation model to obtain a text feature, and then the corresponding image feature with a semantic relationship may be determined based on this, and combined with the image style attribute corresponding to the image style information to generate the initial image image_s_type having this image style attribute.

[0038] In some embodiments, generating an initial image having the image style based on the semantic information of the prompt text is: The process includes inputting the prompt text and the image style information into an image generation model to obtain the initial image, where the image generation model is obtained based on the fusion of a text image network and a style attribute network.

[0039] Here, the image generation model may generate an initial image having this image style based on the prompt text and the determined image style information. The image generation model may also be obtained by fusing a text image network and a style attribute network, where the text image network may generate a corresponding image (or image feature) based on the prompt text, and the style attribute network may obtain an initial image having this image style based on the image (or image feature) generated by the text image network and the determined image style.

[0040] In some embodiments, the image generation model is obtained based on the fusion of a text image network and a style attribute network. The process involves training a neural network based on a first training sample to obtain a first weight matrix for the text-image network, and training a neural network based on a second training sample to obtain a second weight matrix for the style attribute network. This includes obtaining the weight matrix of the image generation model based on the product of the first weight matrix and the second weight matrix.

[0041] Here, an initial neural network may be trained based on the first training sample, for example, by training an initial diffusion model, to obtain a trained text-image network, where the weights of each layer of this trained text-image network form the first weight matrix Q1. An initial neural network may be trained based on the second training sample, for example, by training an initial diffusion model, to obtain a trained style attribute network, where the weights of each layer of this trained style attribute network form the second weight matrix Q2. The weight matrices of the text-image network and the style attribute network are multiplied, and the resulting product Q1*Q2 is used as the weight matrix of the image generation model to fuse the text-image network and the style attribute network.

[0042] In some embodiments, the first training sample includes a text-image pair, and the method is Feature extraction is performed based on the text samples in the aforementioned text image pair to obtain training text features, The process involves performing noise reduction on a random noise image, cross-attention calculation of the results of the noise reduction process and the training text features to obtain a first noise-reduced image. The first loss function is calculated based on the image samples in the text image pair and the first denoising image, The method further includes obtaining the text image network by adjusting the weights of the neural network to minimize the first loss function based on the first loss function.

[0043] Here, an image that conforms to aesthetic criteria may be used as the training image in the first training sample. In this case, the text-image network obtained by training based on this first training sample can output a high-quality image that conforms to aesthetic criteria, thereby enhancing the image generation effect. In this way, it does not require the user to have any degree of artistic ability, and any user can generate a high-quality image based on the prompt text. Specifically, the first training sample may include a text-image pair, which may further include a training text Text_training and a training image Image_training. Feature extraction is performed on the training text Text_training to obtain the training text features F_Text_training. Denoising is performed at least once on the random noise image image_noisy (for example, denoising with a predetermined number of noises M1), and the result of each denoising process and the training text features F_Text_training are cross-attention calculated to obtain the first denoised image. Specifically, the denoising process is guided by employing the semantic characteristics of the training text features, constantly bringing the semantic characteristics of the denoised image closer to the training text features, and predicting a first denoised image relative to the semantic relationships of the training text features. The first denoised image predicted by the text-image network is compared with the training image in the first training sample, and the cross-entropy loss function of both is calculated. The sum of the cross-entropy loss functions of all first training samples may be used as the first loss function. During the training process, the weight parameters of the text-image network are adjusted to minimize this first loss function.

[0044] In some embodiments, the second training sample includes an image having an image style, and the method is Based on the second training sample mentioned above, noise is added to obtain a noisy image. The noise image is subjected to noise reduction processing to obtain a second noise-reduced image. The process involves calculating a second loss function based on the second denoised image and the second training sample, The method further includes adjusting the weights of the neural network to minimize the second loss function based on the second loss function to obtain the style attribute network.

[0045] Here, image data with image style attributes can be used as a second training sample. Then, the style attribute network, trained on this second training sample, can output an image with the specified image style attributes. In this way, users can create images that meet their expectations based on the determined image style attributes, satisfying their individual needs for image style and enhancing the user experience. Combined with a text-image network, users can generate high-quality images of any style based on the image generation model simply by determining image style attributes and entering prompt text, improving the effectiveness and efficiency of image creation. Specifically, the second training sample may include a style image Image_training_type with the image style attribute type. The style image Image_training_type may be subjected to noise addition processing at least once (the noise, such as Gaussian noise, may be increased based on a predetermined number of noises M2) to obtain a noise image Image_training_type_noisy. Then, the noise image may be subjected to noise reduction processing at least once (denoising may be performed based on a predetermined number of noises M3) to obtain a second denoised image Image_training_type_denoisy. The second denoised image, Image_training_type_denoisy, may be compared with the style image Image_training_type in the second training sample, and the cross-entropy loss of both may be calculated. The sum of the cross-entropy loss functions of all second training samples is taken as the second loss function. During the training process, the weight parameters of the style attribute network are adjusted to minimize this second loss function.

[0046] In step S330, a transparency process is performed on the non-target region of the initial image to obtain a target image having transparency channel information.

[0047] Here, the distinguishing feature of a sticker image from a non-sticker image is the presence of a transparency channel (i.e., an alpha channel). In order for the sticker image to ultimately display only the main content, the generated initial picture needs to be partially transparent except for the main content.

[0048] Specifically, referring to Figure 4, which shows a schematic diagram of the target image of this application. In Figure 4, the user may input prompt text text="A girl dancing with joy." based on the interaction interface, and it can be seen that if this prompt text text includes the object girl and the action dancing, an initial image with the content "A girl dancing with joy" can be generated. If the user selects different image style attributes, at least one target image corresponding to the image style attribute can be obtained, for example, image style A, image style B, and image style C in Figure 4. Furthermore, by performing a transparency process on this initial image and setting the area other than the object girl as a transparent area, a target image can be obtained, and this sticker image may be used on any image I, in which case the area of ​​object B may be displayed on image I.

[0049] In some embodiments, a transparency process is performed on the non-target region of the initial image to obtain a target image having transparency channel information. Based on the initial image, target identification is performed, and the target region of the initial image is obtained. This includes setting the target region to a non-transparent region and the non-target region to a transparent region, thereby obtaining a target image having transparent channel information.

[0050] Here, the ability to perform transparency processing on non-target regions may be obtained by simultaneously learning color channel information and transparency channel information in image data having transparency channel information based on an image processing model. Specifically, the initial neural network may be trained based on image data Image_alpha having transparency channel information. For example, the transparency channel information may be removed from a first image data Image_RGB_alpha having transparency channel information and color channel information to obtain a second image data Image_RGB containing only color channel information. This second image data Image_RGB may be used as input layer data, and the first image data Image_RGB_alpha as output layer data. The initial neural network may then be trained to obtain a trained image processing model. When the initial image is input to this image processing model, a target image having transparency channel information may be obtained. That is, the transparency channel information of the target region including the main body is a first numerical value (e.g., 1, i.e., non-transparent region), and the transparency channel information of the non-target region is a second numerical value (e.g., 0, i.e., transparent region).

[0051] In some embodiments, a transparency process is performed on the non-target region of the initial image to obtain a target image having transparency channel information. This method includes obtaining a target image having transparency channel information by setting the transparency channel information of a first pixel in the initial image whose color channel information is equal to or greater than the channel threshold to a first numerical value, and setting the transparency channel information of a second pixel whose color channel information is less than the channel threshold to a second numerical value.

[0052] Here, each pixel in the initial image having the determined image style attributes has color channel information (e.g., RGB channel information), and for example, the color channel information of pixel X contains the RGB channel values ​​abc (where a, b, and c are all [0, 255]). Based on this pixel distribution, transparency processing may be performed on the non-target regions of the initial image. Specifically, since the main body in the initial image is often prominent, pixels with values ​​higher than the channel threshold in the color channel information may be designated as the target region, and other pixels as the non-target region, thereby dividing the target region containing the main body and obtaining a target mask diagram. For pixels in this target mask diagram, their transparency channel information is set to a first value (e.g., 1), i.e., they are non-transparent regions, and for pixels in regions other than the target mask diagram, their transparency channel information is set to a second output (e.g., 0), i.e., they are transparent regions.

[0053] It should be noted that the methods of the embodiments of this disclosure may be performed by a single device, such as a single computer or server. The methods of the embodiments may be used in a distributed setting and may be completed by the interaction of multiple devices. In such a distributed setting, one of these multiple devices may perform only one or more steps of the methods of the embodiments of this disclosure, and these multiple devices may interact with each other to complete the method.

[0054] It should be noted that the above describes some embodiments of this disclosure. Other embodiments are within the scope of the appended claims. In some cases, the operations or steps described in the claims may be performed in a different order than those in the embodiments above, and the desired results may still be achieved. Furthermore, the processes depicted in the drawings do not necessarily require a specific order or sequence shown to achieve the desired results. In some embodiments, multitasking and parallel processing may also be possible or advantageous.

[0055] Based on the same technical concept and corresponding to the method of any of the above embodiments, the present disclosure further provides an image generating apparatus, referring to Figure 5, the image generating apparatus is An information determination module for determining prompt text and image style information, An image generation module for generating an initial image having the image style based on the semantic information of the prompt text, The system includes a transparency processing module for performing transparency processing on non-target regions of the initial image to obtain a target image having transparency channel information.

[0056] For the sake of convenience in description, the above device will be described by dividing it into various modules according to its function. Of course, when implementing this disclosure, the functions of each module can be realized in the same or multiple software and / or hardware.

[0057] The apparatus of the above embodiment is used to realize the corresponding image generation method in any one of the embodiments described above, and has the beneficial effects of the embodiment of the corresponding method, which will not be described further here.

[0058] Based on the same technical concept and corresponding to the method of any of the above embodiments, the present disclosure further provides a non-temporary computer-readable storage medium that stores computer instructions, and the computer instructions are used to cause the computer to execute the image generation method described in any one of the above embodiments.

[0059] The computer-readable media of this embodiment include persistent and non-persistent, removable and non-removable media, and information storage can be realized by any method or technique. The information may be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random-access memory (SRAM), dynamic random-access memory (DRAM), other types of random-access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, read-only disk memory (CD-ROM), digital multifunction disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices, or any other non-transmission media that may be used to store information accessible by computing equipment.

[0060] The computer instructions stored in the storage medium of the above embodiment are used to cause the computer to execute the image generation method described in any one of the above embodiments, and have the beneficial effects of the embodiment of the corresponding method, which will not be explained further here.

[0061] Those skilled in the art should understand that the above consideration of arbitrary embodiments is illustrative and not intended to imply that the scope of the Disclosure (including the claims) is limited to these examples. Technical features in the above embodiments or different embodiments may be combined in any way in the spirit of the Disclosure, the steps may be performed in any order, and many other variations of different aspects of the embodiments of the Disclosure described above exist, which are not provided in detail for the sake of simplicity.

[0062] Furthermore, in order to simplify the explanation and discussion and to avoid making the embodiments of this disclosure difficult to understand, the provided drawings may or may not show known power / ground connections between integrated circuit (IC) chips and other components. Also, in order to avoid making the embodiments of this disclosure difficult to understand, the apparatus may be shown in block diagram form, taking into consideration the following fact: the details of the embodiments of the apparatus in these block diagrams depend largely on the platform on which the embodiments of this disclosure are implemented (i.e., these details should be entirely within the understanding of those skilled in the art). Where exemplary embodiments of this disclosure are described by specifying details (e.g., circuits), the embodiments of this disclosure may be implemented in the absence of these specific details or with modifications to these specific details, as would be obvious to those skilled in the art. Therefore, these descriptions should be considered explanatory, not restrictive.

[0063] Although this disclosure has already been described with specific embodiments linked thereto, many substitutions, modifications, and variations of these embodiments will be apparent to those skilled in the art as described above. For example, other memory architectures (e.g., dynamic RAM (DRAM)) can be used with the embodiments considered.

[0064] The embodiments of this disclosure are intended to include all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made in the spirit and principles of the embodiments of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. An image generation method, Determining prompt text and image style information, To generate an initial image having the image style based on the semantic information of the prompt text, An image generation method comprising performing a transparency process on the non-target region of the initial image to obtain a target image having transparency channel information.

2. Generating an initial image having the image style based on the semantic information of the prompt text is: The process involves performing feature extraction on the aforementioned prompt text to obtain text features, Based on the aforementioned semantic information, determine image features that have a semantic relationship corresponding to the text features, Selecting the corresponding image style attribute based on the aforementioned image style information, The method of claim 1, comprising obtaining an initial image having the image style based on the image style attributes and the image features.

3. Generating an initial image having the image style based on the semantic information of the prompt text is: The method of claim 1, comprising inputting the prompt text and the image style information into an image generation model to obtain the initial image, wherein the image generation model is obtained based on the fusion of a text image network and a style attribute network.

4. The aforementioned image generation model is obtained based on the fusion of a text image network and a style attribute network. The process involves training a neural network based on a first training sample to obtain a first weight matrix for the text-image network, and training a neural network based on a second training sample to obtain a second weight matrix for the style attribute network. The method of claim 3, further comprising obtaining the weight matrix of the image generation model based on the product of the first weight matrix and the second weight matrix.

5. The first training sample includes a text-image pair, and the method is Feature extraction is performed based on the text samples in the aforementioned text image pair to obtain training text features, The process involves performing noise reduction on a random noise image, cross-attention calculation of the results of the noise reduction process and the training text features to obtain a first noise-reduced image. The first loss function is calculated based on the image samples in the text image pair and the first denoising image, The method of claim 4, further comprising adjusting the weights of the neural network to minimize the first loss function based on the first loss function to obtain the text image network.

6. The second training sample includes an image having an image style, and the method is Based on the second training sample mentioned above, noise is added to obtain a noisy image. The noise image is subjected to noise reduction processing to obtain a second noise-reduced image. The process involves calculating a second loss function based on the second denoised image and the second training sample, The method of claim 4, further comprising adjusting the weights of the neural network to minimize the second loss function based on the second loss function to obtain the style attribute network.

7. Performing a transparency process on the non-target region of the initial image to obtain a target image with transparency channel information is: Based on the initial image, target identification is performed, and the target region of the initial image is obtained. The method of claim 1, comprising setting the target region to a non-transparent region and the non-target region to a transparent region, thereby obtaining a target image having transparent channel information.

8. An image generation device, An information determination module for determining prompt text and image style information, An image generation module for generating an initial image having the image style based on the semantic information of the prompt text, An image generation apparatus, comprising a transparency processing module for performing a transparency process on the non-target region of the initial image to obtain a target image having transparency channel information.

9. An electronic device comprising memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method according to any one of claims 1 to 7 is realized.

10. A non-temporary computer-readable storage medium, wherein the non-temporary computer-readable storage medium stores computer instructions, and the computer instructions are used to cause a computer to perform the method according to any one of claims 1 to 7.

11. A computer program product that includes a computer program instruction, and when the computer program instruction is executed on a computer, causes the computer to perform the method according to any one of claims 1 to 7.

Images