Image generation method and apparatus, device, medium, and program product

Abstract

The present disclosure provides an image generation method and apparatus, a device, a storage medium, and a program product. The method comprises: determining information of a prompt text and an image style; on the basis of semantic information of the prompt text, generating an initial image having the image style; and performing transparency processing on a non-target area of the initial image to obtain a target image having transparent channel information.

Description

Image generation method, device, equipment, medium and program product

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on July 25, 2023, with application number 202310921622.7 and invention name “Image generation method, device, equipment, medium and program product”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present disclosure relates to the field of computer technology, and in particular to an image generation method, apparatus, device, medium, and program product. Background Art

[0003] In some applications, users often use stickers to process image data, enriching the presentation of images like pictures and videos. Currently, sticker images require manual production, which is costly and time-consuming, and difficult to achieve aesthetically pleasing results. This makes it difficult for users to create the desired sticker images.

[0004] Summary of the Invention

[0005] The present disclosure proposes an image generation method, device, equipment, storage medium and program product to solve the technical problems of low efficiency and poor effect of sticker image production to a certain extent.

[0006] In a first aspect, the present disclosure provides an image generation method, comprising:

[0007] Information that determines the style of the prompt text and image;

[0008] generating an initial image having the image style based on semantic information of the prompt text;

[0009] Transparency processing is performed on the non-target area of ​​the initial image to obtain a target image with transparent channel information.

[0010] In a second aspect of the present disclosure, an image generating device is provided, comprising:

[0011] An information determination module, used to determine the prompt text and image style information;

[0012] An image generation module, configured to generate an initial image having the image style based on the semantic information of the prompt text;

[0013] The transparent processing module is used to perform transparent processing on the non-target area of ​​the initial image to obtain a target image with transparent channel information.

[0014] In a third aspect of the present disclosure, an electronic device is provided, characterized in that it includes one or more processors, a memory; and one or more programs, wherein the one or more programs are stored in the memory and executed by the one or more processors, and the programs include instructions for executing the method according to the first aspect or the second aspect.

[0015] According to a fourth aspect of the present disclosure, a non-volatile computer-readable storage medium containing a computer program is provided. When the computer program is executed by one or more processors, the processors execute the method described in the first aspect or the second aspect.

[0016] In a fifth aspect of the present disclosure, a computer program product is provided, comprising computer program instructions, which, when executed on a computer, cause the computer to execute the method described in the first aspect.

[0017] As can be seen from the foregoing, the image generation method, apparatus, device, medium, and program product provided herein generate an initial image using the semantic information of the prompt text and the image style information, and then perform transparency processing to obtain a target image with transparent channel information. This method can automatically generate sticker images based on the user's prompt text and desired image style without manual creation, thereby improving the quality and efficiency of sticker image creation. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present disclosure or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are only embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0019] FIG1 is a schematic diagram of an image generation architecture according to an embodiment of the present disclosure.

[0020] FIG2 is a schematic diagram of the hardware structure of an exemplary electronic device according to an embodiment of the present disclosure.

[0021] FIG3 is a schematic flowchart of an image generation method according to an embodiment of the present disclosure.

[0022] FIG4 is a schematic diagram of a target image according to an embodiment of the present disclosure.

[0023] FIG5 is a schematic diagram of an image generating device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0024] In order to make the objectives, technical solutions and advantages of the present disclosure more clearly understood, the present disclosure is further described in detail below in conjunction with specific embodiments and with reference to the accompanying drawings.

[0025] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present disclosure should have the usual meanings understood by people with ordinary skills in the field to which the present disclosure belongs. The "first", "second" and similar words used in the embodiments of the present disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative position relationships. When the absolute position of the described object changes, the relative position relationship may also change accordingly.

[0026] It is understandable that before using the technical solutions disclosed in the various embodiments of this disclosure, the type, scope of use, usage scenarios, etc. of the personal information involved in this disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0027] For example, in response to a user's active request, a prompt message is sent to the user to clearly inform the user that the operation requested will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the electronic device, application, server, storage medium, or other software or hardware that performs the operations of the disclosed technical solution based on the prompt message.

[0028] As an optional but non-limiting implementation, in response to receiving a user's active request, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. Furthermore, the pop-up window may also contain a selection control for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0029] It is understandable that the above notification and user authorization process are merely illustrative and do not limit the implementation of the present disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0030] FIG1 shows a schematic diagram of an image generation architecture according to an embodiment of the present disclosure. Referring to FIG1 , the image generation architecture 100 may include a server 110, a terminal 120, and a network 130 providing a communication link. The server 110 and the terminal 120 may be connected via a wired or wireless network 130. The server 110 may be an independent physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, security services, and CDN.

[0031] Terminal 120 can be implemented in hardware or software. For example, when implemented in hardware, terminal 120 can be any electronic device with a display screen that supports page display, including but not limited to smartphones, tablet computers, e-book readers, laptop computers, and desktop computers. When terminal 120 is implemented in software, it can be installed in the electronic devices listed above; it can be implemented as multiple software or software modules (such as software or software modules used to provide distributed services), or it can be implemented as a single software or software module, and no specific limitations are given here.

[0032] It should be noted that the image generation method provided in the embodiments of the present application can be executed by the terminal 120 or by the server 110. It should be understood that the number of terminals, networks, and servers in FIG1 is for illustration only and is not intended to be limiting. Any number of terminals, networks, and servers may be used as required.

[0033] Figure 2 shows a schematic diagram of the hardware structure of an exemplary electronic device 200 provided by an embodiment of the present disclosure. As shown in Figure 2, electronic device 200 may include: a processor 202, a memory 204, a network module 206, a peripheral interface 208, and a bus 210. The processor 202, memory 204, network module 206, and peripheral interface 208 are communicatively connected to each other within electronic device 200 via bus 210.

[0034] The processor 202 may be a central processing unit (CPU), an image generator, a neural network processor (NPU), a microcontroller (MCU), a programmable logic device (PLD), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), or one or more integrated circuits. The processor 202 may be used to perform functions related to the technology described in this disclosure. In some embodiments, the processor 202 may also include multiple processors integrated into a single logical component. For example, as shown in FIG2 , the processor 202 may include multiple processors 202 a, 202 b, and 202 c.

[0035] The memory 204 can be configured to store data (e.g., instructions, computer codes, etc.). As shown in Figure 2, the data stored in the memory 204 can include program instructions (e.g., program instructions for implementing the image generation method of the embodiment of the present disclosure) and data to be processed (e.g., the memory can store configuration files of other modules, etc.). The processor 202 can also access the program instructions and data stored in the memory 204, and execute the program instructions to operate on the data to be processed. The memory 204 can include a volatile storage device or a non-volatile storage device. In some embodiments, the memory 204 can include a random access memory (RAM), a read-only memory (ROM), an optical disc, a magnetic disk, a hard disk, a solid-state drive (SSD), a flash memory, a memory stick, etc.

[0036] The network module 206 can be configured to provide the electronic device 200 with communication with other external devices via a network. The network can be any wired or wireless network capable of transmitting and receiving data. For example, the network can be a wired network, a local wireless network (e.g., Bluetooth, WiFi, near field communication (NFC)), a cellular network, the Internet, or a combination thereof. It will be appreciated that the type of network is not limited to the specific examples above. In some embodiments, the network module 306 can include any number of network interface controllers (NICs), radio frequency modules, transceivers, modems, routers, gateways, adapters, cellular network chips, and the like.

[0037] The peripheral interface 208 can be configured to connect the electronic device 200 to one or more peripheral devices to implement information input and output. For example, the peripheral devices can include input devices such as a keyboard, a mouse, a touchpad, a touch screen, a microphone, and various sensors, and output devices such as a display, a speaker, a vibrator, and an indicator light.

[0038] The bus 210 can be configured to transmit information between the various components of the electronic device 200 (e.g., the processor 202, the memory 204, the network module 206, and the peripheral interface 208), such as an internal bus (e.g., a processor-memory bus), an external bus (USB port, PCI-E bus), etc.

[0039] It should be noted that although the architecture of the electronic device 200 shown above only includes the processor 202, the memory 204, the network module 206, the peripheral interface 208, and the bus 210, in a specific implementation, the architecture of the electronic device 200 may also include other components necessary for normal operation. In addition, those skilled in the art will understand that the architecture of the electronic device 200 may also include only the components necessary to implement the embodiments of the present disclosure, and does not necessarily include all the components shown in the figure.

[0040] In some applications, users often use sticker materials to process image data to enrich the presentation of image data such as pictures and videos. For example, sticker materials can be directly pasted onto pictures or videos. Currently, sticker images are typically created individually by designers. This method is costly and time-consuming, and requires certain artistic skills to create sticker images that meet aesthetic standards. Ordinary users often lack these skills, making it difficult to create the sticker images they desire. Therefore, improving the efficiency and effectiveness of sticker image creation has become a pressing technical challenge.

[0041] In light of this, embodiments of the present disclosure provide an image generation method, apparatus, device, storage medium, and program product. After generating an initial image using the semantic information of the prompt text and the image style information, a transparency process is performed to obtain a target image with transparent channel information. This method automatically generates sticker images based on the user's prompt text and desired image style without manual creation, improving the quality and efficiency of sticker image creation.

[0042] Referring to FIG3 , FIG3 shows a schematic flow chart of an image generation method according to an embodiment of the present disclosure. The image generation method according to an embodiment of the present disclosure can be deployed on a client. In FIG3 , the image generation method 300 can further include the following steps.

[0043] In step S310 , information about the prompt text and image style is determined.

[0044] The prompt text indicates the content of the image and contains semantic information. Specifically, the prompt text can include an object and its action, such as "A boy with a bright smile." Image style can refer to the presentation style of the image, such as oil painting, pixelated, childlike, or stick figure. It should be understood that the above image style attributes are merely examples and are not intended to be limiting. Image style attributes can also include more styles, which are not limited here.

[0045] Specifically, the user can determine the prompt text and image style information based on the interactive interface. For example, the user can input the prompt text based on the input component in the interactive interface. Different image styles can be presented in the interactive interface, which may include at least one preset image style, such as image style A, image style B, image style C, etc. Among them, an image style can be set as a default value. When the user does not select an image style, an image is generated based on the image style corresponding to the default value. The user can also determine the image style of the image to be generated based on an image style selection operation (such as clicking, etc.) in the interactive interface.

[0046] In step S320 , an initial image having the image style is generated based on the semantic information of the prompt text.

[0047] The initial image has semantic information corresponding to the prompt text. For example, if the prompt text is "A boy with a bright smile," the initial image may include a boy with a bright smile. Furthermore, the initial image may also have the image style determined in step S310. For example, if the image style attribute determined in step S310 is an oil painting style, the boy with a bright smile in the initial image may be presented in the oil painting style; if the image style determined in step S310 is a pixelated style, the boy with a bright smile in the initial image may be presented in the pixelated style.

[0048] In some embodiments, generating an initial image having the image style based on the semantic information of the prompt text includes:

[0049] Performing feature extraction on the prompt text to obtain text features;

[0050] determining, based on the semantic information, an image feature having a corresponding semantic relationship with the text feature;

[0051] Selecting corresponding image style attributes based on the image style information;

[0052] Based on the image style attributes and the image features, an initial image with the image style is obtained.

[0053] Specifically, feature extraction can be performed on the prompt text text to obtain text features F_text. Because the prompt text contains semantic information, the text features F_text can be used to determine corresponding image features F_image based on this semantic information. For example, based on the text features F_text, a match is performed within an image feature set, and the image feature with the highest semantic similarity to the text features F_text is determined as the image feature F_image having a corresponding semantic relationship with the text features F_text. The image feature set can be a pre-set database or can be obtained by feature extraction based on image data. The image feature set can include at least one image feature with semantic information. Then, based on the determined image style information, a corresponding image style attribute type can be determined. Based on the image style attribute type and the image features F_image, a corresponding initial image image_s_type having the image style attribute type can be generated. For example, feature extraction can be performed on the prompt text based on an image generation model to obtain text features. Image features corresponding to the semantic relationship can then be determined based on these features. Combined with the image style attributes corresponding to the image style information, an initial image image_s_type having the image style attribute type can be generated.

[0054] In some embodiments, generating an initial image having the image style based on the semantic information of the prompt text includes:

[0055] The prompt text and the image style information are input into an image generation model to obtain the initial image; wherein the image generation model is obtained based on the fusion of a text image network and a style attribute network.

[0056] The image generation model can generate an initial image with the image style based on the prompt text and the determined image style information. The image generation model can be obtained by fusing a text-image network and a style attribute network. The text-image network can generate a corresponding image (or image feature) based on the prompt text, and the style attribute network can combine the image (or image feature) generated by the text-image network with the determined image style to obtain an initial image with the image style.

[0057] In some embodiments, the image generation model is obtained based on the fusion of a text-image network and a style attribute network, and includes:

[0058] Training a neural network based on the first training sample to obtain a first weight matrix of the text image network; and training a neural network based on the second training sample to obtain a second weight matrix of the style attribute network;

[0059] A weight matrix of the image generation model is obtained based on the product of the first weight matrix and the second weight matrix.

[0060] The initial neural network can be trained based on the first training sample, for example, the initial diffusion model, to obtain a trained text-image network. The weights of each layer of the trained text-image network form a first weight matrix Q1. The initial neural network can be trained based on the second training sample, for example, the initial diffusion model is further trained to obtain a trained style attribute network. The weights of each layer of the trained style attribute network form a second weight matrix Q2. The weight matrices of the text-image network and the style attribute network are multiplied, and the resulting product Q1*Q2 is used as the weight matrix of the image generation model to fuse the text-image network and the style attribute network.

[0061] In some embodiments, the first training sample includes a text-image pair; and the method further includes:

[0062] Extracting features based on text samples in the text image pair to obtain training text features;

[0063] Performing denoising on the random noise image and performing cross-attention calculation on the denoising result and the training text feature to obtain a first denoised image;

[0064] calculating a first loss function based on image samples in the text-image pair and the first denoised image;

[0065] The weights of the neural network are adjusted based on the first loss function to minimize the first loss function, thereby obtaining the text image network.

[0066] In this case, an image that meets aesthetic standards can be used as the training image in the first training sample. The text-image network trained based on the first training sample can then output high-quality images that meet aesthetic standards, thereby improving the image generation effect. In this way, any user can generate high-quality images based on the prompt text, without requiring a certain level of artistic ability. Specifically, the first training sample can include a text-image pair, which can further include training text Text_training and a training image Image_training. Feature extraction is performed on the training text Text_training to obtain training text features F_Text_training. At least one denoising process can be performed on the random noise image image_noisy (e.g., denoising with a preset number of noise points M1). The result of each denoising process is then cross-attended with the training text features F_Text_training to obtain a first denoised image. In other words, the semantic characteristics of the training text features are used to guide the denoising process, so that the semantic characteristics of the image after each denoising process continuously approach the training text features, and a first denoised image with a semantic relationship relative to the training text features is predicted. The first denoised image predicted by the text image network can be compared with the training images in the first training sample to calculate the cross-entropy loss function between the two. The sum of the cross-entropy loss functions of all first training samples is used as the first loss function. During training, the weight parameters of the text image network are adjusted to minimize the first loss function.

[0067] In some embodiments, the second training sample includes an image having an image style; and the method further includes:

[0068] Adding noise based on the second training sample to obtain a noisy image;

[0069] Performing denoising processing on the noisy image to obtain a second denoised image;

[0070] Calculating a second loss function based on the second denoised image and the second training sample;

[0071] The weights of the neural network are adjusted based on the second loss function to minimize the second loss function, thereby obtaining the style attribute network.

[0072] Image data labeled with image style attributes can be used as the second training sample. The style attribute network trained based on this second training sample can then output images with the specified image style attributes. This allows users to create images that meet their expectations based on the determined image style attributes, satisfying their personalized needs for image style and improving the user experience. Combined with a text-to-image network, users only need to determine the image style attributes and enter prompt text to generate high-quality images of any style based on the image generation model, improving the effectiveness and efficiency of image creation. Specifically, the second training sample can include a style image Image_training_type labeled with the image style attribute type. The style image Image_training_type can be subjected to at least one noise addition process (noise can be added based on a preset number of noise points M2, such as Gaussian noise) to obtain a noisy image Image_training_type_noisy. The noisy image can then be subjected to at least one denoising process (noise can be removed based on a preset number of noise points M3) to obtain a second denoised image Image_training_type_denoisy. The second denoised image (Image_training_type_denoisy) and the style image (Image_training_type) in the second training sample can be compared to calculate the cross-entropy loss between them. The sum of the cross-entropy loss functions for all second training samples is used as the second loss function. During training, the weight parameters of the style attribute network are adjusted to minimize this second loss function.

[0073] In step S330 , a transparency process is performed on the non-target area of ​​the initial image to obtain a target image with transparent channel information.

[0074] Among them, the characteristic that sticker images are different from non-sticker images is that they have a transparent channel (i.e., alpha channel). In order to make the sticker image only show the main content, the generated initial image needs to be partially transparent except for the main content.

[0075] Specifically, referring to FIG4 , FIG4 shows a schematic diagram of the target image of the present application. In FIG4 , the user can input a prompt text text = "A girl dancing with joy." based on the interactive interface. It can be seen that the prompt text text includes the object girl and the action dancing, and an initial image with the content "a girl dancing with joy" can be generated. When the user selects different image style attributes, at least one target image corresponding to the image style attribute can be obtained, such as image style A, image style B, and image style C in FIG4 . The initial image is then made transparent, and the area outside the object girl is set as a transparent area, thereby obtaining the target image. The sticker image can be used in any image I. At this time, the area of ​​object B can be displayed in image I.

[0076] In some embodiments, performing transparency processing on the non-target area of ​​the initial image to obtain the target image with transparent channel information includes:

[0077] Performing target recognition based on the initial image to obtain a target area of ​​the initial image;

[0078] The target area is set as a non-transparent area, and the non-target area is set as a transparent area, so as to obtain a target image with transparent channel information.

[0079] Among them, the color channel information and transparent channel information in the image data with transparent channel information can be learned simultaneously based on the image processing model to obtain the ability to perform transparent processing on non-target areas. Specifically, the initial neural network can be trained based on the image data Image_alpha with transparent channel information. For example, the transparent channel information can be removed from the first image data Image_RGB_alpha with transparent channel information and color channel information to obtain the second image data Image_RGB containing only color channel information. The second image data Image_RGB is used as the input layer data, and the first image data Image_RGB_alpha is used as the output layer data. The initial neural network is trained to obtain a trained image processing model. The initial image is input into the image processing model to obtain a target image with transparent channel information, that is, the transparent channel information of the target area including the subject is a first value (for example, 1, i.e., a non-transparent area), and the transparent channel information of the non-target area is a second value (for example, 0, i.e., a transparent area).

[0080] In some embodiments, performing transparency processing on the non-target area of ​​the initial image to obtain the target image with transparent channel information includes:

[0081] The transparent channel information of the first pixel whose color channel information in the initial image is greater than or equal to the channel threshold is set to a first value, and the transparent channel information of the second pixel whose color channel information is less than the channel threshold is set to a second value, thereby obtaining a target image with transparent channel information.

[0082] In which, each pixel in the initial image with the determined image style attributes has color channel information (such as RGB channel information). For example, the color channel information of pixel X includes RGB channel values ​​​​abc (a, b, c are all [0, 255]). The non-target area of ​​the initial image can be made transparent based on this pixel distribution. Specifically, since the subject in the initial image is often significant, the pixels above the channel threshold in the color channel information can be used as the target area, and the other pixels can be used as the non-target area, thereby segmenting the target area including the subject to obtain a target mask map. For the pixels in the target mask map, the transparent channel information is set to the first value (such as 1), that is, the non-transparent area; the transparent channel information of the pixels in the area outside the target mask map is set to the second output (such as 0), that is, the transparent area.

[0083] It should be noted that the method of the embodiments of the present disclosure can be performed by a single device, such as a computer or server. The method of the embodiments of the present disclosure can also be applied in a distributed scenario, where multiple devices cooperate to perform the method. In such a distributed scenario, one of the multiple devices may only perform one or more steps of the method of the embodiments of the present disclosure, and the multiple devices will interact with each other to complete the method.

[0084] It should be noted that the above description is limited to some embodiments of the present disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in an order different from that described in the above embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0085] Based on the same technical concept, corresponding to any of the above embodiments and methods, the present disclosure further provides an image generating device, as shown in FIG5 , comprising:

[0086] An information determination module, used to determine the prompt text and image style information;

[0087] An image generation module, configured to generate an initial image having the image style based on the semantic information of the prompt text;

[0088] The transparent processing module is used to perform transparent processing on the non-target area of ​​the initial image to obtain a target image with transparent channel information.

[0089] For the convenience of description, the above devices are described as being functionally divided into various modules. Of course, when implementing the present disclosure, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0090] The apparatus of the above embodiment is used to implement the corresponding image generation method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0091] Based on the same technical concept, corresponding to any of the above-mentioned embodiment methods, the present disclosure also provides a non-transitory computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable the computer to execute the image generation method described in any of the above embodiments.

[0092] The computer-readable media of this embodiment include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0093] The computer instructions stored in the storage medium of the above embodiment are used to enable the computer to execute the image generation method described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0094] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present disclosure (including the claims) is limited to these examples. Within the scope of the present disclosure, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present disclosure as described above, which are not provided in detail for the sake of simplicity.

[0095] In addition, to simplify the description and discussion, and so as not to obscure the embodiments of the present disclosure, known power / ground connections to integrated circuit (IC) chips and other components may or may not be shown in the provided figures. In addition, devices may be shown in the form of block diagrams to avoid obscuring the embodiments of the present disclosure, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform on which the embodiments of the present disclosure are to be implemented (i.e., these details should be fully within the purview of those skilled in the art). Where specific details (e.g., circuits) are set forth to describe exemplary embodiments of the present disclosure, it will be apparent to those skilled in the art that the embodiments of the present disclosure may be implemented without these specific details or with variations in these specific details. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0096] Although the present disclosure has been described in conjunction with specific embodiments thereof, many alternatives, modifications, and variations of these embodiments will be apparent to those skilled in the art based on the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.

[0097] The embodiments of the present disclosure are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present disclosure should be included in the scope of protection of the present disclosure.

Claims

1. A method for generating an image, comprising: Determine the information of prompt text and image style; generating an initial image having the image style based on the semantic information of the prompt text; A transparent processing is performed on the non-target area of ​​the initial image to obtain a target image with transparent channel information.

2. The method according to claim 1, wherein Generating an initial image having the image style based on the semantic information of the prompt text includes: Performing feature extraction on the prompt text to obtain text features; Determining, based on the semantic information, an image feature having a corresponding semantic relationship with the text feature; Selecting a corresponding image style attribute based on the image style information; Based on the image style attribute and the image feature, an initial image with the image style is obtained.

3. The method according to claim 1, wherein Generating an initial image having the image style based on the semantic information of the prompt text includes: The prompt text and the information of the image style are input into an image generation model to obtain the initial image; wherein the image generation model is obtained based on the fusion of a text image network and a style attribute network.

4. The method according to claim 3, wherein: The image generation model is obtained based on the fusion of the text image network and the style attribute network, and includes: Training a neural network based on the first training sample to obtain a first weight matrix of the text image network; and training a neural network based on the second training sample to obtain the style attribute network to obtain a second weight matrix; Based on the product of the first weight matrix and the second weight matrix, a weight matrix of the image generation model is obtained.

5. The method according to claim 4, wherein: The first training sample includes a text-image pair; the method further includes: Extracting features based on text samples in the text image pair to obtain training text features; Performing denoising processing on the random noise image and performing cross-attention calculation on the result of the denoising processing and the training text feature to obtain a first denoised image; Calculating a first loss function based on image samples in the text image pair and the first denoised image; The weight of the neural network is adjusted based on the first loss function to minimize the first loss function, thereby obtaining the text image network.

6. The method according to claim 4, wherein: The second training sample includes an image with an image style; and the method further includes: Adding noise based on the second training sample to obtain a noisy image; Performing denoising processing on the noisy image to obtain a second denoised image; Calculate a second loss function based on the second denoised image and the second training sample; The weight of the neural network is adjusted based on the second loss function to minimize the second loss function, thereby obtaining the style attribute network.

7. The method according to claim 1, wherein: Performing transparency processing on the non-target area of ​​the initial image to obtain a target image with transparent channel information includes: Performing target recognition based on the initial image to obtain a target area of ​​the initial image; The target area is set as a non-transparent area, and the non-target area is set as a transparent area, so as to obtain a target image with transparent channel information.

8. An image generating device, comprising: An information determination module, used to determine the information of the prompt text and image style; An image generation module, used for generating an initial image having the image style based on the semantic information of the prompt text; The transparent processing module is used to perform transparent processing on the non-target area of ​​the initial image to obtain a target image with transparent channel information.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 1 to 7 when executing the program. 10 . A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to execute the method according to claim 1 .

11. A computer program product, comprising computer program instructions, which, when executed on a computer, enable the computer to execute the method according to any one of claims 1 to 7.

Images