Image generation method and device, equipment, medium and program product

Abstract

The invention provides an image generation method and device, equipment, a storage medium and a program product. The method comprises the following steps: determining information of a prompt text and an image style; generating an initial image with the image style based on semantic information of the prompt text; and performing transparency processing on the non-target area of the initial image to obtain a target image with transparent channel information.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to an image generation method, device, equipment, medium and program product. Background Art

[0002] In some applications, users often use sticker materials to process image data to enrich the expression of image data such as pictures and videos. Currently, sticker images need to be manually produced, which is costly and time-consuming to produce, and is not easy to meet aesthetic standards. Users cannot produce the sticker images they expect. Summary of the invention

[0003] The present disclosure proposes an image generation method, device, equipment, storage medium and program product to solve the technical problems of low efficiency and poor effect of sticker image production to a certain extent.

[0004] In a first aspect, the present disclosure provides an image generation method, comprising:

[0005] Determine the information of prompt text and image style;

[0006] generating an initial image having the image style based on the semantic information of the prompt text;

[0007] A transparent processing is performed on the non-target area of ​​the initial image to obtain a target image with transparent channel information.

[0008] In a second aspect of the present disclosure, there is provided an image generating device, comprising:

[0009] An information determination module, used to determine the information of the prompt text and image style;

[0010] An image generation module, used for generating an initial image having the image style based on the semantic information of the prompt text;

[0011] The transparent processing module is used to perform transparent processing on the non-target area of ​​the initial image to obtain a target image with transparent channel information.

[0012] According to a third aspect of the present disclosure, an electronic device is provided, characterized in that it includes one or more processors, a memory; and one or more programs, wherein the one or more programs are stored in the memory and executed by the one or more processors, and the program includes instructions for executing the method described in the first aspect or the second aspect.

[0013] According to a fourth aspect of the present disclosure, a non-volatile computer-readable storage medium containing a computer program is provided. When the computer program is executed by one or more processors, the processors execute the method described in the first aspect or the second aspect.

[0014] In a fifth aspect of the present disclosure, a computer program product is provided, comprising computer program instructions, which, when executed on a computer, cause the computer to execute the method described in the first aspect.

[0015] From the above, it can be seen that the image generation method, device, equipment, medium and program product provided by the present disclosure generate an initial image through the semantic information of the prompt text and the information of the image style, and then perform transparency processing to obtain a target image with transparent channel information. It can automatically generate sticker images according to the user's prompt text and the desired image style without manual production, thereby improving the production effect and production efficiency of sticker images. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the present disclosure or related technologies, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. Obviously, the drawings described below are only embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0017] Figure 1 A schematic diagram of an image generation architecture according to an embodiment of the present disclosure.

[0018] Figure 2 The figure is a schematic diagram of the hardware structure of an exemplary electronic device according to an embodiment of the present disclosure.

[0019] Figure 3 It is a schematic flowchart of the image generating method according to an embodiment of the present disclosure.

[0020] Figure 4 A schematic diagram of a target image according to an embodiment of the present disclosure.

[0021] Figure 5 Schematic diagram of an image generating device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0022] In order to make the objectives, technical solutions and advantages of the present disclosure more clearly understood, the present disclosure is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0023] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present disclosure should be understood by people with ordinary skills in the field to which the present disclosure belongs. The "first", "second" and similar words used in the embodiments of the present disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. "Including" or "comprising" and similar words mean that the elements or objects appearing before the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0024] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0025] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0026] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0027] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet the relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0028] Figure 1 FIG. 1 is a schematic diagram showing an image generation architecture of an embodiment of the present disclosure. Figure 1, the image generation architecture 100 may include a server 110, a terminal 120, and a network 130 providing a communication link. The server 110 and the terminal 120 may be connected via a wired or wireless network 130. The server 110 may be an independent physical server, or a server cluster or distributed system consisting of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, security services, and CDN.

[0029] The terminal 120 may be implemented in hardware or software. For example, when the terminal 120 is implemented in hardware, it may be various electronic devices having a display screen and supporting page display, including but not limited to smart phones, tablet computers, e-book readers, laptop portable computers, and desktop computers, etc. When the terminal 120 device is implemented in software, it may be installed in the electronic devices listed above; it may be implemented as multiple software or software modules (such as software or software modules used to provide distributed services), or it may be implemented as a single software or software module, which is not specifically limited here.

[0030] It should be noted that the image generation method provided in the embodiment of the present application can be executed by the terminal 120 or by the server 110. It should be understood that Figure 1 The number of terminals, networks and servers in the embodiment is only for illustration and is not intended to limit the number of terminals, networks and servers. Any number of terminals, networks and servers may be provided as required.

[0031] Figure 2 FIG. 2 shows a schematic diagram of the hardware structure of an exemplary electronic device 200 provided in an embodiment of the present disclosure. Figure 2 As shown, the electronic device 200 may include: a processor 202, a memory 204, a network module 206, a peripheral interface 208 and a bus 210. The processor 202, the memory 204, the network module 206 and the peripheral interface 208 are connected to each other in communication within the electronic device 200 through the bus 210.

[0032] Processor 202 may be a central processing unit (CPU), an image generator, a neural network processor (NPU), a microcontroller (MCU), a programmable logic device, a digital signal processor (DSP), an application specific integrated circuit (ASIC), or one or more integrated circuits. Processor 202 may be used to perform functions related to the technology described in this disclosure. In some embodiments, processor 202 may also include multiple processors integrated into a single logical component. For example, Figure 2 As shown, processor 202 may include a plurality of processors 202a, 202b, and 202c.

[0033] The memory 204 may be configured to store data (eg, instructions, computer code, etc.). Figure 2 As shown, the data stored in the memory 204 may include program instructions (e.g., program instructions for implementing the image generation method of the embodiment of the present disclosure) and data to be processed (e.g., the memory may store configuration files of other modules, etc.). The processor 202 may also access the program instructions and data stored in the memory 204, and execute the program instructions to operate on the data to be processed. The memory 204 may include a volatile storage device or a non-volatile storage device. In some embodiments, the memory 204 may include a random access memory (RAM), a read-only memory (ROM), an optical disk, a magnetic disk, a hard disk, a solid-state drive (SSD), a flash memory, a memory stick, etc.

[0034] The network module 206 can be configured to provide communication with other external devices to the electronic device 200 via a network. The network can be any wired or wireless network capable of transmitting and receiving data. For example, the network can be a wired network, a local wireless network (e.g., Bluetooth, WiFi, near field communication (NFC) etc.), a cellular network, the Internet or a combination thereof. It is understood that the type of network is not limited to the above specific examples. In some embodiments, the network module 306 can include any number of network interface controllers (NICs), radio frequency modules, transceivers, modems, routers, gateways, adapters, cellular network chips, etc., in any combination.

[0035] The peripheral interface 208 can be configured to connect the electronic device 200 to one or more peripheral devices to achieve information input and output. For example, the peripheral devices can include input devices such as a keyboard, a mouse, a touch pad, a touch screen, a microphone, and various sensors, and output devices such as a display, a speaker, a vibrator, and an indicator light.

[0036] The bus 210 can be configured to transmit information between various components of the electronic device 200 (e.g., the processor 202, the memory 204, the network module 206, and the peripheral interface 208), such as an internal bus (e.g., a processor-memory bus), an external bus (USB port, PCI-E bus), etc.

[0037] It should be noted that, although the architecture of the electronic device 200 only shows the processor 202, the memory 204, the network module 206, the peripheral interface 208 and the bus 210, in the specific implementation process, the architecture of the electronic device 200 may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the architecture of the electronic device 200 may also only include the components necessary for implementing the embodiments of the present disclosure, and does not necessarily include all the components shown in the figure.

[0038] In some applications, users often use sticker materials to process image data to enrich the expression of image data such as pictures and videos. For example, sticker materials are directly pasted into pictures or videos. Currently, sticker images are usually produced by designers one by one. This method has high production costs and a long production cycle, and to a certain extent requires designers to have certain artistic skills to produce sticker images that meet aesthetic standards. Ordinary users often do not have these abilities of professional designers and cannot produce the sticker images they expect. Therefore, how to improve the efficiency and effect of sticker image production has become a technical problem that needs to be solved urgently.

[0039] In view of this, the embodiments of the present disclosure provide an image generation method, device, equipment, storage medium and program product. After the initial image is generated by the semantic information of the prompt text and the information of the image style, a target image with transparent channel information is obtained by transparent processing. The sticker image can be automatically generated according to the user's prompt text and the desired image style without manual production, thereby improving the production effect and production efficiency of the sticker image.

[0040] See also Figure 3 , Figure 3 The schematic flow chart of the image generation method according to the embodiment of the present disclosure is shown. The image generation method according to the embodiment of the present disclosure can be deployed on the client. Figure 3 In the method 300 for generating an image, the method 300 for generating an image may further include the following steps.

[0041] In step S310, information about the prompt text and image style is determined.

[0042] The prompt text is used to indicate the content of the image and has semantic information. Specifically, the prompt text may include an object and may also include the action of the object, such as "A boy with bright smile". The image style may refer to the presentation style of the image, such as oil painting style, pixelated style, childlike style, stick figure style, etc. It should be understood that the above-mentioned image style attributes are only examples and are not intended to limit them. The image style attributes may also include more styles, which are not limited here.

[0043] Specifically, the user can determine the information of the prompt text and the image style based on the interactive interface. For example, the user can input the prompt text based on the input component in the interactive interface. Different image styles can be presented in the interactive interface, which may include at least one preset image style, such as image style A, image style B, image style C, etc. Among them, an image style can be set as a default value, and when the user does not select the image style, an image is generated based on the image style corresponding to the default value. The user can also determine the image style of the image to be generated based on the selection operation of the image style in the interactive interface (such as clicking, etc.).

[0044] In step S320, an initial image having the image style is generated based on the semantic information of the prompt text.

[0045] The initial image has semantic information corresponding to the prompt text. For example, if the prompt text is "A boy with bright smile", the content of the initial image may include a boy with a bright smile. In addition, the initial image may also have the image style determined in step S310. For example, if the image style attribute determined in step S310 is an oil painting style, the boy with a bright smile in the initial image may be presented in an oil painting style; if the image style determined in step S310 is a pixelated style, the boy with a bright smile in the initial image may be presented in a pixelated style.

[0046] In some embodiments, generating an initial image having the image style based on the semantic information of the prompt text includes:

[0047] Performing feature extraction on the prompt text to obtain text features;

[0048] Determining, based on the semantic information, an image feature having a corresponding semantic relationship with the text feature;

[0049] Selecting a corresponding image style attribute based on the image style information;

[0050] Based on the image style attribute and the image feature, an initial image with the image style is obtained.

[0051] Specifically, feature extraction can be performed on the prompt text text to obtain text feature F_text. Since the prompt text has semantic information, the text feature F_text can determine the corresponding image feature F_image based on the semantic information. For example, matching is performed in the image feature set based on the text feature F_text, and the image feature with the highest semantic similarity with the text feature F_text is determined as the image feature F_image having a corresponding semantic relationship with the text feature F_text. Among them, the image feature set can be a pre-set database, and can be obtained by feature extraction based on image data. The image feature set can include at least one image feature with semantic information. Then, the corresponding image style attribute type is determined based on the determined image style information. Based on the image style attribute type and the above-mentioned image feature F_image, a corresponding initial image image_s_type with the image style attribute type can be generated. For example, feature extraction can be performed on the prompt text based on the image generation model, and after obtaining the text feature, the image feature with the corresponding semantic relationship is determined based on it; then, the image style attribute corresponding to the image style information is combined to generate the initial image image_s_type with the image style attribute.

[0052] In some embodiments, generating an initial image having the image style based on the semantic information of the prompt text includes:

[0053] The prompt text and the information of the image style are input into an image generation model to obtain the initial image; wherein the image generation model is obtained based on the fusion of a text image network and a style attribute network.

[0054] The image generation model can generate an initial image with the image style based on the prompt text and the determined image style information. The image generation model can be obtained by fusing a text image network and a style attribute network. The text image network can generate a corresponding image (or image feature) based on the prompt text, and the style attribute network can combine the image (or image feature) generated by the text image network and the determined image style to obtain an initial image with the image style.

[0055] In some embodiments, the image generation model is obtained based on the fusion of the text image network and the style attribute network, including:

[0056] Training a neural network based on the first training sample to obtain a first weight matrix of the text image network; and training a neural network based on the second training sample to obtain the style attribute network to obtain a second weight matrix;

[0057] Based on the product of the first weight matrix and the second weight matrix, a weight matrix of the image generation model is obtained.

[0058] The initial neural network can be trained based on the first training sample, for example, the initial diffusion model can be trained to obtain a trained text image network, and the weights of each layer of the trained text image network form a first weight matrix Q1. The initial neural network can be trained based on the second training sample, for example, the initial diffusion model can be trained to obtain a trained style attribute network, and the weights of each layer of the trained style attribute network form a second weight matrix Q2. The weight matrices of the text image network and the style attribute network are multiplied, and the product result Q1*Q2 is used as the weight matrix of the image generation model to fuse the text image network and the style attribute network.

[0059] In some embodiments, the first training sample includes a text-image pair; and the method further includes:

[0060] Extracting features based on text samples in the text image pair to obtain training text features;

[0061] Performing denoising processing on the random noise image and performing cross-attention calculation on the result of the denoising processing and the training text feature to obtain a first denoised image;

[0062] Calculating a first loss function based on image samples in the text image pair and the first denoised image;

[0063] The weight of the neural network is adjusted based on the first loss function to minimize the first loss function, thereby obtaining the text image network.

[0064] Among them, an image that meets the aesthetic standard can be used as a training image in the first training sample, and the text image network trained based on the first training sample can output a high-quality image that meets the aesthetic standard, thereby improving the effect of image generation. In this way, there is no need to require the user to have a certain degree of artistic ability, and any user can generate a high-quality image based on the prompt text. Specifically, the first training sample can include a text image pair, and the text image pair can further include a training text Text_training and a training image Image_training. The training text Text_training is feature extracted to obtain a training text feature F_Text_training. The random noise image image_noisy can be subjected to at least one denoising process (for example, denoising with a preset number of noise points M1), and the result of each denoising process is cross-attention calculated with the training text feature F_Text_training to obtain a first denoised image. That is, the semantic characteristics of the training text feature are used to guide the denoising process, so that the semantic characteristics of the image after each denoising are constantly close to the training text feature, and the first denoised image with a semantic relationship relative to the training text feature is predicted. The first denoised image predicted by the text image network can be compared with the training image in the first training sample, and the cross entropy loss function of the two can be calculated; and the sum of the cross entropy loss functions of all the first training samples can be used as the first loss function. During the training process, the weight parameters of the text image network are adjusted to minimize the first loss function.

[0065] In some embodiments, the second training sample includes an image having an image style; and the method further includes:

[0066] Adding noise based on the second training sample to obtain a noisy image;

[0067] Performing denoising processing on the noisy image to obtain a second denoised image;

[0068] Calculate a second loss function based on the second denoised image and the second training sample;

[0069] The weight of the neural network is adjusted based on the second loss function to minimize the second loss function, thereby obtaining the style attribute network.

[0070] Among them, image data annotated with image style attributes can be used as the second training sample, and the style attribute network trained based on the second training sample can output an image with the specified image style attribute. In this way, the user can produce an image that meets his or her expectations based on the determined image style attributes, meet the user's personalized needs for image style, and improve the user experience. Combined with the text image network, the user only needs to determine the image style attributes and input the prompt text to generate high-quality images of any style based on the image generation model, thereby improving the effect and efficiency of image production. Specifically, for the second training sample, a style image Image_training_type annotated with the image style attribute type can be included, and the style image Image_training_type can be subjected to at least one noise addition process (noise can be added based on a preset number of noise points M2, such as Gaussian noise) to obtain a noise image Image_training_type_noisy. Then, the noise image is subjected to at least one denoising process (denoising can be performed based on a preset number of noise points M3) to obtain a second denoised image Image_training_type_denoisy. The second denoised image Image_training_type_denoisy and the style image Image_training_type in the second training sample can be compared to calculate the cross entropy loss of the two. The sum of the cross entropy loss functions of all second training samples is used as the second loss function. During the training process, the weight parameters of the style attribute network are adjusted to minimize the second loss function.

[0071] In step S330, a transparency process is performed on the non-target area of ​​the initial image to obtain a target image with transparent channel information.

[0072] Among them, the sticker image is different from the non-sticker image in that it has a transparent channel (ie, alpha channel). In order to make the sticker image only show the main content, the generated initial image needs to be partially transparent except for the main content.

[0073] Specifically, see Figure 4 , Figure 4 A schematic diagram showing the target image of the present application is shown. Figure 4 In the example, the user can input the prompt text text="A girl dancing with joy." based on the interactive interface. It can be seen that the prompt text text includes the object girl and the action dancing, and the initial image with the content "a girl dancing with joy" can be generated. When the user selects different image style attributes, at least one target image corresponding to the image style attribute can be obtained, for example Figure 4The image style A, image style B, and image style C in the image are shown in Figure 1. Then the initial image is made transparent, and the area outside the object girl is set as a transparent area, so as to obtain the target image. The sticker image can be used in any image I. At this time, the area of ​​object B can be displayed in image I.

[0074] In some embodiments, performing transparency processing on the non-target area of ​​the initial image to obtain a target image with transparent channel information includes:

[0075] Performing target recognition based on the initial image to obtain a target area of ​​the initial image;

[0076] The target area is set as a non-transparent area, and the non-target area is set as a transparent area, so as to obtain a target image with transparent channel information.

[0077] Among them, the color channel information and transparent channel information in the image data with transparent channel information can be learned simultaneously based on the image processing model to obtain the ability to perform transparent processing on the non-target area. Specifically, the initial neural network can be trained based on the image data Image_alpha with transparent channel information. For example, the transparent channel information can be removed from the first image data Image_RGB_alpha with transparent channel information and color channel information to obtain the second image data Image_RGB containing only the color channel information. The second image data Image_RGB is used as the input layer data, and the first image data Image_RGB_alpha is used as the output layer data. The initial neural network is trained to obtain a trained image processing model. The initial image is input into the image processing model to obtain a target image with transparent channel information, that is, the transparent channel information of the target area including the subject is a first value (for example, 1, i.e., a non-transparent area), and the transparent channel information of the non-target area is a second value (for example, 0, i.e., a transparent area).

[0078] In some embodiments, performing transparency processing on the non-target area of ​​the initial image to obtain a target image with transparent channel information includes:

[0079] Based on the transparent channel information of the first pixel whose color channel information in the initial image is greater than or equal to the channel threshold, set to a first value, and set the transparent channel information of the second pixel whose color channel information is less than the channel threshold to a second value, a target image with transparent channel information is obtained.

[0080] Among them, each pixel in the initial image with the determined image style attribute has color channel information (such as RGB channel information). For example, the color channel information of pixel X includes RGB channel values ​​​​of abc (a, b, c are all [0, 255]). The non-target area of ​​the initial image can be transparently processed based on the pixel distribution. Specifically, since the subject in the initial image is often significant, the pixels above the channel threshold in the color channel information can be used as the target area, and the other pixels can be used as the non-target area, so that the target area including the subject is segmented to obtain a target mask map. For the pixels in the target mask map, the transparent channel information is set to the first value (such as 1), that is, the non-transparent area; the transparent channel information of the pixels in the area outside the target mask map is set to the second output (such as 0), that is, the transparent area.

[0081] It should be noted that the method of the embodiment of the present disclosure can be performed by a single device, such as a computer or a server. The method of the present embodiment can also be applied in a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of the embodiment of the present disclosure, and the multiple devices will interact with each other to complete the described method.

[0082] It should be noted that the above describes some embodiments of the present disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the above embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0083] Based on the same technical concept, corresponding to any of the above-mentioned embodiments and methods, the present disclosure also provides an image generating device, see Figure 5 , the image generating device comprises:

[0084] An information determination module, used to determine the information of the prompt text and image style;

[0085] An image generation module, used for generating an initial image having the image style based on the semantic information of the prompt text;

[0086] The transparent processing module is used to perform transparent processing on the non-target area of ​​the initial image to obtain a target image with transparent channel information.

[0087] For the convenience of description, the above device is described by dividing it into various modules according to its functions. Of course, when implementing the present disclosure, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0088] The device of the above embodiment is used to implement the corresponding image generation method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0089] Based on the same technical concept, corresponding to any of the above-mentioned embodiment methods, the present disclosure also provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the image generation method described in any of the above embodiments.

[0090] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0091] The computer instructions stored in the storage medium of the above embodiments are used to enable the computer to execute the image generation method described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0092] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present disclosure (including the claims) is limited to these examples. Based on the concept of the present disclosure, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present disclosure as described above, which are not provided in detail for the sake of simplicity.

[0093] In addition, to simplify the description and discussion, and in order not to make the embodiments of the present disclosure difficult to understand, the known power / ground connections to the integrated circuit (IC) chips and other components may or may not be shown in the provided figures. In addition, the device can be shown in the form of a block diagram to avoid making the embodiments of the present disclosure difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform on which the embodiments of the present disclosure will be implemented (that is, these details should be fully within the scope of understanding of those skilled in the art). Where specific details (e.g., circuits) are set forth to describe exemplary embodiments of the present disclosure, it is apparent to those skilled in the art that the embodiments of the present disclosure can be implemented without these specific details or with changes in these specific details. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0094] Although the present disclosure has been described in conjunction with specific embodiments of the present disclosure, many replacements, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.

[0095] The embodiments of the present disclosure are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present disclosure should be included in the scope of protection of the present disclosure.

Claims

1. A method for generating an image, comprising: Determine the information of prompt text and image style; generating an initial image having the image style based on the semantic information of the prompt text; A transparent processing is performed on the non-target area of ​​the initial image to obtain a target image with transparent channel information.

2. The method according to claim 1, wherein Generating an initial image having the image style based on the semantic information of the prompt text includes: Performing feature extraction on the prompt text to obtain text features; Determining, based on the semantic information, an image feature having a corresponding semantic relationship with the text feature; Selecting a corresponding image style attribute based on the image style information; Based on the image style attribute and the image feature, an initial image with the image style is obtained.

3. The method according to claim 1, wherein Generating an initial image having the image style based on the semantic information of the prompt text includes: The prompt text and the information of the image style are input into an image generation model to obtain the initial image; wherein the image generation model is obtained based on the fusion of a text image network and a style attribute network.

4. The method according to claim 3, wherein: The image generation model is obtained based on the fusion of the text image network and the style attribute network, and includes: Training a neural network based on the first training sample to obtain a first weight matrix of the text image network; and training a neural network based on the second training sample to obtain the style attribute network to obtain a second weight matrix; Based on the product of the first weight matrix and the second weight matrix, a weight matrix of the image generation model is obtained.

5. The method according to claim 4, wherein: The first training sample includes a text-image pair; the method further includes: Extracting features based on text samples in the text image pair to obtain training text features; Performing denoising processing on the random noise image and performing cross-attention calculation on the result of the denoising processing and the training text feature to obtain a first denoised image; Calculating a first loss function based on image samples in the text image pair and the first denoised image; The weight of the neural network is adjusted based on the first loss function to minimize the first loss function, thereby obtaining the text image network.

6. The method according to claim 4, wherein: The second training sample includes an image with an image style; and the method further includes: Adding noise based on the second training sample to obtain a noisy image; Performing denoising processing on the noisy image to obtain a second denoised image; Calculate a second loss function based on the second denoised image and the second training sample; The weight of the neural network is adjusted based on the second loss function to minimize the second loss function, thereby obtaining the style attribute network.

7. The method according to claim 1, wherein: Performing transparency processing on the non-target area of ​​the initial image to obtain a target image with transparent channel information includes: Performing target recognition based on the initial image to obtain a target area of ​​the initial image; The target area is set as a non-transparent area, and the non-target area is set as a transparent area, so as to obtain a target image with transparent channel information.

8. An image generating device, comprising: An information determination module, used to determine the information of the prompt text and image style; An image generation module, used for generating an initial image having the image style based on the semantic information of the prompt text; The transparent processing module is used to perform transparent processing on the non-target area of ​​the initial image to obtain a target image with transparent channel information.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 1 to 7 when executing the program. 10 . A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to execute the method according to claim 1 .

11. A computer program product, comprising computer program instructions, which, when executed on a computer, enable the computer to execute the method according to any one of claims 1 to 7.

Images