Image generation method, device, medium and equipment based on denoising network

Abstract

The present invention provides a denoising network-based image generation method, apparatus, medium and device, which are applied to multimodal artificial intelligence content generation scenarios. A plurality of cartoon image samples and a plurality of randomly initialized noise samples are obtained and input into an improved U-net network for noise training to obtain a trained U-net network. After denoising target text data for generating cartoon images using the compression path in the trained U-net network, the target text data for generating cartoon images is restored into a clear cartoon image using convolutional sublayers, squeezing sublayers, excitation sublayers and fully connected sublayers integrated into all upsampling layers of the U-net network, thereby improving the sensitivity of feature extraction and accurately restoring images with a high degree of match to semantic information.

Description

Technical Field

[0001] The present invention relates to the field of dynamic diffuse generation technology, and in particular to an image generation method, device, medium and equipment based on a denoising network. Background Art

[0002] In recent years, image denoising technology has been widely used in the field of computer vision and has become an important means of digital image processing. It aims to reduce various noises introduced or artificially added during the imaging process. It is usually used as an important part of the image preprocessing stage or to enhance image clarity.

[0003] Existing technologies include image denoising through the U-net denoising module in the stable-diffusion model. It takes a randomly initialized noise vector as input and then restores the noise vector through a backbone network. This function enables the model to generate diverse images based on semantic expression through learned parameter information. This technology is applied to multimodal artificial intelligence generated content (AIGC) scenarios and can greatly improve the creativity of image generation. However, due to the complexity of the AIGC multimodal field and the variability of image scenes, it cannot accurately restore images with a high degree of match with semantic information. Summary of the Invention

[0004] The present invention provides a denoising network-based image generation method, apparatus, medium and device, the purpose of which is to accurately restore an image with a high degree of matching with semantic information.

[0005] To achieve the above objectives, the present invention provides an image generation method based on a denoising network, which is applied to multimodal artificial intelligence content generation scenarios, including:

[0006] Step 1: obtain multiple animation image samples and multiple randomly initialized noise samples;

[0007] Step 2: Input multiple cartoon image samples and multiple randomly initialized noise samples into the improved U-net network for noise training to obtain a trained U-net network;

[0008] Step 3: Input the target text data of the animated image to be generated into the trained U-net network for denoising to generate the denoised animated image;

[0009] The improved U-net network is based on the U-net network model, and all upsampling layers of the U-net network include a convolution sublayer for extracting semantic information features of the target text data, a squeezing sublayer for compressing the extracted semantic information features, an excitation sublayer for obtaining the weights of the semantic information features, and a fully connected sublayer for fusing the semantic information features using the weights of the semantic information features.

[0010] The input end of the convolutional sublayer is the input end of the upsampling layer in the U-net network, which is used to connect to the output end of the network layer above the upsampling layer.

[0011] The output of the convolutional sublayer is connected to the input of the squeeze sublayer and the input of the fully connected sublayer respectively;

[0012] The output end of the squeezing sublayer is connected to the input end of the excitation sublayer;

[0013] The output of the excitation sublayer is connected to the input of the fully connected sublayer;

[0014] The output end of the fully connected sublayer is the output end of the upsampling layer in the U-net network, which is used to connect to the input end of the next level network layer of the upsampling layer.

[0015] Furthermore, the randomly initialized noise image is a random noise matrix that conforms to the Gaussian distribution, and the spatial dimension of the random noise matrix is ​​consistent with the spatial dimension of the animation image.

[0016] Furthermore, the convolution sublayer is used to extract the semantic information output by the previous network layer of the upsampling layer, obtain the convolutional feature map and input it into the fully connected sublayer and the squeeze sublayer;

[0017] The squeeze sublayer is used to compress the convolutional feature map into the channel descriptor and calculate the channel information through global average pooling to obtain a one-dimensional feature vector;

[0018] The excitation sublayer is used to learn the feature points of the one-dimensional feature vector and obtain the feature weights using the feature points;

[0019] The fully connected sublayer is used to fuse the feature weights with the convolutional feature map to obtain the fused feature map and input it into the next level network layer of the upsampling layer.

[0020] Furthermore, multiple cartoon image samples and multiple randomly initialized noise samples are input into the improved U-net network for noise training, including:

[0021] For each animation image sample in turn, all randomly initialized noise samples are used to perform noise processing on the animation image sample to different degrees to obtain multiple noise images;

[0022] The improved U-net network is trained with noise using multiple noisy images.

[0023] Furthermore, the loss function during noise training is for:

[0024]

[0025] Among them, cos(·) represents the scoring function, y + 、y - represent positive samples and negative samples respectively, Represents the sample pair set constructed using the scoring function, z x represents the sample vector of anime image, represents the output comparison sample vector, and ξ represents the adjustment parameter.

[0026] Furthermore, the adjustment parameter is determined by the sorting distance between positive and negative samples in the sample pair set;

[0027] The expression for the adjustment parameter is:

[0028] ξ=γ*(rank(y - )-rank(y + ))

[0029] Among them, γ represents the weight control coefficient, rank(y - ) represents the ranking score of negative samples, rank(y + ) represents the ranking score of the positive sample, rank(y - )-rank(y + ) represents the ranking distance between sample pairs.

[0030] Further,

[0031] The present invention also provides an image generation device based on a denoising network, which is applied to multimodal artificial intelligence content generation scenarios, including:

[0032] An acquisition module, used to acquire multiple animation image samples and multiple randomly initialized noise samples;

[0033] A training module is used to input multiple cartoon image samples and multiple randomly initialized noise samples into the improved U-net network for noise training to obtain a trained U-net network;

[0034] A denoising module is used to input the target text data of the animated image to be generated into the trained U-net network for denoising, thereby generating a denoised animated image;

[0035] The improved U-net network is based on the U-net network model, and all upsampling layers of the U-net network include a convolution sublayer for extracting semantic information features of the target text data, a squeezing sublayer for compressing the extracted semantic information features, an excitation sublayer for obtaining the weights of the semantic information features, and a fully connected sublayer for fusing the semantic information features using the weights of the semantic information features.

[0036] The input end of the convolutional sublayer is the input end of the upsampling layer in the U-net network, which is used to connect to the output end of the network layer above the upsampling layer.

[0037] The output of the convolutional sublayer is connected to the input of the squeeze sublayer and the input of the fully connected sublayer respectively;

[0038] The output end of the squeezing sublayer is connected to the input end of the excitation sublayer;

[0039] The output of the excitation sublayer is connected to the input of the fully connected sublayer;

[0040] The output end of the fully connected sublayer is the output end of the upsampling layer in the U-net network, which is used to connect to the input end of the next level network layer of the upsampling layer.

[0041] The present invention also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the image generation method based on the denoising network is implemented.

[0042] The present invention also provides a terminal device, including a memory, a processor, and a computer program stored in the memory and runnable on the processor. When the processor executes the computer program, an image generation method based on a denoising network is implemented.

[0043] The above solution of the present invention has the following beneficial effects:

[0044] The present invention is applied to multimodal artificial intelligence content generation scenarios, by obtaining multiple cartoon image samples and multiple randomly initialized noise samples; inputting the multiple cartoon image samples and the multiple randomly initialized noise samples into an improved U-net network for noise training to obtain a trained U-net network; inputting target text data of the cartoon image to be generated into the trained U-net network for denoising to generate a denoised cartoon image; the improved U-net network is based on the U-net network model, and all upsampling layers of the U-net network include convolution sublayers, squeezing sublayers, excitation sublayers, and fully connected sublayers; compared with the prior art In comparison, the present invention utilizes the compression path in the U-net network to denoise the target text data for generating the animation image, and then utilizes a convolution sublayer for extracting semantic information features of the target text data, a squeeze sublayer for compressing the extracted semantic information features, an excitation sublayer for obtaining the weights of the semantic information features, and a fully connected sublayer for fusing the semantic information features using the weights of the semantic information features to restore the target text data for generating the animation image into a clear animation image, thereby improving the sensitivity to feature extraction and accurately restoring an image with a high degree of matching with the semantic information.

[0045] Other beneficial effects of the present invention will be described in detail in the subsequent specific implementation section. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 A schematic diagram of a flow chart of an embodiment of the present invention;

[0047] Figure 2 Schematic diagram of the structure of the downsampling sublayer in an embodiment of the present invention. DETAILED DESCRIPTION

[0048] To make the technical problems, technical solutions, and advantages to be solved by the present invention more clear, the following is a detailed description with reference to the accompanying drawings and specific embodiments. It is obvious that the embodiments described are only some of the embodiments of the present invention, not all of them. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0049] In the description of the present invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings and are intended solely to facilitate and simplify the description of the present invention. They are not intended to indicate or imply that the devices or components referred to must have, be constructed, or operate in a specific orientation, and therefore should not be construed as limitations on the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0050] In the description of the present invention, it should be noted that, unless otherwise expressly specified or limited, the terms "mounted," "connected," and "connected" should be understood broadly. For example, they may refer to a locking connection, a detachable connection, or an integral connection; they may refer to a mechanical connection or an electrical connection; they may refer to a direct connection or an indirect connection through an intermediate medium; and they may refer to internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on the specific circumstances.

[0051] In addition, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0052] In response to existing problems, the present invention provides an image generation method, device, medium and equipment based on a denoising network.

[0053] like Figure 1 As shown, an embodiment of the present invention provides an image generation method based on a denoising network, which is applied to multimodal artificial intelligence content generation scenarios, including:

[0054] Step 1: obtain multiple animation image samples and multiple randomly initialized noise samples;

[0055] Step 2: Input multiple cartoon image samples and multiple randomly initialized noise samples into the improved U-net network for noise training to obtain a trained U-net network;

[0056] Step 3: Input the target text data of the animated image to be generated into the trained U-net network for denoising to generate the denoised animated image;

[0057] The improved U-net network is based on the U-net network model, and all upsampling layers of the U-net network include a convolution sublayer for extracting semantic information features of the target text data, a squeezing sublayer for compressing the extracted semantic information features, an excitation sublayer for obtaining the weights of the semantic information features, and a fully connected sublayer for fusing the semantic information features using the weights of the semantic information features.

[0058] The input end of the convolutional sublayer is the input end of the upsampling layer in the U-net network, which is used to connect to the output end of the network layer above the upsampling layer.

[0059] The output of the convolutional sublayer is connected to the input of the squeeze sublayer and the input of the fully connected sublayer respectively;

[0060] The output end of the squeezing sublayer is connected to the input end of the excitation sublayer;

[0061] The output of the excitation sublayer is connected to the input of the fully connected sublayer;

[0062] The output end of the fully connected sublayer is the output end of the upsampling layer in the U-net network, which is used to connect to the input end of the next level network layer of the upsampling layer.

[0063] The basic model of the improved U-net network in the embodiment of the present invention is a traditional U-net network, which includes a compression path and an expansion path. The compression path includes nine convolutional layers and four downsampling layers. In the compression path, the image needs to be downsampled once for dimensionality reduction after each convolution process through two convolutional layers. After the last downsampling layer is processed, a convolution process is performed again to output a low-dimensional image and input the expansion path. The expansion path includes 10 convolutional layers and 4 upsampling layers consisting of a convolution sublayer, a squeeze sublayer, an excitation sublayer, and a fully connected sublayer. The first convolution layer in the expansion path performs convolution processing on the low-dimensional feature map and then inputs it into the first upsampling layer for upsampling. After that, it goes through two convolution processes, one upsampling process, two convolution processes, one upsampling process, two convolution processes, one upsampling process, and three convolution processes in sequence to output the denoised animated image. The last convolution layer is a 1*1 convolution kernel, and the other convolution layers are all 3*3 convolution kernels.

[0064] In the embodiment of the present invention, the upper network layer of the upsampling layer and the lower network layer of the upsampling layer are both convolutional layers.

[0065] Specifically, the noise image is randomly initialized to a random noise matrix that conforms to the Gaussian distribution, and the spatial dimension of the random noise matrix is ​​consistent with the spatial dimension of the animation image.

[0066] Specifically, the convolution sublayer is used to extract the semantic information output by the previous network layer of the upsampling layer, obtain the convolution feature map and input it into the fully connected sublayer and the squeeze sublayer;

[0067] The squeeze sublayer is used to compress the convolutional feature map into the channel descriptor and calculate the channel information through global average pooling to obtain a one-dimensional feature vector for each channel;

[0068] The excitation sublayer is used to learn the feature points of the one-dimensional feature vector and obtain the feature weights using the feature points;

[0069] The fully connected sublayer is used to fuse the feature weights with the convolutional feature map to obtain the fused feature map and input it into the next level network layer of the upsampling layer.

[0070] In the embodiment of the present invention, the convolution sublayer is a convolution layer in the traditional sense; the squeezing sublayer is a squeezing layer in the traditional sense, which uses a global average pooling operation to compress the channel space dimension of the feature map from H*W to 1*1, with an input dimension of (c, h, w) and an output dimension of (c, 1, 1); the excitation sublayer is an excitation layer in the traditional sense, with an input size of C and an output size of C / r, where r is the dimensionality reduction rate. In the embodiment of the present invention, the dimensionality reduction rate is 16, which is used to learn the hot spots and areas of interest of the feature map; the fully connected sublayer is a fully connected layer in the traditional sense.

[0071] Specifically, multiple cartoon image samples and multiple randomly initialized noise samples are input into the improved U-net network for noise training, including:

[0072] For each animation image sample in turn, all randomly initialized noise samples are used to perform noise processing on the animation image sample to different degrees to obtain multiple noise images;

[0073] The improved U-net network is trained with noise using multiple noisy images.

[0074] Specifically, the embodiment of the present invention uses the N-pairs contrast loss function to train the improved U-net network. The training process is as follows:

[0075] An initial scoring function is defined. In this embodiment of the present invention, a cosine similarity function is used as the initial scoring function to evaluate the similarity between the generated image sample and the real image sample.

[0076] For each real generated image sample, multiple negative image samples are generated through network forward inference;

[0077] Use the scoring function to calculate the similarity between the real sample and all negative samples, and sort the negative samples according to the similarity to obtain the sorting result;

[0078] According to the sorting results, the negative sample closest to the true sample is selected as the "good" sample (y+), and the negative sample with the least similarity is selected as the "bad" sample (y-).

[0079] According to the formula ξ=γ*(rank(y - )-rank(y +)) Calculate the similarity difference between positive and negative sample pairs to dynamically adjust parameters;

[0080] The network training loss is calculated according to the following formula:

[0081]

[0082] Among them, z x is the vector representation of the real sample, and are the vector representations of good samples and bad samples respectively.

[0083] Specifically, the loss function of the improved U-net network is for:

[0084]

[0085] Among them, cos(·) represents the scoring function, y + 、y - represent positive samples and negative samples respectively, Represents a set of sample pairs constructed using a scoring function. Superscripts + and - indicate the quality of the compared samples. x represents the sample vector of anime image, represents the output comparison sample vector, and ξ represents the adjustment parameter.

[0086] Specifically, the adjustment parameter is determined by the sorting distance between positive and negative samples in the sample pair set;

[0087] The expression for the adjustment parameter is:

[0088] ξ=γ*(rank(y - )-rank(y + ))

[0089] Among them, γ represents the weight control coefficient, and through experimental verification, γ=0.512, rank(y - ) represents the ranking score of negative samples, rank(y + ) represents the ranking score of the positive sample, rank(y - )-rank(y + Represents the ranking distance between sample pairs.

[0090] In an embodiment of the present invention, the main idea of ​​the N-pairs contrast loss is: in the original contrast loss, all negative samples are treated equally. However, in the text generation scenario, some generated sequences may be very similar to the real sequences, while others are very different. Therefore, there are good and bad negative samples in the set. Therefore, in order to increase the discard rate of poor-quality contrast samples, improve the accuracy of the network, retain more high-quality samples, and enhance the generalization ability of the network, it is necessary to use the N-pairs contrast loss to train the network.

[0091] In an embodiment of the present invention, target text data to be generated for an animated image is input into a trained U-net network, and the text semantic information and the noise image of the target text data are first separated through two convolution processes, and the text semantic information of the target text data is copied to the eighth convolution layer of the extension path, and the noise image is sent to the first downsampling layer for downsampling processing, which is repeated three times until the noise image and the text semantic information of the target text data are completely separated. The text semantic information of the target text data is then input into the first convolution layer of the extension path, and the text semantic information of the copied target text data is upsampled and convolved in the extension path to finally output a complete animated image.

[0092] The embodiment of the present invention is applied to the multimodal artificial intelligence content generation scenario, by obtaining multiple cartoon image samples and multiple randomly initialized noise samples; inputting the multiple cartoon image samples and the multiple randomly initialized noise samples into the improved U-net network for noise training to obtain the trained U-net network; inputting the target text data of the cartoon image to be generated into the trained U-net network for denoising to generate the denoised cartoon image; the improved U-net network is based on the U-net network model, and all upsampling layers of the U-net network include convolution sublayers, squeezing sublayers, excitation sublayers, and fully connected sublayers; compared with the prior art In comparison, the embodiment of the present invention utilizes the compression path in the U-net network to denoise the target text data for generating the animation image, and then utilizes a convolution sublayer for extracting semantic information features of the target text data, a squeeze sublayer for compressing the extracted semantic information features, an excitation sublayer for obtaining the weights of the semantic information features, and a fully connected sublayer for fusing the semantic information features using the weights of the semantic information features to restore the target text data for generating the animation image into a clear animation image, thereby improving the sensitivity to feature extraction and accurately restoring an image with a high degree of match with the semantic information.

[0093] An embodiment of the present invention further provides an image generation device based on a denoising network, which is applied to a multimodal artificial intelligence content generation scenario, including:

[0094] An acquisition module, used to acquire multiple animation image samples and multiple randomly initialized noise samples;

[0095] A training module is used to input multiple cartoon image samples and multiple randomly initialized noise samples into the improved U-net network for noise training to obtain a trained U-net network;

[0096] A denoising module is used to input the target text data of the animated image to be generated into the trained U-net network for denoising, thereby generating a denoised animated image;

[0097] The improved U-net network is based on the U-net network model, and all upsampling layers of the U-net network include a convolution sublayer for extracting semantic information features of the target text data, a squeezing sublayer for compressing the extracted semantic information features, an excitation sublayer for obtaining the weights of the semantic information features, and a fully connected sublayer for fusing the semantic information features using the weights of the semantic information features.

[0098] The input end of the convolutional sublayer is the input end of the upsampling layer in the U-net network, which is used to connect to the output end of the network layer above the upsampling layer.

[0099] The output of the convolutional sublayer is connected to the input of the squeeze sublayer and the input of the fully connected sublayer respectively;

[0100] The output end of the squeezing sublayer is connected to the input end of the excitation sublayer;

[0101] The output of the excitation sublayer is connected to the input of the fully connected sublayer;

[0102] The output end of the fully connected sublayer is the output end of the upsampling layer in the U-net network, which is used to connect to the input end of the next level network layer of the upsampling layer.

[0103] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the embodiment of the present invention. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0104] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the embodiments of the present invention. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0105] An embodiment of the present invention further provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, a denoising network-based image generation method is implemented.

[0106] If the integrated module is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned method embodiments, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, it can implement the steps of the above-mentioned various method embodiments. The computer program includes computer program code, which can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device capable of carrying the computer program code to a construction device / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electric carrier signal, a telecommunication signal and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0107] An embodiment of the present invention also provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, an image generation method based on a denoising network is implemented.

[0108] The terminal device may be a computing device such as a desktop computer, a notebook computer, a PDA, a server, a server cluster, a cloud server, etc. The terminal device may include, but is not limited to, a processor and a memory.

[0109] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0110] In some embodiments, the memory may be an internal storage unit of the terminal device, such as a hard disk or memory of the terminal device. In other embodiments, the memory may also be an external storage device of the terminal device, such as a plug-in hard disk equipped on the terminal device, a smart memory card (SMC, Smart Media Card), a secure digital (SD, Secure Digital) card, a flash card, etc. Furthermore, the memory may include both an internal storage unit of the terminal device and an external storage device. The memory is used to store an operating system, an application program, a boot loader, data, and other programs, such as the program code of the computer program. The memory may also be used to temporarily store data that has been output or is about to be output.

[0111] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the embodiment of the present invention. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0112] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the embodiments of the present invention. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0113] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A denoising network-based image generation method, applied to multimodal artificial intelligence content generation scenarios, characterized by: include: Step 1: obtain multiple animation image samples and multiple randomly initialized noise samples; Step 2: Inputting the plurality of the cartoon image samples and the plurality of the randomly initialized noise samples into the improved U-net network for noise training to obtain a trained U-net network; Step 3: Input the target text data of the to-be-generated animated image into the trained U-net network for denoising to generate a denoised animated image; The improved U-net network is based on the U-net network model, and all upsampling layers of the U-net network include a convolution sublayer for extracting semantic information features of the target text data, a squeezing sublayer for compressing the extracted semantic information features, an excitation sublayer for obtaining weights of the semantic information features, and a fully connected sublayer for fusing the semantic information features using the weights of the semantic information features; The input end of the convolution sublayer is the input end of the upsampling layer in the U-net network and is connected to the output end of the upper-level network layer of the upsampling layer; The output end of the convolution sublayer is connected to the input end of the squeeze sublayer and the input end of the fully connected sublayer respectively; The output end of the extrusion sublayer is connected to the input end of the excitation sublayer; The output end of the excitation sublayer is connected to the input end of the fully connected sublayer; The output end of the fully connected sublayer is the output end of the upsampling layer in the U-net network and is used to connect to the input end of the next level network layer of the upsampling layer; The convolution sublayer is used to extract features from the semantic information output by the upper network layer of the upsampling layer, obtain a convolved feature map and input it into the fully connected sublayer and the squeeze sublayer; The squeezing sublayer is used to compress the convolved feature map into a channel descriptor and perform channel information statistics by global average pooling to obtain a one-dimensional feature vector; The excitation sublayer is used to learn the feature points of the one-dimensional feature vector and obtain feature weights using the feature points; The fully connected sublayer is used to fuse the feature weights with the convolved feature map to obtain a fused feature map and input it into the next level network layer of the upsampling layer.

2. The image generation method based on the denoising network according to claim 1, characterized in that The randomly initialized noise sample is a random noise matrix that conforms to a Gaussian distribution, and the spatial dimension of the random noise matrix is ​​consistent with the spatial dimension of the animation image.

3. The image generation method based on the denoising network according to claim 2, characterized in that: Inputting the plurality of the cartoon image samples and the plurality of the randomly initialized noise samples into the improved U-net network for noise training, comprising: For each of the animation image samples, performing noise processing of different degrees on the animation image sample using all the randomly initialized noise samples in turn to obtain multiple noise images; The improved U-net network is trained with noise using a plurality of the noise images.

4. The image generation method based on denoising network according to claim 3, characterized in that: The loss function in the noise training process for: ； in, represents the scoring function, represent positive samples and negative samples respectively, represents the set of sample pairs constructed using the scoring function, represents the sample vector of anime image, Represents the output comparison sample vector, Indicates adjustment parameters.

5. The image generation method based on denoising network according to claim 4, characterized in that: The adjustment parameters Determined by the sorting distance between the positive samples and the negative samples in the sample pair set; The adjustment parameters The expression is: ； in, represents the weight control coefficient, represents the ranking score of negative samples, represents the ranking score of the positive sample, Represents the ranking distance between sample pairs.

6. An image generation device based on a denoising network, characterized in that: Applied to multimodal AI content generation scenarios, including: An acquisition module, used to acquire multiple animation image samples and multiple randomly initialized noise samples; A training module, configured to input the plurality of the cartoon image samples and the plurality of the randomly initialized noise samples into an improved U-net network for noise training to obtain a trained U-net network; A denoising module is used to input the target text data of the cartoon image to be generated into the trained U-net network for denoising, thereby generating a denoised cartoon image; The improved U-net network is based on the U-net network model, and all upsampling layers of the U-net network include a convolution sublayer for extracting semantic information features of the target text data, a squeezing sublayer for compressing the extracted semantic information features, an excitation sublayer for obtaining weights of the semantic information features, and a fully connected sublayer for fusing the semantic information features using the weights of the semantic information features; The input end of the convolution sublayer is the input end of the upsampling layer in the U-net network and is connected to the output end of the upper-level network layer of the upsampling layer; The output end of the convolution sublayer is connected to the input end of the squeeze sublayer and the input end of the fully connected sublayer respectively; The output end of the extrusion sublayer is connected to the input end of the excitation sublayer; The output end of the excitation sublayer is connected to the input end of the fully connected sublayer; The output end of the fully connected sublayer is the output end of the upsampling layer in the U-net network and is used to connect to the input end of the next level network layer of the upsampling layer; The convolution sublayer is used to extract features from the semantic information output by the upper network layer of the upsampling layer, obtain a convolved feature map and input it into the fully connected sublayer and the squeeze sublayer; The squeezing sublayer is used to compress the convolved feature map into a channel descriptor and perform channel information statistics by global average pooling to obtain a one-dimensional feature vector; The excitation sublayer is used to learn the feature points of the one-dimensional feature vector and obtain feature weights using the feature points; The fully connected sublayer is used to fuse the feature weights with the convolved feature map to obtain a fused feature map and input it into the next level network layer of the upsampling layer.

7. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the image generation method based on the denoising network according to any one of claims 1 to 5 is implemented.

8. A terminal device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the image generation method based on the denoising network according to any one of claims 1 to 5 is implemented.

Images