Image processing method and device, equipment, storage medium and product

Abstract

The embodiment of the invention discloses an image processing method and device, equipment, a storage medium and a product. The method comprises the following steps: obtaining a generated image of a target part containing a target object, carrying out noise addition processing on the generated image to obtain a sample noise image, fusing a subspace noise image corresponding to the generated image with the sample noise image to obtain a composite noise image, and outputting the composite noise image. The composite noise image contains noise of areas except the area to which the target part in the sample noise image belongs, and performing iterative noise reduction processing on the composite noise image based on the diffusion parameter and M pieces of second prompt information (the prompt words of the target part are different) to obtain M images of the target object. Therefore, local adjustment can be stably performed on the generated image by multiplexing diffusion parameters, cue words irrelevant to the target part and noise of areas except the area to which the target part belongs in the sample noise image.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an image processing method, an image processing apparatus, a computer device, a computer-readable storage medium, and an image processing product. Background Art

[0002] With the progress of scientific and technological research, the technology of Vincent graphs has developed rapidly. Vincent graph technology refers to the technology of generating corresponding images through prompt information. Vincent graph technology is based on the diffusion model, and different images can be generated based on different prompt information. Research has found that in the process of adjusting the generated image, it is difficult to stably adjust the generated image locally; for example, after adjusting the expression prompt words in the prompt information of the first image, the second image is generated based on the adjusted prompt information. In addition to the expression of the object in the second image being different from that in the first image, other elements of the object (such as hairstyle, gender, etc.) are also different from those in the first image. Summary of the invention

[0003] The embodiments of the present application provide an image processing method, apparatus, device, computer-readable storage medium, and product, which can stably perform local adjustments on a generated image.

[0004] On the one hand, an embodiment of the present application provides an image processing method, including:

[0005] Acquire a generated image, where the generated image is obtained by iteratively denoising the latent space noise image based on the diffusion parameter and the first prompt information, the first prompt information does not include a prompt word related to the target part, and the generated image includes the target part of the target object;

[0006] Perform noise processing on the generated image to obtain a sample noise image;

[0007] The latent space noise image is fused with the sample noise image to obtain a composite noise image; the composite noise image contains the noise of the area other than the area to which the target part belongs in the sample noise image;

[0008] Based on the diffusion parameter and M second prompt information, the composite noise image is iteratively denoised to obtain M images of the target object; the visual elements of each image except the target part are the same, and any second prompt information is obtained by adding the prompt word of the target part to the first prompt information, and M is a positive integer.

[0009] On the one hand, an embodiment of the present application provides an image processing device, the image processing device comprising:

[0010] An acquisition unit is used to acquire a generated image, where the generated image is obtained by iteratively denoising the latent space noise image based on the diffusion parameter and the first prompt information, the first prompt information does not include a prompt word related to the target part, and the generated image includes the target part of the target object;

[0011] A processing unit, used for performing noise processing on the generated image to obtain a sample noise image;

[0012] and used to fuse the latent space noise image with the sample noise image to obtain a composite noise image; the composite noise image includes the noise of the area other than the area to which the target part belongs in the sample noise image;

[0013] And it is used to iteratively reduce the noise of the composite noise image based on the diffusion parameter and M second prompt information to obtain M images of the target object; the visual elements except the target part are the same between the images, and any second prompt information is obtained by adding the prompt word of the target part to the first prompt information, and M is a positive integer.

[0014] In one implementation, the processing unit is used to fuse the latent space noise image with the sample noise image to obtain a composite noise image, specifically for:

[0015] Extract the noise of the target area in the latent space noise image;

[0016] The noise in the target area of ​​the latent space noise image is fused with the sample noise image to obtain a composite noise image.

[0017] In one embodiment, the processing unit is used to fuse the noise of the target area in the latent space noise image with the sample noise image to obtain a composite noise image, specifically for:

[0018] Extract the noise of the area other than the area where the target part belongs in the sample noise image;

[0019] The noise of the target area in the latent space noise image is fused with the noise of the area other than the target area in the sample noise image to obtain a composite noise image.

[0020] In one implementation, the processing unit is used to extract the noise in the region to which the target part belongs in the latent space noise image, specifically to:

[0021] Perform image segmentation processing on the generated image to obtain the area to which the target part in the generated image belongs;

[0022] Generate a first mask based on the region to which the target part belongs in the generated image, the first mask being used to cover the region other than the region to which the target part belongs in the generated image;

[0023] The noise in the region to which the target part belongs in the latent space noise image is extracted through the first mask.

[0024] In one implementation, the processing unit is used to perform noise processing on the generated image to obtain a sample noise image, specifically for:

[0025] Perform high-frequency information weakening processing on the generated image to obtain a blurred image of the generated image;

[0026] The blurred image is subjected to noise processing to obtain a sample noise image.

[0027] In one implementation, the processing unit is used to perform noise processing on the blurred image to obtain a sample noise image, specifically for:

[0028] Call the diffusion model to iteratively add noise to the blurred image according to the diffusion parameters to obtain a sample noise image;

[0029] The input data for the first noise addition is a blurred image, the input data for the jth noise addition is the j-1th noise addition result, and j is an integer greater than 1.

[0030] In one implementation, the processing unit is used to call the diffusion model to perform iterative noise processing on the blurred image according to the diffusion parameters to obtain a sample noise image, specifically for:

[0031] The control network is loaded into the diffusion model to obtain an updated diffusion model; the control network is used to improve the clarity of the input image;

[0032] The updated diffusion model is called to iteratively add noise to the blurred image according to the diffusion parameters to obtain a sample noise image.

[0033] In one implementation, the processing unit is used to perform high-frequency information weakening processing on the generated image to obtain a blurred image of the generated image, specifically for:

[0034] Performing filtering on the generated image to obtain a blurred image of the generated image; or,

[0035] The generated image is transformed into a frequency domain to obtain a frequency domain representation of the generated image, the spectrum information of the high-frequency part of the generated image is removed from the frequency domain representation to obtain an updated frequency domain representation, and the updated frequency domain representation is inversely transformed into a frequency domain to obtain a blurred image of the generated image.

[0036] In one implementation, the processing unit performs iterative noise reduction processing on the composite noise image based on the diffusion parameter and the M second prompt information to obtain the M images of the target object, including:

[0037] Add the prompt words of the i-th group of target parts to the first prompt information to obtain the i-th second prompt information, where i is a positive integer less than or equal to M;

[0038] The diffusion model is called to perform iterative denoising on the composite noise image according to the diffusion parameters and the i-th second prompt information to obtain the i-th image of the target object.

[0039] In one embodiment, the target part is the face, and the prompt word of the target part is an expression prompt word; the processing unit is further used for:

[0040] Packing the M images of the target object to obtain an expression package of the target object;

[0041] In response to the expression package of the target object being triggered, at least one expression image of the target object is displayed in the expression display area.

[0042] In one embodiment, the processing unit is used to obtain a generated image, specifically to:

[0043] Obtain diffusion parameters, latent space noise image and first prompt information;

[0044] The diffusion model is called to perform iterative denoising on the latent space noise image according to the diffusion parameter and the first prompt information to obtain a generated image.

[0045] In one embodiment, the diffusion parameter includes at least one of the following: a sampler identifier, a noise loading schedule, a random seed, and a sampling step number.

[0046] Accordingly, the present application provides a computer device, the computer device comprising:

[0047] a memory, wherein a computer program is stored in the memory;

[0048] The processor is used to load a computer program to implement the above-mentioned image processing method.

[0049] Accordingly, the present application provides a computer-readable storage medium, which stores a computer program, and the computer program is suitable for being loaded by a processor and executing the above-mentioned image processing method.

[0050] Accordingly, the present application provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the above-mentioned image processing method.

[0051] In the embodiment of the present application, a generated image containing a target part of a target object is obtained, and a noise process is performed on the generated image to obtain a sample noise image. The latent space noise image corresponding to the generated image is fused with the sample noise image to obtain a composite noise image, wherein the composite noise image contains noise in areas other than the area to which the target part belongs in the sample noise image. Based on diffusion parameters and M second prompt information (the prompt words of the target part are different), the composite noise image is iteratively denoised to obtain M images of the target object. It can be seen that by reusing diffusion parameters, prompt words unrelated to the target part, and noise in areas other than the area to which the target part belongs in the sample noise image, the generated image can be locally adjusted stably. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0053] Figure 1a A schematic diagram of a diffusion process provided in an embodiment of the present application;

[0054] Figure 1b An image processing scene graph provided in an embodiment of the present application;

[0055] Figure 2 A flowchart of an image processing method provided in an embodiment of the present application;

[0056] Figure 3 A flowchart of another image processing method provided in an embodiment of the present application;

[0057] Figure 4a A schematic diagram of generating an image provided in an embodiment of the present application;

[0058] Figure 4b A schematic diagram of a high-frequency information weakening process provided in an embodiment of the present application;

[0059] Figure 4c A schematic diagram of loading a control network into a diffusion model provided in an embodiment of the present application;

[0060] Figure 4d A schematic diagram of a sample noise image generation process provided in an embodiment of the present application;

[0061] Figure 4e A schematic diagram of generating a first mask and a second mask provided in an embodiment of the present application;

[0062] Figure 4f A schematic diagram of a fusion process of a composite noise image provided in an embodiment of the present application;

[0063] Figure 4g A schematic diagram of an image generation process provided in an embodiment of the present application;

[0064] Figure 4h A schematic diagram of an expression image provided in an embodiment of the present application;

[0065] Figure 4i A schematic diagram of a page display provided in an embodiment of the present application;

[0066] Figure 5 A schematic diagram of the structure of an image processing device provided in an embodiment of the present application;

[0067] Figure 6 A schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0068] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0069] The present application embodiment relates to technologies related to a diffusion model. The diffusion model is briefly introduced below:

[0070] Diffusion model: The diffusion model is inspired by the diffusion process in physics, which is how an ordered structure is gradually transformed into a disordered state and then reversed. Figure 1a A diffusion process schematic diagram provided in an embodiment of the present application. Figure 1a As shown in Figure 2, the working principle of the diffusion model can be divided into two main stages: forward process and reverse process.

[0071] Forward Process: This stage simulates the diffusion process from data distribution to random noise distribution. At the beginning, there is a clean data sample (such as an image) x0, and then noise is gradually added to the sample, after multiple steps, until the sample becomes completely noise. This process is usually implemented by applying Gaussian noise, and each step will make the sample closer to a pure noise distribution.

[0072] Reverse Process: The reverse process is the inverse process of the forward process, and its goal is to recover the original data from the noisy data.

[0073] Based on the above-mentioned technology related to the diffusion model, the embodiment of the present application provides an image processing solution, which can stably perform local adjustments on the generated image. Figure 1b An image processing scene graph provided in an embodiment of the present application, such as Figure 1b As shown, the image processing scenario provided by the present application includes a terminal device 101 and a server 102. The image processing solution provided by the present application can be executed by the terminal device 101 or the server 102; when the image processing solution provided by the present application is executed by the terminal device 101, the server 102 may not be included in the image processing scenario. Among them, the terminal device may include but is not limited to: smart phones (such as Android phones, IOS phones, etc.), tablet computers, portable personal computers, mobile Internet devices (Mobile Internet Devices, MID), intelligent voice interaction devices, smart home appliances, vehicle terminals, aircraft, wearable devices, etc., and the embodiments of the present application do not limit this; the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and basic cloud computing services such as big data and artificial intelligence platforms, and the embodiments of the present application do not limit this.

[0074] It should be noted that Figure 1b The number of terminal devices and servers is only used as an example and does not constitute an actual limitation of the present application. The terminal device 101 and the server 102 can be connected via wired or wireless means, and the present application does not limit this.

[0075] The general process of the image processing solution provided in this application is as follows:

[0076] (1) The server 102 obtains a generated image. The generated image is obtained by iteratively denoising the latent space noise image based on the diffusion parameter and the first prompt information. For the specific process, please refer to Figure 1aThe reverse process in will not be repeated here. The first prompt information does not include a prompt word related to the target part, and the generated image includes the target part of the target object. The target object may include but is not limited to a person, a virtual character, a cartoon animal, etc., and the target part may include but is not limited to a face, a hand, hair, a foot, a leg, etc. In one embodiment, the server 102 obtains the diffusion parameters, the latent space noise image and the first prompt information, and then calls the diffusion model to iteratively denoise the latent space noise image according to the diffusion parameters and the first prompt information to obtain the generated image.

[0077] (2) The server 102 performs noise processing on the generated image to obtain a sample noise image. There are at least the following differences between the sample noise image and the latent space noise image: the sample noise image carries the first prompt information, which can be understood as the style of the sample noise image being affected by the first prompt information (the style of the latent space noise image is not affected by the first prompt information). In one embodiment, the server 102 performs high-frequency information weakening processing on the generated image to obtain a blurred image of the generated image. High-frequency information may include but is not limited to pixel details such as texture, light and shadow. High-frequency information weakening processing can be achieved by filtering or frequency domain conversion. After obtaining the blurred image, the server 102 performs noise processing on the blurred image to obtain a sample noise image. In another embodiment, the server 102 can directly perform noise processing on the generated image to obtain a sample noise image; for example, the server 102 can call a diffusion model to iteratively perform noise processing on the generated image according to the diffusion parameters to obtain a sample noise image.

[0078] (3) The server 102 fuses the latent space noise image with the sample noise image to obtain a composite noise image. The composite noise image contains the noise of the area other than the area to which the target part belongs in the sample noise image. In one embodiment, the server 102 extracts the noise of the area to which the target part belongs in the latent space noise image, and fuses the noise of the area to which the target part belongs in the latent space noise image with the sample noise image to obtain a composite noise image. In one embodiment, the server 102 extracts the noise of the area other than the area to which the target part belongs in the sample noise image, and fuses the noise of the area to which the target part belongs in the latent space noise image with the noise of the area other than the area to which the target part belongs in the sample noise image to obtain a composite noise image. In another embodiment, the server 102 fuses the noise of the area to which the target part belongs in the latent space noise image with the entire sample noise image to obtain a composite noise image. In this case, the noise of the area other than the area to which the target part belongs in the composite noise image is the noise in the sample noise image, and the noise of the area to which the target part belongs is obtained by fusion of the noise in the sample noise image and the noise in the area to which the target part belongs in the latent space noise image.

[0079] (4) The server 102 performs iterative noise reduction processing on the composite noise image based on the diffusion parameter and the M second prompt information to obtain M images of the target object; wherein, the visual elements other than the target part are the same between the images, and any second prompt information is obtained by adding the prompt word of the target part to the first prompt information, and M is a positive integer. For example, the target part is the face, and the prompt word of the target part is the expression prompt word; that is, M groups of different expression prompt words are added to the first prompt information to obtain M second prompt information. The server 102 calls the diffusion model to perform iterative noise reduction processing on the composite noise image according to the diffusion parameter and the i-th second prompt information to obtain the i-th expression image of the target object. For the obtained M expression images, except for the face area, other visual elements (such as background, hairstyle, etc.) are the same, and the expressions of the target object in the face area are different, but the facial identity features of the target object are the same. The facial identity features are facial features (such as facial features, etc.) used to indicate the identity of the target object.

[0080] In the embodiment of the present application, a generated image containing a target part of a target object is obtained, and a noise process is performed on the generated image to obtain a sample noise image. The latent space noise image corresponding to the generated image is fused with the sample noise image to obtain a composite noise image, wherein the composite noise image contains noise in areas other than the area to which the target part belongs in the sample noise image. Based on diffusion parameters and M second prompt information (the prompt words of the target part are different), the composite noise image is iteratively denoised to obtain M images of the target object. It can be seen that by reusing diffusion parameters, prompt words unrelated to the target part, and noise in areas other than the area to which the target part belongs in the sample noise image, the generated image can be locally adjusted stably.

[0081] Based on the above-mentioned image processing scheme, the embodiment of the present application proposes a more detailed image processing method. The image processing method proposed in the embodiment of the present application will be described in detail below in conjunction with the accompanying drawings.

[0082] See also Figure 2 , Figure 2 A flowchart of an image processing method provided in an embodiment of the present application, the image processing method can be executed by a computer device; for example, Figure 1b The terminal device 101 shown in FIG. 1 is executed, or the terminal device 101 is executed Figure 1b The server 102 shown in FIG. Figure 2 As shown, the image processing method may include the following steps S201 to S204:

[0083] S201, obtaining a generated image.

[0084] The generated image is obtained by iteratively denoising the latent space noise image based on the diffusion parameter and the first prompt information; wherein the generated image includes the target part of the target object, and the first prompt information does not include prompt words related to the target part; for example, the target part is the face, and prompt words related to the face include face shape, expression, etc. The target part is the part that needs to be locally adjusted. The target object may include but is not limited to a person, a virtual character, a cartoon animal, etc., and the target part may include but is not limited to the face, hands, hair, feet, legs, etc. The image processing method provided in this application will be described below mainly using the target part being the face as an example.

[0085] In one embodiment, a computer device obtains diffusion parameters, first prompt information, and a latent space noise image; wherein the latent space noise image may be initialized according to the diffusion parameters, and the diffusion parameters include at least one of the following: a sampler identifier, a noise loading schedule, a random seed, and a sampling step number. The sampler identifier is used to indicate the sampler used in the diffusion model, the noise loading schedule indicates the specific implementation process of iterative noise addition processing and iterative noise reduction processing, the random seed is used to indicate the starting condition, and the sampling step number is used to indicate the number of iterations.

[0086] After obtaining the diffusion parameters, the first prompt information and the latent space noise image, the computer device calls the diffusion model to perform iterative noise reduction processing on the latent space noise image according to the diffusion parameters and the first prompt information to obtain a generated image.

[0087] S202: Perform noise processing on the generated image to obtain a sample noise image.

[0088] The difference between the sample noise image and the latent space noise image is that the sample noise image carries the first prompt information, while the latent space noise image does not carry the first prompt information. It can be understood that the style of the sample noise image is affected by the first prompt information, while the style of the latent space noise image is not affected by the first prompt information.

[0089] In one embodiment, the computer device performs high-frequency information weakening processing on the generated image to obtain a blurred image of the generated image. The high-frequency information weakening processing is used to reduce or remove high-frequency information in the generated image, and the high-frequency information may include but is not limited to pixel details such as texture, light and shadow. The high-frequency information weakening processing can be achieved by filtering or frequency domain conversion.

[0090] In one embodiment, the computer device performs filtering processing on the generated image (ie, filters high-frequency information in the generated image) through a filter (such as a Gaussian filter, etc.) to obtain a blurred image of the generated image.

[0091] In another embodiment, the computer device performs a frequency domain conversion (such as Fourier transform) on the generated image to obtain a frequency domain representation of the generated image, and the frequency domain representation includes the spectrum information of the high frequency part and the spectrum information of the low frequency part of the generated image. The computer device removes the spectrum information of the high frequency part from the frequency domain representation to obtain an updated frequency domain representation. Further, the computer device performs an inverse frequency domain conversion (such as inverse Fourier transform) on the updated frequency domain representation to obtain a blurred image of the generated image.

[0092] It should be noted that by weakening the high-frequency information of the generated image, the edges in the generated image can be softened (blurred), the pixel details in the generated image can be reduced, and more degrees of freedom can be provided for subsequent operations (that is, richer and more diversified results can be obtained).

[0093] After obtaining the blurred image, the computer device performs noise processing on the blurred image to obtain a sample noise image. In one embodiment, the computer device calls the diffusion model to perform iterative noise processing on the blurred image according to the diffusion parameter (the number of iterations is indicated by the diffusion parameter) to obtain a sample noise image; wherein the input data for the first noise addition is the blurred image, the input data for the jth noise addition is the j-1th noise addition result, and j is an integer greater than 1 and less than or equal to the number of iterations.

[0094] In another embodiment, the computer device may directly perform noise processing on the generated image according to the diffusion parameters to obtain a sample noise image; for example, the computer device may call a diffusion model to iteratively perform noise processing on the generated image according to the diffusion parameters to obtain a sample noise image.

[0095] S203 , fusing the latent space noise image with the sample noise image to obtain a composite noise image.

[0096] The composite noise image includes noise in areas other than the area where the target part belongs in the sample noise image. Taking the target part as a face as an example, the composite noise image includes noise in non-face areas in the sample noise image.

[0097] In one embodiment, the computer device extracts the noise of the target area in the latent space noise image. In one embodiment, the computer device performs image segmentation processing on the generated image to obtain the target area in the generated image, and generates a first mask based on the target area in the generated image; wherein the first mask is used to cover the area in the generated image except the target area. Then, the computer device extracts the noise of the target area in the latent space noise image through the first mask.

[0098] After obtaining the noise of the target area in the latent space noise image, the computer device fuses the noise of the target area in the latent space noise image with the sample noise image to obtain a composite noise image. In other words, the noise in the latent space noise image only affects the noise in the target area in the composite noise image; the noise in the area other than the target area in the composite noise image is the noise in the area other than the target area in the sample noise image.

[0099] Similar methods also include: extracting the noise in the area where the target part belongs in the sample noise image, fusing the latent space noise image with the sample noise image to obtain a composite noise image, and replacing the noise in the area other than the area where the target part belongs in the composite noise image with the noise in the area other than the area where the target part belongs in the sample noise image to obtain an updated composite noise image.

[0100] In one embodiment, after extracting the noise of the area to which the target part belongs in the latent space noise image, the computer device can further extract the noise of the area other than the area to which the target part belongs in the sample noise image, and fuse the noise of the area to which the target part belongs in the latent space noise image with the noise of the area other than the area to which the target part belongs in the sample noise image to obtain a composite noise image.

[0101] In another embodiment, the computer device fuses the noise of the target area in the latent space noise image with the entire sample noise image to obtain a composite noise image. In this case, the noise of the target area is obtained by fusing the noise in the sample noise image with the noise of the target area in the latent space noise image.

[0102] S204 , performing iterative noise reduction processing on the composite noise image based on the diffusion parameter and the M second prompt information to obtain M images of the target object.

[0103] The visual elements of each image except the target part are the same; for example, if the target part is the face, and the prompt word of the target part is an expression prompt word, then the M images are expression images, and the M expression images have the same visual elements (such as background, hairstyle, etc.) except the face area, and the expressions of the target objects in the face area are different, but the facial identity features of the target objects are the same, and the facial identity features are facial features (such as facial features, etc.) used to indicate the identity of the target objects. For another example, if the target part is the hair of the target object, and the prompt word of the target part is a color prompt word, then the M images are images of the target object with hair of different colors, and the M images have the same visual elements (such as background, face, hairstyle, etc.) except the hair color.

[0104] Any second prompt information is obtained by adding the prompt words of the target part to the first prompt information, and M is a positive integer; wherein the i-th second prompt information is obtained by adding the prompt words of the i-th group of target parts to the first prompt information.

[0105] In one embodiment, the computer device performs iterative noise reduction processing on the composite noise image based on the diffusion parameter and the M second prompt information to obtain the M images of the target object, including: the computer device adds the prompt words of the i-th group of target parts to the first prompt information to obtain the i-th second prompt information, where i is a positive integer less than or equal to M. After obtaining the i-th second prompt information, the computer device calls the diffusion model to perform iterative noise reduction processing on the composite noise image according to the diffusion parameter and the i-th second prompt information to obtain the i-th image of the target object.

[0106] In the embodiment of the present application, a generated image containing a target part of a target object is obtained, and a noise process is performed on the generated image to obtain a sample noise image. The latent space noise image corresponding to the generated image is fused with the sample noise image to obtain a composite noise image, wherein the composite noise image contains noise in areas other than the area to which the target part belongs in the sample noise image. Based on diffusion parameters and M second prompt information (the prompt words of the target part are different), the composite noise image is iteratively denoised to obtain M images of the target object. It can be seen that by reusing diffusion parameters, prompt words unrelated to the target part, and noise in areas other than the area to which the target part belongs in the sample noise image, the generated image can be locally adjusted stably.

[0107] See also Figure 3 , Figure 3 A flowchart of another image processing method provided in an embodiment of the present application, the image processing method can be executed by a computer device; for example, Figure 1b The terminal device 101 shown in FIG. 1 is executed, or the terminal device 101 is executed Figure 1b The server 102 shown in FIG. Figure 3 As shown, the image processing method may include the following steps S301 to S307:

[0108] S301, obtaining a generated image.

[0109] Figure 4a A schematic diagram of generating an image provided in an embodiment of the present application. Figure 4a As shown, the diffusion model is called to perform iterative denoising on the latent space noise image ε according to the diffusion parameters (carried in the diffusion model) and the first prompt information, so as to obtain the generated image x0.

[0110] S302: performing high-frequency information weakening processing on the generated image to obtain a blurred image of the generated image.

[0111] Figure 4b A schematic diagram of a high-frequency information weakening process provided in an embodiment of the present application. Figure 4b As shown, compared with the generated image x0, the blurred image has softer (blurred) edges and reduced pixel details, which can provide more freedom for subsequent operations (i.e., richer and more diversified results can be obtained).

[0112] S303 , calling the diffusion model to perform iterative noise addition processing on the blurred image according to the diffusion parameters to obtain a sample noise image.

[0113] In one embodiment, a computer device loads a control network (such as a tile control network) into a diffusion model to obtain an updated diffusion model; wherein the control network is used to improve the clarity of an input image. In one embodiment, the diffusion model includes at least one encoder block and a middle block, and the computer device loads the control network into each encoder block and the middle block, respectively, to obtain an updated diffusion model. It can be understood that, compared to a diffusion model without a control network loaded, a diffusion model loaded with a control network can generate an image with higher clarity, further realizing refined control of the image.

[0114] Figure 4c A schematic diagram of loading a control network into a diffusion model is provided in an embodiment of the present application. Figure 4c As shown, the diffusion model includes encoder block A-encoder block D, intermediate block, and decoder block A-decoder block D, and encoder block A-encoder block D and decoder block A-decoder block D correspond to each other. The computer device loads the control network into encoder block A-encoder block D and intermediate block respectively, and the updated diffusion model can be obtained. It should be noted that Figure 4c The number of encoder blocks and decoder blocks included in the diffusion model is only for example and does not constitute an actual limitation of the present application.

[0115] After obtaining the updated diffusion model, the computer device calls the updated diffusion model to iteratively add noise to the blurred image according to the diffusion parameters to obtain a sample noise image. By adding a control network to the diffusion model, the updated diffusion model can have the function of improving the image clarity, thereby obtaining a sample noise image with higher clarity.

[0116] Figure 4d A schematic diagram of a sample noise image generation process provided in an embodiment of the present application. Figure 4d As shown, by calling the updated diffusion model to iteratively add noise to the blurred image according to the diffusion parameters, a sample noise image can be obtained.

[0117] S304 , fusing the noise of the area other than the area to which the target part belongs in the latent space noise image with the noise of the area to which the target part belongs in the sample noise image to obtain a composite noise image.

[0118] The computer device (such as through the yolo model, the Grounding Dino model, etc.) performs image segmentation processing on the generated image to obtain the area to which the target part belongs in the generated image (i.e., the area corresponding to the target part in the generated image), or obtains the area other than the area to which the target part belongs in the generated image. Further, the computer device generates a first mask and a second mask through the area to which the target part belongs in the generated image (or through the area other than the area to which the target part belongs in the generated image). The first mask is used to cover the area other than the area to which the target part belongs in the generated image, and the second mask is used to cover the area to which the target part belongs in the generated image. Figure 4e A schematic diagram of generating a first mask and a second mask is provided in an embodiment of the present application. Figure 4e As shown, the target part is the face, and the generated image x0 is subjected to image segmentation processing, and the first mask and the second mask can be obtained based on the image segmentation result; wherein the first mask is used to cover the non-face area (i.e., the area other than the area where the target part belongs), and the second mask is used to cover the face area (i.e., the area where the target part belongs).

[0119] After obtaining the first mask and the second mask, the computer device extracts the noise of the target area in the latent space noise image through the first mask, and extracts the noise of the area other than the target area in the sample noise image through the second mask. Then the computer device fuses the noise of the target area in the latent space noise image with the noise of the area other than the target area in the sample noise image to obtain a composite noise image.

[0120] Figure 4f A schematic diagram of a fusion process of a composite noise image provided in an embodiment of the present application is shown in FIG. Figure 4f As shown in the figure, the target part is the face. By masking the latent space noise image with the first mask, the noise in the face area in the latent space noise image can be obtained. By masking the sample noise image with the second mask, the noise in the non-face area in the sample noise image can be obtained. Then, the noise in the face area in the latent space noise image and the noise in the non-face area in the sample noise image are fused to obtain a composite noise image.

[0121] S305 . Perform iterative noise reduction processing on the composite noise image based on the diffusion parameter and the M second prompt information to obtain M images of the target object.

[0122] Figure 4gA schematic diagram of an image generation process provided in an embodiment of the present application. Figure 4g As shown, the target part is the face, and the expression prompt word in the second prompt information is "smile". Based on the second prompt information, the composite noise image is subjected to iterative noise reduction processing, and an expression image in which the expression of the target object is smiling can be obtained. Similarly, by adding different expression prompt words (such as sadness, surprise, etc.) to the first prompt information, different second prompt information can be obtained, and then different expression images can be obtained.

[0123] Figure 4h This is a schematic diagram of an expression image provided in an embodiment of the present application. Figure 4h As shown, the other visual elements (such as background, hairstyle, etc.) of each expression image except the facial area are the same, the expressions of the target objects in the facial area are different, but the facial identity features of the target objects are the same. The facial identity features are facial features (such as facial features, etc.) used to indicate the identity of the target object.

[0124] S306 , packing the M images of the target object to obtain an expression package of the target object.

[0125] In one implementation, the target part is the face, the prompt word for the target part is an expression prompt word, and the computer device adds M images of the target object to the same set to obtain an expression package of the target object.

[0126] S307: In response to the expression package of the target object being triggered, display at least one expression image of the target object in the expression display area.

[0127] Figure 4i A schematic diagram of a page display provided in an embodiment of the present application. Figure 4i As shown, in the conversation page 401, in response to the expression package triggering mark 402 of the target object being triggered, at least one expression image of the target object is displayed in the expression display area 403. When there are multiple expression images, they can be sorted and displayed based on preset conditions (such as the frequency of use of the expression images). Further, when any expression image in the expression package is triggered, the triggered expression image is displayed in the conversation box 404.

[0128] In the embodiment of the present application, a generated image of a target part including a target object is obtained, the generated image is subjected to noise processing to obtain a sample noise image, the latent space noise image corresponding to the generated image is fused with the sample noise image to obtain a composite noise image, the composite noise image contains the noise of the sample noise image in the area other than the area to which the target part belongs, based on the diffusion parameter and M second prompt information (the prompt words of the target part are different), the composite noise image is subjected to iterative noise reduction processing to obtain M images of the target object. It can be seen that by reusing the diffusion parameter, the prompt words irrelevant to the target part, and the noise of the area other than the area to which the sample noise image belongs to the target part, the generated image can be stably adjusted locally. In addition, by weakening the high-frequency information of the generated image, the edge in the generated image can be softened (blurred), the pixel details in the generated image can be reduced, and more degrees of freedom can be provided for subsequent operations; by adding a control network to the diffusion model, the updated diffusion model can have the function of improving the image clarity, thereby obtaining a sample noise image with higher clarity. The image processing method provided in the present application can use the generated image as a sample (reference) image without providing an additional sample image, and local adjustment of the generated image can be achieved by modifying the prompt word of the target part, which is more convenient.

[0129] The method of the embodiment of the present application is described in detail above. In order to facilitate better implementation of the above scheme of the embodiment of the present application, the device of the embodiment of the present application is provided below accordingly.

[0130] See also Figure 5 , Figure 5 A schematic diagram of the structure of an image processing device provided in an embodiment of the present application is shown in FIG. Figure 5 The image processing apparatus shown may be mounted in a computer device, which may specifically be a terminal device or a server. Figure 5 The image processing apparatus shown can be used to perform the above Figure 2 and Figure 3 Some or all of the functions of the described method embodiments. Figure 5 , the image processing device comprises:

[0131] An acquisition unit 501 is used to acquire a generated image, where the generated image is obtained by iteratively denoising the latent space noise image based on the diffusion parameter and the first prompt information, the first prompt information does not include a prompt word related to the target part, and the generated image includes the target part of the target object;

[0132] A processing unit, used for performing noise processing on the generated image to obtain a sample noise image;

[0133] and used to fuse the latent space noise image with the sample noise image to obtain a composite noise image; the composite noise image includes the noise of the area other than the area to which the target part belongs in the sample noise image;

[0134] And it is used to iteratively reduce the noise of the composite noise image based on the diffusion parameter and M second prompt information to obtain M images of the target object; the visual elements except the target part are the same between the images, and any second prompt information is obtained by adding the prompt word of the target part to the first prompt information, and M is a positive integer.

[0135] In one implementation, the processing unit 502 is used to fuse the latent space noise image with the sample noise image to obtain a composite noise image, specifically for:

[0136] Extract the noise of the target area in the latent space noise image;

[0137] The noise in the target area of ​​the latent space noise image is fused with the sample noise image to obtain a composite noise image.

[0138] In one implementation, the processing unit 502 is used to fuse the noise of the target area in the latent space noise image with the sample noise image to obtain a composite noise image, specifically for:

[0139] Extract the noise of the area other than the area where the target part belongs in the sample noise image;

[0140] The noise of the target area in the latent space noise image is fused with the noise of the area other than the target area in the sample noise image to obtain a composite noise image.

[0141] In one implementation, the processing unit 502 is used to extract the noise in the region to which the target part belongs in the latent space noise image, specifically to:

[0142] Perform image segmentation processing on the generated image to obtain the area to which the target part in the generated image belongs;

[0143] Generate a first mask based on the region to which the target part belongs in the generated image, the first mask being used to cover the region other than the region to which the target part belongs in the generated image;

[0144] The noise in the region to which the target part belongs in the latent space noise image is extracted through the first mask.

[0145] In one implementation, the processing unit 502 is used to perform noise processing on the generated image to obtain a sample noise image, specifically for:

[0146] Perform high-frequency information weakening processing on the generated image to obtain a blurred image of the generated image;

[0147] The blurred image is subjected to noise processing to obtain a sample noise image.

[0148] In one implementation, the processing unit 502 is used to perform noise processing on the blurred image to obtain a sample noise image, specifically for:

[0149] Call the diffusion model to iteratively add noise to the blurred image according to the diffusion parameters to obtain a sample noise image;

[0150] The input data for the first noise addition is a blurred image, the input data for the jth noise addition is the j-1th noise addition result, and j is an integer greater than 1.

[0151] In one implementation, the processing unit 502 is used to call the diffusion model to perform iterative noise processing on the blurred image according to the diffusion parameters to obtain a sample noise image, specifically for:

[0152] The control network is loaded into the diffusion model to obtain an updated diffusion model; the control network is used to improve the clarity of the input image;

[0153] The updated diffusion model is called to iteratively add noise to the blurred image according to the diffusion parameters to obtain a sample noise image.

[0154] In one implementation, the processing unit 502 is used to perform high-frequency information weakening processing on the generated image to obtain a blurred image of the generated image, specifically for:

[0155] Performing filtering on the generated image to obtain a blurred image of the generated image; or,

[0156] The generated image is transformed into a frequency domain to obtain a frequency domain representation of the generated image, the spectrum information of the high-frequency part of the generated image is removed from the frequency domain representation to obtain an updated frequency domain representation, and the updated frequency domain representation is inversely transformed into a frequency domain to obtain a blurred image of the generated image.

[0157] In one implementation, the processing unit 502 performs iterative noise reduction processing on the composite noise image based on the diffusion parameter and the M second prompt information to obtain the M images of the target object, including:

[0158] Add the prompt words of the i-th group of target parts to the first prompt information to obtain the i-th second prompt information, where i is a positive integer less than or equal to M;

[0159] The diffusion model is called to perform iterative denoising on the composite noise image according to the diffusion parameters and the i-th second prompt information to obtain the i-th image of the target object.

[0160] In one implementation, the target part is the face, and the prompt word for the target part is an expression prompt word; the processing unit 502 is further configured to:

[0161] Packing the M images of the target object to obtain an expression package of the target object;

[0162] In response to the expression package of the target object being triggered, at least one expression image of the target object is displayed in the expression display area.

[0163] In one implementation, the processing unit 502 is used to obtain a generated image, specifically to:

[0164] Obtain diffusion parameters, latent space noise image and first prompt information;

[0165] The diffusion model is called to perform iterative denoising on the latent space noise image according to the diffusion parameter and the first prompt information to obtain a generated image.

[0166] In one embodiment, the diffusion parameter includes at least one of the following: a sampler identifier, a noise loading schedule, a random seed, and a sampling step number.

[0167] According to one embodiment of the present application, Figure 2 and Figure 3 Some of the steps involved in the image processing method shown can be represented by Figure 5 The image processing apparatus shown in the figure is executed by each unit. For example, Figure 2 The step S201 shown in FIG. 1 can be performed by Figure 5 The acquisition unit 501 shown in FIG. 1 is executed, and steps S202 to S204 can be performed by Figure 5 The processing unit 502 shown executes; Figure 3 The step S301 shown in FIG. Figure 5 The acquisition unit 501 shown in FIG. 1 is executed, and steps S302 to S307 can be performed by Figure 5 Processing unit 502 is shown executing. Figure 5 The various units in the image processing device shown can be separately or completely combined into one or several other units to form, or one (some) of the units can also be split into multiple smaller units in function to form, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present application. The above-mentioned units are divided based on logical functions. In practical applications, the function of one unit can also be implemented by multiple units, or the functions of multiple units can be implemented by one unit. In other embodiments of the present application, the image processing device may also include other units. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented by the collaboration of multiple units.

[0168] According to another embodiment of the present application, the program can be executed by running a program on a general computing device such as a computer device including a central processing unit (CPU), a random access memory medium (RAM), a read-only memory medium (ROM), and other processing elements and storage elements. Figure 2 and Figure 3 A computer program (including program code) for each step involved in the corresponding method shown in Figure 5 The image processing device shown in the figure can realize the image processing method of the embodiment of the present application. The computer program can be recorded on a computer-readable recording medium, for example, and loaded into the above-mentioned computing device through the computer-readable recording medium and run therein.

[0169] Based on the same inventive concept, the principles and beneficial effects of solving the problems by the image processing device provided in the embodiment of the present application are similar to the principles and beneficial effects of solving the problems by the image processing method in the method embodiment of the present application. Please refer to the principles and beneficial effects of the implementation of the method. For the sake of concise description, they will not be repeated here.

[0170] See also Figure 6 , Figure 6 The following is a schematic diagram of the structure of a computer device provided in an embodiment of the present application. The computer device may be a terminal device or a server. Figure 6 As shown, the computer device at least includes a processor 601, a communication interface 602 and a memory 603. The processor 601, the communication interface 602 and the memory 603 can be connected via a bus or other means. The processor 601 (or central processing unit (CPU)) is the computing core and control core of the computer device, which can parse various instructions in the computer device and process various data of the computer device. For example, the CPU can be used to parse the power on and off instructions issued by the object to the computer device, and control the computer device to perform power on and off operations; for example, the CPU can transmit various interactive data between the internal structures of the computer device, and so on. The communication interface 602 can optionally include a standard wired interface, a wireless interface (such as WI-FI, a mobile communication interface, etc.), which can be used to send and receive data under the control of the processor 601; the communication interface 602 can also be used for the transmission and interaction of data within the computer device. The memory 603 (Memory) is a memory device in the computer device, which is used to store programs and data. It can be understood that the memory 603 here can include both the built-in memory of the computer device and the extended memory supported by the computer device. The memory 603 provides a storage space, which stores the operating system of the computer device, including but not limited to: Android system, iOS system, Windows Phone system, etc., which is not limited in this application.

[0171] The embodiment of the present application also provides a computer-readable storage medium (Memory), which is a memory device in a computer device for storing programs and data. It is understandable that the computer-readable storage medium here can include both built-in storage media in the computer device and, of course, extended storage media supported by the computer device. The computer-readable storage medium provides a storage space that stores the processing system of the computer device. In addition, a computer program suitable for being loaded and executed by the processor 601 is also stored in the storage space. It should be noted that the computer-readable storage medium here can be a high-speed RAM memory, or a non-volatile memory, such as at least one disk storage; optionally, it can also be at least one computer-readable storage medium located away from the aforementioned processor.

[0172] In one embodiment, the processor 601 performs the following operations by running the computer program in the memory 603:

[0173] Acquire a generated image, where the generated image is obtained by iteratively denoising the latent space noise image based on the diffusion parameter and the first prompt information, the first prompt information does not include a prompt word related to the target part, and the generated image includes the target part of the target object;

[0174] Perform noise processing on the generated image to obtain a sample noise image;

[0175] The latent space noise image is fused with the sample noise image to obtain a composite noise image; the composite noise image contains the noise of the area other than the area to which the target part belongs in the sample noise image;

[0176] Based on the diffusion parameter and M second prompt information, the composite noise image is iteratively denoised to obtain M images of the target object; the visual elements of each image except the target part are the same, and any second prompt information is obtained by adding the prompt word of the target part to the first prompt information, and M is a positive integer.

[0177] As an optional embodiment, the processor 601 fuses the latent space noise image with the sample noise image to obtain a specific embodiment of the composite noise image:

[0178] Extract the noise of the target area in the latent space noise image;

[0179] The noise in the target area of ​​the latent space noise image is fused with the sample noise image to obtain a composite noise image.

[0180] As an optional embodiment, the processor 601 fuses the noise of the target area in the latent space noise image with the sample noise image to obtain a composite noise image. The specific embodiment is:

[0181] Extract the noise of the area other than the area where the target part belongs in the sample noise image;

[0182] The noise of the target area in the latent space noise image is fused with the noise of the area other than the target area in the sample noise image to obtain a composite noise image.

[0183] As an optional embodiment, a specific embodiment of the processor 601 extracting the noise of the area to which the target part belongs in the latent space noise image is:

[0184] Perform image segmentation processing on the generated image to obtain the area to which the target part in the generated image belongs;

[0185] Generate a first mask based on the region to which the target part belongs in the generated image, the first mask being used to cover the region other than the region to which the target part belongs in the generated image;

[0186] The noise in the region to which the target part belongs in the latent space noise image is extracted through the first mask.

[0187] As an optional embodiment, the processor 601 performs noise processing on the generated image to obtain a specific embodiment of a sample noise image as follows:

[0188] Perform high-frequency information weakening processing on the generated image to obtain a blurred image of the generated image;

[0189] The blurred image is subjected to noise processing to obtain a sample noise image.

[0190] As an optional embodiment, the processor 601 performs noise processing on the blurred image to obtain a specific embodiment of a sample noise image as follows:

[0191] Call the diffusion model to iteratively add noise to the blurred image according to the diffusion parameters to obtain a sample noise image;

[0192] The input data for the first noise addition is a blurred image, the input data for the jth noise addition is the j-1th noise addition result, and j is an integer greater than 1.

[0193] As an optional embodiment, the processor 601 calls the diffusion model to perform iterative noise processing on the blurred image according to the diffusion parameter to obtain a specific embodiment of the sample noise image:

[0194] The control network is loaded into the diffusion model to obtain an updated diffusion model; the control network is used to improve the clarity of the input image;

[0195] The updated diffusion model is called to iteratively add noise to the blurred image according to the diffusion parameters to obtain a sample noise image.

[0196] As an optional embodiment, the processor 601 performs high-frequency information weakening processing on the generated image to obtain a blurred image of the generated image. The specific embodiment is:

[0197] Performing filtering on the generated image to obtain a blurred image of the generated image; or,

[0198] The generated image is transformed into a frequency domain to obtain a frequency domain representation of the generated image, the spectrum information of the high-frequency part of the generated image is removed from the frequency domain representation to obtain an updated frequency domain representation, and the updated frequency domain representation is inversely transformed into a frequency domain to obtain a blurred image of the generated image.

[0199] As an optional embodiment, the processor 601 performs iterative noise reduction processing on the composite noise image based on the diffusion parameter and the M second prompt information to obtain the M images of the target object, including:

[0200] Add the prompt words of the i-th group of target parts to the first prompt information to obtain the i-th second prompt information, where i is a positive integer less than or equal to M;

[0201] The diffusion model is called to perform iterative denoising on the composite noise image according to the diffusion parameters and the i-th second prompt information to obtain the i-th image of the target object.

[0202] As an optional embodiment, the target part is the face, and the prompt word of the target part is an expression prompt word; the processor 601 further performs the following operations by running the computer program in the memory 603:

[0203] Packing the M images of the target object to obtain an expression package of the target object;

[0204] In response to the expression package of the target object being triggered, at least one expression image of the target object is displayed in the expression display area.

[0205] As an optional embodiment, the specific embodiment of the processor 601 acquiring the generated image is:

[0206] Obtain diffusion parameters, latent space noise image and first prompt information;

[0207] The diffusion model is called to perform iterative denoising on the latent space noise image according to the diffusion parameter and the first prompt information to obtain a generated image.

[0208] As an optional embodiment, the diffusion parameter includes at least one of the following: a sampler identifier, a noise loading schedule, a random seed, and a sampling step number.

[0209] Based on the same inventive concept, the principles and beneficial effects of solving the problems provided by the computer device in the embodiment of the present application are similar to the principles and beneficial effects of solving the problems provided by the image processing method in the method embodiment of the present application. Please refer to the principles and beneficial effects of the implementation of the method. For the sake of concise description, they will not be repeated here.

[0210] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored. The computer program is suitable for being loaded by a processor and executing the image processing method of the above method embodiment.

[0211] The embodiment of the present application also provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the above-mentioned image processing method.

[0212] The steps in the method of the embodiment of the present application can be adjusted in order, combined and deleted according to actual needs.

[0213] The modules in the device of the embodiment of the present application can be merged, divided and deleted according to actual needs.

[0214] In the embodiments of the present application, the "module" or "unit" involved refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0215] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, and the readable storage medium can include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0216] What is disclosed above is only a preferred embodiment of the present application, and it certainly cannot be used to limit the scope of rights of the present application. Ordinary technicians in this field can understand that all or part of the processes of implementing the above embodiment and equivalent changes made according to the claims of the present application are still within the scope covered by the application.

Claims

1. An image processing method, characterized in that: The method comprises: Acquire a generated image, wherein the generated image is obtained by iteratively denoising the latent space noise image based on the diffusion parameter and the first prompt information, wherein the first prompt information does not include a prompt word related to the target part, and the generated image includes the target part of the target object; Performing noise processing on the generated image to obtain a sample noise image; The latent space noise image is fused with the sample noise image to obtain a composite noise image; the composite noise image contains the noise of the area in the sample noise image except the area to which the target part belongs; Based on the diffusion parameter and M second prompt information, the composite noise image is iteratively denoised to obtain M images of the target object; the visual elements of each image except the target part are the same, and any second prompt information is obtained by adding the prompt word of the target part to the first prompt information, and M is a positive integer.

2. The method according to claim 1, characterized in that The step of fusing the latent space noise image with the sample noise image to obtain a composite noise image includes: Extracting noise in the region where the target part belongs in the latent space noise image; The noise of the area to which the target part belongs in the latent space noise image is fused with the sample noise image to obtain a composite noise image.

3. The method according to claim 2, characterized in that The step of fusing the noise of the region to which the target part belongs in the latent space noise image with the sample noise image to obtain a composite noise image includes: Extracting noise from areas other than the area to which the target part belongs in the sample noise image; The noise of the region to which the target part belongs in the latent space noise image is fused with the noise of the region other than the region to which the target part belongs in the sample noise image to obtain a composite noise image.

4. The method according to claim 2, characterized in that The extracting the noise of the region to which the target part belongs in the latent space noise image includes: Performing image segmentation processing on the generated image to obtain the region to which the target part in the generated image belongs; generating a first mask based on the region to which the target part belongs in the generated image, wherein the first mask is used to cover the region in the generated image except the region to which the target part belongs; The noise of the area to which the target part belongs in the latent space noise image is extracted by using the first mask.

5. The method according to claim 1, characterized in that The step of performing noise processing on the generated image to obtain a sample noise image includes: Performing high-frequency information weakening processing on the generated image to obtain a blurred image of the generated image; The blurred image is subjected to noise addition processing to obtain a sample noise image.

6. The method according to claim 5, characterized in that The step of performing noise processing on the blurred image to obtain a sample noise image includes: Calling a diffusion model to perform iterative noise addition processing on the blurred image according to the diffusion parameters to obtain a sample noise image; The input data for the first noise addition is the blurred image, the input data for the jth noise addition is the j-1th noise addition result, and j is an integer greater than 1.

7. The method according to claim 6, characterized in that The calling of the diffusion model to iteratively add noise to the blurred image according to the diffusion parameter to obtain a sample noise image includes: Loading the control network into the diffusion model to obtain an updated diffusion model; the control network is used to improve the clarity of the input image; The updated diffusion model is called to perform iterative noise addition processing on the blurred image according to the diffusion parameters to obtain a sample noise image.

8. The method according to claim 5, characterized in that The step of performing high frequency information weakening processing on the generated image to obtain a blurred image of the generated image includes: Performing filtering processing on the generated image to obtain a blurred image of the generated image; or, The generated image is subjected to frequency domain conversion to obtain a frequency domain representation of the generated image, the spectrum information of the high-frequency part of the generated image is removed from the frequency domain representation to obtain an updated frequency domain representation, and the updated frequency domain representation is subjected to inverse frequency domain conversion to obtain a blurred image of the generated image.

9. The method according to claim 1, characterized in that The process of performing iterative noise reduction processing on the composite noise image based on the diffusion parameter and the M second prompt information to obtain the M images of the target object includes: Add the prompt words of the target part of the i-th group to the first prompt information to obtain the i-th second prompt information, where i is a positive integer less than or equal to M; The diffusion model is called to perform iterative noise reduction processing on the composite noise image according to the diffusion parameter and the i-th second prompt information to obtain the i-th image of the target object.

10. The method according to claim 1, characterized in that The target part is a face, and the prompt word of the target part is an expression prompt word; the method further includes: Packing the M images of the target object to obtain an expression package of the target object; In response to the expression package of the target object being triggered, at least one expression image of the target object is displayed in the expression display area.

11. The method according to claim 1, characterized in that The step of obtaining and generating an image comprises: Obtain diffusion parameters, latent space noise image and first prompt information; The diffusion model is called to perform iterative noise reduction processing on the latent space noise image according to the diffusion parameter and the first prompt information to obtain a generated image.

12. The method according to any one of claims 1 to 11, characterized in that: The diffusion parameters include at least one of the following: a sampler identifier, a noise loading schedule, a random seed, and a sampling step number.

13. An image processing device, characterized in that: The image processing device comprises: an acquisition unit, configured to acquire a generated image, wherein the generated image is obtained by iteratively denoising the latent space noise image based on the diffusion parameter and the first prompt information, wherein the first prompt information does not include a prompt word related to the target part, and the generated image includes the target part of the target object; A processing unit, used for performing noise processing on the generated image to obtain a sample noise image; and for fusing the latent space noise image with the sample noise image to obtain a composite noise image; the composite noise image includes noise in the sample noise image in an area other than the area to which the target part belongs; And for iteratively denoising the composite noise image based on the diffusion parameter and M second prompt information to obtain M images of the target object; the visual elements of each image except the target part are the same, any second prompt information is obtained by adding the prompt word of the target part to the first prompt information, and M is a positive integer.

14. A computer device, characterized in that: include: a memory, wherein a computer program is stored in the memory; A processor, used to load the computer program to implement the image processing method according to any one of claims 1 to 12.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded by a processor and executing the image processing method according to any one of claims 1 to 12.

16. A computer program product, characterized in that The computer program product comprises a computer program, and the computer program is suitable for being loaded by a processor and executing the image processing method according to any one of claims 1 to 12.

Images