Image processing method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides an image processing method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining an initial image, carrying out the processing of the initial image, obtaining a segmented image, and enabling the segmented image to be used for indicating a target object in the initial image; a synthetic image is obtained through a pre-trained image processing model and the segmented image, and the image processing model is used for generating a matched image background for the target object, so that visual physical interference does not occur between the target object in the output synthetic image and the image background. By utilizing the image generation capability of the image processing model, visual physical interference does not occur between the target object and the image background in the synthetic image, so that the authenticity of the synthetic image is improved, segmentation traces between the background and the target object are reduced, and the consistency between the background and the target object is improved.

Claims

1. An image processing method, It is characterized in that include: Acquire an initial image, and process the initial image to obtain a segmented image, wherein the segmented image is used to indicate a target object in the initial image; A composite image is obtained by using a pre-trained image processing model and the segmented image, wherein the image processing model is used to generate a matching image background for the target object so that there is no visual physical interference between the target object and the image background in the output composite image.

2. The method according to claim 1, It is characterized in that The image processing model includes an encoder unit, a decoder unit and a diffusion unit; A synthetic image is obtained by using a pre-trained image processing model and the segmented image, including: By means of the encoder unit, feature extraction is performed on the initial image to obtain a first feature map; Adding noise to the first feature map by the diffusion unit to generate a noisy feature map; Directionally denoising the noisy feature map by the diffusion unit, and in the directional denoising process, weightedly superimposing a target area of ​​the first feature map to obtain a denoised feature map, wherein the target area is an image area determined based on the segmented image; The de-noising feature map is decoded by the decoder unit to generate the synthetic image.

3. The method according to claim 2, It is characterized in that The diffusion unit includes at least one de-noising unit arranged in an orderly manner, and the noisy feature map is subjected to directional de-noising by the diffusion unit, and in the directional de-noising process, the target area of ​​the first feature map is weightedly superimposed to obtain the de-noised feature map, including: For each of the de-noising units, the following steps are performed in sequence: Get the sequence value of the current de-noising unit; If the sequence value is less than the sequence threshold, the current de-noising unit is used to perform directionally de-noising on the current noisy feature map to obtain an intermediate de-noising feature map output by the current de-noising unit, and then the current de-noising unit is set as the next de-noising unit, the current noisy feature map is set as the intermediate de-noising feature map, and the step of obtaining the sequence value of the current de-noising unit is returned to execute; If the sequence value is greater than the sequence threshold and less than the last sequence corresponding to the last denoising unit, the target area of ​​the first feature map is weightedly superimposed on the current denoising feature map to obtain the superimposed feature map corresponding to the current denoising unit, and then the superimposed feature map is directionally denoised to obtain the intermediate denoising feature map output by the current denoising unit, and then the current denoising unit is set as the next denoising unit, the current denoising feature map is set as the intermediate denoising feature map, and the step of obtaining the sequence value of the current denoising unit is returned to be executed; If the sequence value is equal to the last sequence corresponding to the last denoising unit, the target area of ​​the first feature map is weightedly superimposed on the current denoised feature map to obtain the superimposed feature map corresponding to the current denoising unit, and then the superimposed feature map is directionally denoised to obtain the denoised feature map.

4. The method according to claim 3, It is characterized in that The method further comprises: Acquire a segmentation confidence corresponding to the segmented image, wherein the segmentation confidence represents the accuracy of segmentation of the target object; Obtaining a target weighting coefficient according to the segmentation confidence; The weighted superposition of the target area of ​​the first feature map to the current noise-added feature map to obtain the superimposed feature map corresponding to the current de-noising unit includes: Based on the target weighting coefficient, the target area of ​​the first feature map is weighted and superimposed on the current noisy feature map to obtain the superimposed feature map corresponding to the current de-noising unit.

5. The method according to claim 1, It is characterized in that A synthetic image is obtained by using a pre-trained image processing model and the segmented image, including: Acquire prompt word information, where the prompt word information is used to guide the image processing model to generate an image background having target image content; The initial image, the prompt word information and the segmented image are input into the image processing model to generate the synthesized image.

6. The method according to claim 5, It is characterized in that The obtaining of prompt word information includes: Acquiring shooting parameters of the segmented image, where the shooting parameters represent shooting angle characteristics and / or illumination characteristics of a target object in the segmented image; According to the shooting parameters, a corresponding description text is obtained; The prompt word information is generated according to the description text.

7. The method according to claim 1, It is characterized in that The method further comprises: Acquire model scene information, where the model scene information is used to characterize an image style and / or image scene of an image background generated by the image processing model; Obtaining a target image processing model according to the model scene information; A synthetic image is obtained by using a pre-trained image processing model and the segmented image, including: The segmented image is processed by the target image processing model to obtain a composite image.

8. The method according to claim 1, It is characterized in that The segmented image includes a first segmented image or a second segmented image, wherein: The first segmented image is a transparent background image containing the target object; The second segmented image is a mask image used to locate the target object.

9. The method according to claim 1, It is characterized in that After obtaining the composite image, the method further includes at least one of the following: Based on the segmented image, superimposing the image texture corresponding to the target object into the synthesized image; The synthetic image is processed by a generative adversarial network model to obtain a super-resolution synthetic image, wherein the image resolution of the super-resolution synthetic image is greater than the image resolution of the synthetic image.

10. An image processing device, It is characterized in that include: An acquisition module, used for acquiring an initial image, and processing the initial image to obtain a segmented image, wherein the segmented image is used for indicating a target object in the initial image; A processing module is used to obtain a composite image through a pre-trained image processing model and the segmented image, wherein the image processing model is used to generate a matching image background for the target object so that there is no visual physical interference between the target object and the image background in the output composite image.

11. An electronic device, It is characterized in that include: Processor and memory; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the image processing method according to any one of claims 1 to 9.

12. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores computer-executable instructions, and when the processor executes the computer-executable instructions, the image processing method according to any one of claims 1 to 9 is implemented.

13. A computer program product comprising a computer program, It is characterized in that When the computer program is executed by a processor, the image processing method according to any one of claims 1 to 9 is implemented.

Images