Image cloning method and system based on PossionGAN network

Abstract

The present invention discloses an image cloning method and system based on a PossionGAN network, comprising: extracting an image to be cloned, selecting a background image, and determining the position of the image to be cloned in the background image; generating a binary template image according to the image to be cloned and the background image, and marking the area belonging to the image to be cloned and the area belonging to the background image respectively; calculating the gradient map of the image to be cloned and the gradient map of the background image; and replacing the gradient value in the gradient map of the image to be cloned into the gradient map of the background image to obtain a merged gradient map; replacing the pixel value in the image to be cloned into the background image to obtain a merged image; using the PossionGAN network to train the fused image, and using the trained generator G network and the merged image to generate a fused RGB image. The method of the present invention eliminates the need to calculate a large number of matrix inversion operations during cloning, and can be repeated to achieve a seamless image fusion effect.

Description

Technical Field

[0001] The present invention relates to image processing and machine learning technologies, and in particular to an image cloning method and system based on a PossionGAN network. Background Art

[0002] Seamless cloning has always been a hot topic in video image processing. Cloning can achieve the fusion of images in different backgrounds, making them look real and give people a real and fantastic visual experience. It has important applications in computer animation, virtual reality, and image editing.

[0003] The traditional seamless splicing based on the Possion equation can achieve seamless image fusion from the perspective of partial differential equations and has achieved good results. The calculation formula is shown as follows:

[0004]

[0005] in, is the gradient operation, is the Laplace operator, is the divergence of v. f is the unknown scalar value of the closed region, f * is a known scalar value of the enclosed area. S is R 2 is a closed set on S, and Ω is a closed subset on S. Using the guided domain vector field v = (u, v), the image can obtain the solution of the equation under the vector field. However, it faces the problem of solving the inverse operation of large matrices and the setting of the local intensity of the guided domain vector field requires manual participation, and the process of the previous calculation cannot be used. When processing videos, repeated calculations are required for the same target domain background.

[0006] In the prior art, the seamless stereo image cloning method based on straight line constraints uses a perspective scaling function, a straight line constraint function, a parallax constraint function, and a position fixing function in a grid deformation algorithm to adjust the size of a specified area of ​​a source stereo image to ensure that the parallax of the cloned stereo image is consistent and that no structural distortion occurs. However, this method is for the same scene environment, in which local area images are copied, and a network deformation method is used to correct the geometric shape distortion caused by observation. However, it is not suitable for cloning images under different lighting environments.

[0007] In recent years, the research on GAN network has been a hot topic in the field of artificial intelligence. Through the research on this network, many cutting-edge effects have been achieved, such as the generation of people and landscape photos, pixel-to-pixel mapping of images, image restoration, etc. The GAN network is a special structure network that integrates the discriminator D (classifier) ​​and the generator G (regressor), in which the discriminator D is used to supervise the learning process of the generator G, so that the generator G can effectively learn and integrate data information. Based on the GAN network, image restoration (Inpainting), image stylization (Art Style Transfer), image-to-image translation, etc. can be achieved. Therefore, the existing machine learning-based methods mainly edit or partially repair the entire image, but cannot correct the illumination of the local image, and have a high dependence on the data set, resulting in the image cloning problem that needs to be further solved.

[0008] In summary, the existing technologies have the following shortcomings: (1) The image restoration method has poor visual effects in filling the missing areas, and there are problems with blurring and restoration distortion. If it is used for image cloning, it is not only necessary to solve the distortion problem, but also to build a new theory to achieve guided restoration, which is quite difficult. (2) When using the GAN network to learn the illumination information of the image, there is a problem of image content pairing. During training, a large amount of data with consistent content in local areas must be obtained, and the problem of global consistency between the entire image and the local image must be solved, which is even more complicated. (3) The traditional Possion method relies on large-scale matrix inversion operations to solve partial differential equations. The matrix calculation process is also difficult to reuse. It is essentially different from the GAN network machine learning method and is difficult to directly combine and apply. Summary of the invention

[0009] Purpose of the invention: One purpose of the present invention is to provide an image cloning method based on a PossionGAN network by combining a GAN network and a Poisson equation.

[0010] Another object of the present invention is to provide an image cloning system based on a PossionGAN network.

[0011] Technical solution: An image cloning method based on a PossionGAN network of the present invention comprises the following steps:

[0012] Extract the image to be cloned img_A, select the background image img_B, and determine the position of the image to be cloned img_A in the background image img_B;

[0013] Generate a binary template image mask according to the image to be cloned img_A and the background image img_B, and mark the area belonging to the image to be cloned img_A and the area belonging to the background image img_B respectively;

[0014] Calculate the gradient map grad_A of the image to be cloned img_A and the gradient map grad_B of the background image img_B; replace the gradient value in the gradient map grad_A with the gradient map grad_B according to the marked binary template map mask to obtain the merged gradient map grad_comb;

[0015] According to the marked binary template mask, the pixel values ​​in the image to be cloned img_A are replaced with the background image img_B to obtain the merged image img_comb;

[0016] The PossionGAN network is used to train the fused image, and the generative network G in the trained PossionGAN network and the merged image img_comb are used to generate the fused RGB image.

[0017] Furthermore, the length and width of the image to be cloned img_A are wA and hA respectively, the length and width of the background image img_B are wB and hB respectively, wA is less than or equal to wB and hA is less than or equal to hB; the position of the upper left corner of the image to be cloned img_A in the background image B is offset_x and offset_y, and offset_x+wA<=wB, offset_y+hA<=hB.

[0018] Furthermore, the calculation method of the gradient map grad_A of the image to be cloned img_A and the gradient map grad_B of the background image img_B is:

[0019] The Sobel gradient operator is used to perform gradient derivation on each channel image of the to-be-cloned image img_A and the background image img_B, and the horizontal gradient image set grad_x of the to-be-cloned image img_A is obtained. A , vertical gradient image set grad_y A , and the horizontal gradient image set grad_x of the background image img_B B , vertical gradient image set grad_y B ;

[0020] Then, according to the horizontal gradient image set grad_x of the image to be cloned img_A A and vertical gradient image set grad_y A Get the gradient map grad_A of the image img_A to be cloned;

[0021] According to the horizontal gradient image set grad_x of the background image img_B B and vertical gradient image set grad_y B Get the gradient map grad_B of the background image img_B.

[0022] Furthermore, when replacing the gradient values ​​in the gradient map grad_A into the gradient map grad_B, starting from the position of the upper left corner of the image to be cloned img_A in the background image B, until the one-to-one corresponding gradient values ​​in the gradient map grad_B are replaced, a merged gradient map grad_comb is obtained.

[0023] Furthermore, when replacing the pixel values ​​in the image to be cloned img_A into the background image img_B, starting from the position of the upper left corner of the image to be cloned img_A in the background image B, the pixel values ​​in the background image img_B are replaced one by one, thereby obtaining a merged image img_comb.

[0024] Furthermore, the training method of using the PossionGAN network to fuse images is:

[0025] (1) Set the number of training times N of the PossionGAN network, the illumination information limit term λ1, the pixel limit term λ2, and the learning rate a;

[0026] (2) The network parameter matrix of the generated network G in the PossionGAN network is The generated image is G(img_comb), and the training of the generative network G is limited by illumination, and is recorded as |▽G(img_comb)-grad_comb|; where ▽ represents the gradient operation;

[0027] (3) Use the merged image img_comb to limit the range of pixel values ​​during training of the generated network G, denoted by |G(img_comb)-img_comb|;

[0028] (4) The network parameter matrix of the identification network D in the PossionGAN network is It convolves the generated image G(img_comb) to obtain the feature map matrix D(G(img_comb)), and sends the feature map matrix D(G(img_comb)) to the sigmoid function for discrimination to obtain the judgment value matrix sigmoid(G(img_comb)), where the discriminant network parameter matrix is

[0029] (5) Calculate the error between the image G(img_comb) generated by the generative network G in the PossionGAN network and the merged image img_comb as: G(img_comb)-img_comb, and calculate the cumulative sum gen of the absolute values ​​of the errors;

[0030] (6) Calculate the discrimination error of the discriminant network D in the PossionGAN network for the generated image G(img_comb), and calculate the cumulative sum dis of 1-sigmoid(G(img_comb));

[0031] (7) The cost of calculating the generation network G and the identification network D is:

[0032] J=dis+λ1*|▽G(img_comb)-grad_comb|+λ2*gen;

[0033] (8) Define PossionGAN network parameters Calculate the derivatives of each parameter in the PossionGAN network

[0034] (9) Update the PossionGAN network parameters to: Where a is the learning rate;

[0035] (10) Repeat steps (2) to (9) until t = N, and then terminate the loop.

[0036] In another embodiment of the present invention, an image cloning system based on a PossionGAN network includes an image input selection module, an image content extraction module, a PossionGAN cloning fusion calculation module, and a parameter and picture saving and output module;

[0037] An image input selection module is used to upload a user image, select a background image or video from a database, and provide a starting position of the user image in the background image;

[0038] The image content extraction module is used to extract the region of interest in the user image to obtain the image img_A to be cloned;

[0039] PossionGAN cloning fusion calculation module, used to complete the training of PossionGAN network and the fusion of the image to be cloned img_A and the background image;

[0040] The parameter and image saving output module saves the trained parameters of the PossionGAN network and outputs the image fusion results.

[0041] Preferably, the PossionGAN network includes a generation network G and a discrimination network D, the generation network adopts a U-type network, and the discrimination network D adopts a depth network with the same depth as the U-type network.

[0042] In another embodiment of the present invention, a device includes a memory and a processor, wherein:

[0043] A memory for storing computer programs that can be run on the processor;

[0044] The processor is used to execute the steps of the above-mentioned image cloning method based on the PossionGAN network when running the computer program.

[0045] In another embodiment of the present invention, a storage medium stores a computer program, and when the computer program is executed by at least one processor, the steps of the above-mentioned image cloning method based on the PossionGAN network are implemented.

[0046] Beneficial effects: Compared with the prior art, the advantages of the present invention are: (1) the original image to be cloned is used directly to guide cloning, rather than repairing; (2) no paired image data set is required, only the information of the existing background image is used; (3) the neural network learning method is adopted, and the parameters can be retained for reuse. In addition, the method of the present invention proposes to use the competitive method in the GAN network to learn the distribution information of the image, and innovatively proposes a method to restrict the illumination information of the image in the Poisson equation, which increases the difference between the generated image and the original image. Figure 1 The GAN network structure is designed to ensure that when the GAN network generates an image, not only the consistency of the non-cloned area is retained, but also the lighting conditions of the image cloned area are consistent with those of the non-cloned area, so as to obtain a method for the final seamless fusion effect. After multiple experiments, the method of the present invention obtains the effect of image cloning, and the training parameters can be reused. At the same time, the present invention can also complete the cloning of videos. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 is a flow chart of the method of the present invention;

[0048] Figure 2 is the image to be cloned;

[0049] Figure 3 When background image;

[0050] Figure 4 It is a schematic diagram of a U-shaped network structure;

[0051] Figure 5 is the fused image;

[0052] Figure 6It is a system block diagram of the present invention. DETAILED DESCRIPTION

[0053] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0054] The PossionGAN algorithm in the image cloning method based on the PossionGAN network of the present invention is mainly based on the GAN (Generative Advertise Neural Network) network method, combined with the idea of ​​Poisson image cloning. For the local image to be cloned, the competitive method in the GAN network is first used to learn the distribution information of the image, and the theory of the illumination restriction of the image in the Poisson equation is introduced, and the generated image is increased. Figure 1 The method of the present invention eliminates the need to calculate a large number of matrix inversion operations during cloning and can be reused in the same or similar scenes. Figure 1 As shown, the specific steps include:

[0055] S1. Extract the region of interest from an image data to obtain the image img_A to be cloned, such as Figure 2 As shown, the length and width are wA = 512 and hA = 512 respectively, and another image is selected as the background image img_B, such as Figure 3 As shown, the non-cloned image has a length and a width of wB=1024 and hB=1024. wA is less than or equal to wB and hA is less than or equal to hB.

[0056] S2. Set the position of the upper left corner of the image to be cloned img_A in the background image img_B to offset_x=360 and offset_y=400, and ensure that offset_x+wA<=wB, offset_y+hA<=hB.

[0057] S3, using the gradient operator to calculate the gradients of the image to be cloned img_A and the background image img_B, respectively, to obtain the gradient map grad_A of the image to be cloned img_A and the gradient map grad_B of the background image img_B, respectively. The specific steps for calculating the gradient are:

[0058] S31, using the Sobel gradient operator to calculate the gradients of each channel image in the to-be-cloned image img_A and the background image img_B, respectively, wherein the Sobel gradient operator is an n*n matrix, n is an odd number; the Sobel gradient operator in this embodiment is a 3*3 matrix, defining a horizontal gradient image set grad_x and a vertical gradient image set grad_y;

[0059] S32, define the center as the origin, the horizontal index i and the vertical index j are integers between -n / 2 and n / 2, the value of the Sobel gradient operator at the corresponding position in the horizontal direction is i / (i*i+j*j), and the value of the gradient operator at the corresponding position in the vertical direction is j / (i*i+j*j); in this embodiment, the Sobel gradient operator Sobel_x in the horizontal direction and the Sobel gradient operator Sobel_y in the vertical direction are shown as follows:

[0060]

[0061] S33, combining the gradient images of each channel in the RGB image in the horizontal direction and the vertical direction, and obtaining:

[0062] The horizontal gradient image set is: grad_x = [grad_x_R, grad_x_G, grad_x_B];

[0063] The vertical gradient image set is: grad_y = [grad_y_R, grad_y_G, grad_y_B].

[0064] S4. Generate a binary template image mask according to the image to be cloned img_A and the background image img_B, wherein the area belonging to the background image img_B is marked as 0, and the area belonging to the image to be cloned img_A is marked as 1, and the area marked as 1 is the clone area Ω; specifically:

[0065] S41, define loop variables i, j;

[0066] S42, traverse loop variables i=1, 2, 3, ..., hB;

[0067] S43, traverse loop variables j=1, 2, 3, ..., wB;

[0068] S44, if i>=offset_y and i<=offset_y+hA and j>=offset_x and j<=offset_x+wA, and the RGB channel values ​​in img_A(i-offset_y, j-offset_x) are not all 0, if the judgment is true, mask(i, j)=1;

[0069] S45. In other cases, mask(i, j)=0;

[0070] S46. Optimize the mask template by performing morphological operations on the mask template.

[0071] S5. Replace the gradient values ​​in the gradient image grad_A with those in the gradient image grad_B according to the marked binary template image mask, starting from the positions offset_x and offset_y, until the corresponding gradient values ​​in the gradient image grad_B are replaced, and the merged gradient image grad_comb is obtained. Specifically:

[0072] S51, define loop variables i, j;

[0073] S52, traverse loop variables i=1, 2, 3, ..., hB;

[0074] S53, traverse loop variables j=1, 2, 3, ..., wB;

[0075] S54, if mask(i, j)=1, if the judgment is true, execute step S55, otherwise execute step S56;

[0076] S55. grad_comb(i, j)=grad_A(i-offset_y, j-offset_x);

[0077] S56. grad_comb(i,j)=grad_B(i,j).

[0078] S6, according to the marked binary template image mask, replace the pixel values ​​in the to-be-cloned image img_A with the background image img_B, starting from the positions offset_x and offset_y, until the one-to-one corresponding pixel values ​​in the background image img_B are replaced, and the merged image img_comb is obtained; specifically:

[0079] S61, define loop variables i, j;

[0080] S62, traverse loop variables i=1, 2, 3, ..., hB;

[0081] S63, traverse loop variables j=1, 2, 3, ..., wB;

[0082] S64, determine if mask(i, j)=1, if so, execute step S65, otherwise execute step S66;

[0083] S65. img_comb(i, j)=img_A(i-offset_y, j-offset_x);

[0084] S66. img_comb(i,j)=img_B(i,j).

[0085] S7. Use the PossionGAN network to train the fusion image. The PossionGAN network structure contains a generator G network and a discriminator D network, and introduces restrictions on illumination information. Specifically:

[0086] S71, setting the number of training times of the PossionGAN network N = 20, the light information restriction item λ1 = 100, the pixel restriction item λ2 = 120, and the learning rate a = 0.0002;

[0087] S72, the generator G network adopts a U-shaped network structure, such as Figure 4 As shown, the depth of the generator G network is set to 3 layers. On the left side of the U-shaped network, the 0th layer uses a convolutional neural network (CNN) with n=64 cores to convolve the RGB image to obtain a 64-channel feature map, and then performs further learning with Res-Net with 64 convolution kernels to obtain a 64-channel feature map; for the first layer, d=1, a convolutional neural network (CNN) with 2*64 cores and MaxPooling are used to reduce the image to half of the original size, and then a Res-Net with 2*64 convolution kernels is used for further learning to obtain a 2*64 channel feature map with a side length of 1 / 2. The second layer at the bottom of the U-shaped network uses 2 2 *64 core convolutional neural network learning and m=4 layer Res-Net deep learning. For the right side of the U-shaped network, perform the reverse operation, double the size of the feature map after the previous convolution, and reduce the number of channels to half of the original, and combine it with the feature map of the corresponding depth on the left side of the U-shaped network with half the number of channels, and obtain 2 2 *64 channels, side length is 1 / 2 2 The feature map is reused 2 2 *The Res-Net with 64 convolution kernels is further learned until the 0th layer is connected to a convolution network with 3 convolution kernels to convert the feature map into an RGB image G (img_comb), where the network parameter matrix of the generator G network is W U t ;

[0088] S73, training the illumination restriction generator G network, record |▽G(img_comb)-grad_comb|, specifically, obtain the Sobel gradient of the image G(img_comb) generated by the U-type network, and the gradient is obtained as in steps S31 to S33;

[0089] S74, using the merged image img_comb to limit the range of pixel values ​​during the training of the generator G network, denoted by |G(img_comb)-img_comb|, and setting the influence coefficient of this item on the generator G network to λ2;

[0090] S75, the discriminator D network uses the same depth as the U-type network, depth = 3 network. The 0th layer uses a 64-core convolutional neural network (CNN) to convolve the RGB image to obtain a 64-channel feature map, and then performs n convolution kernels Res-Net further learning to obtain a 64-channel feature map; for the first layer, a 2*64 core convolutional neural network (CNN) and MaxPooling are used to reduce the image to half of the original, and then a 2*64 convolution kernel Res-Net is used for further learning to obtain a 2*64 channel feature map D (G (img_comb)) with a side length of 1 / 2. For the second layer, 2 3 * 64 core convolutional neural network learning and m = 4 layer Res-Net deep learning, get 2 2 *64 channels, side length is 1 / 2 2 The feature map of the discriminator D network parameter matrix is

[0091] S76, the feature map matrix D(G(img_comb)) generated by the discriminator D network is sent to the sigmoid function for discrimination, and 2 3 *64*1 / 2 3 *1 / 2 3 The judgment value matrix is ​​sigmoid(D(G(img_comb))), where the discriminant network parameter matrix is

[0092] S77, calculating the error between the image G(img_comb) generated by the generator G network in the PossionGAN network and the merged image img_comb, and calculating the cumulative sum gen of the absolute value of the error G(img_comb)-img_comb;

[0093] S78. Calculate the error of the discriminator D network in the PossionGAN network for the generated image G(img_comb), and calculate the cumulative sum dis of 1-sigmoid(D(G(img_comb)));

[0094] S79. Calculate the cost of the generator G network and the discriminator D network as:

[0095] J=dis+λ1*|▽G(img_comb)-grad_comb|+λ2*gen;

[0096] S710. Define PossionGAN network parameters Calculate the derivatives of various parameters in the network

[0097] S711, update the PossionGAN network parameters, Where a=0.0002 is the learning rate;

[0098] S712, loop from step S72 to step S711 until t=N, terminate the loop, and save the neural network parameters

[0099] S8. Send the img_comb image to the generator G in the PossionGAN network after parameter update to generate an RGB image output.

[0100] like Figure 5 As shown, Figure 2 The dolphins in Figure 3 In the sky, Figure 5 Dolphin colors compared to Figure 2 A major change has taken place, which not only retains the original skin color of the dolphin, but also keeps the lighting consistent with the background image, and maintains a high level of detail, such as the smooth color of the dolphin's belly and the lighting changes on the back consistent with the original image, achieving complete image fusion.

[0101] Another embodiment of the present invention also provides an image cloning system based on the PossionGAN network. Figure 6 As shown, the system includes an image input selection module, an image content extraction module, a PossionGAN cloning fusion and calculation module, and a parameter and image saving output module, wherein:

[0102] The image input selection module is used to upload the user image and send the image to the image content extraction module. The background image or video is selected from the database and the starting position of the user image in the background image is provided; the background image or video and the starting position are sent to the PossionGAN clone fusion calculation module;

[0103] The image content extraction module extracts the content of the image uploaded by the user to obtain the area of ​​interest, that is, the image to be cloned; then the extracted image to be cloned is sent to the PossionGAN clone fusion calculation module;

[0104] The PossionGAN cloning fusion calculation module executes the above-mentioned image seamless splicing method based on the PossionGAN network, completes the training of the PossionGAN network and the fusion of the image, and outputs the PossionGAN network parameters and images;

[0105] The parameter and image saving output module saves the trained parameters of the PossionGAN network and provides them to the PossionGAN clone fusion calculation module for use in the next training or generation; at the same time, it outputs the image fusion result.

[0106] In this exemplary embodiment, a device is also provided, including a memory and a processor, wherein the memory is used to store a computer program that can be run on the processor; and the processor is used to execute the steps of the above-mentioned image cloning method based on the PossionGAN network when running the computer program. And achieve the same technical effect as the above-mentioned method.

[0107] In this exemplary embodiment, a computer-readable storage medium including program instructions is also provided, and when the program instructions are executed by a processor, the steps of the above-mentioned image cloning method based on the PossionGAN network are implemented, and the technical effect consistent with the above-mentioned method is achieved.

Claims

1. An image cloning method based on PossionGAN network, characterized in that: The following steps are involved: Extract the image to be cloned img_A, select the background image img_B, and determine the position of the image to be cloned img_A in the background image img_B; Generate a binary template image mask according to the image to be cloned img_A and the background image img_B, and mark the area belonging to the image to be cloned img_A and the area belonging to the background image img_B respectively; Calculate the gradient map grad_A of the image to be cloned img_A and the gradient map grad_B of the background image img_B; replace the gradient value in the gradient map grad_A with the gradient map grad_B according to the marked binary template map mask to obtain the merged gradient map grad_comb; According to the marked binary template mask, the pixel values ​​in the image to be cloned img_A are replaced with the background image img_B to obtain the merged image img_comb; Use the PossionGAN network to train the fused image, and use the generator G network in the trained PossionGAN network and the merged image img_comb to generate the fused RGB image; The training method for fusion image using PossionGAN network is: (1) Set the number of training times N of the PossionGAN network, the illumination information limit term λ1, the pixel limit term λ2, and the learning rate a; (2) The network parameter matrix of the generator G network in the PossionGAN network is The generated image is G(img_comb), which is trained using the illumination-constrained generator G network, denoted by |▽G(img_comb)-grad_comb|; where ▽ represents the gradient operation; (3) Use the merged image img_comb to limit the range of pixel values ​​during network training of the generator G, denoted by |G(img_comb)-img_comb|; (4) The network parameter matrix of the discriminator D network in the PossionGAN network is It convolves the generated image G(img_comb) to obtain the feature map matrix D(G(img_comb)), and sends the feature map matrix D(G(img_comb)) to the sigmoid function for discrimination to obtain the judgment value matrix sigmoid(G(img_comb)), where the discriminant network parameter matrix is ​​W S t ; (5) Calculate the error between the image G(img_comb) generated by the generator G network in the PossionGAN network and the merged image img_comb: G(img_comb)-img_comb, and calculate the cumulative sum gen of the absolute value of the error; (6) Calculate the discrimination error of the discriminator D network in the PossionGAN network for the generated image G(img_comb), and calculate the cumulative sum dis of 1-sigmoid(G(img_comb)); (7) The cost values ​​of the generator G network and the discriminator D network are calculated as: (8) Define PossionGAN network parameters Calculate the derivatives of each parameter in the PossionGAN network (9) Update the PossionGAN network parameters to: Where a is the learning rate; (10) Repeat steps (2) to (9) until t = N, and then terminate the loop.

2. According to claim 1, the image cloning method based on PossionGAN network is characterized in that: The length and width of the image to be cloned img_A are wA and hA respectively, and the length and width of the background image img_B are wB and hB respectively, wA is less than or equal to wB and hA is less than or equal to hB; the position of the upper left corner of the image to be cloned img_A in the background image B is offset_x and offset_y, and offset_x+wA<=wB, offset_y+hA<=hB.

3. The image cloning method based on PossionGAN network according to claim 1, characterized in that: The calculation method of the gradient map grad_A of the image to be cloned img_A and the gradient map grad_B of the background image img_B is: The Sobel gradient operator is used to perform gradient derivation on each channel image of the to-be-cloned image img_A and the background image img_B, and the horizontal gradient image set grad_x of the to-be-cloned image img_A is obtained. A , vertical gradient image set grad_y A , and the horizontal gradient image set grad_x of the background image img_B B , vertical gradient image set grad_y B ; Then, according to the horizontal gradient image set grad_x of the image to be cloned img_A A and vertical gradient image set grad_y A Get the gradient map grad_A of the image img_A to be cloned; According to the horizontal gradient image set grad_x of the background image img_B B and vertical gradient image set grad_y B Get the gradient map grad_B of the background image img_B.

4. The image cloning method based on PossionGAN network according to claim 1, characterized in that: When replacing the gradient values ​​in the gradient map grad_A into the gradient map grad_B, start from the position of the upper left corner of the image to be cloned img_A in the background image B until the one-to-one corresponding gradient values ​​in the gradient map grad_B are replaced, and the merged gradient map grad_comb is obtained.

5. The image cloning method based on PossionGAN network according to claim 1, characterized in that: When replacing the pixel values ​​in the image to be cloned img_A into the background image img_B, start from the position of the upper left corner of the image to be cloned img_A in the background image B until the one-to-one corresponding pixel values ​​in the background image img_B are replaced to obtain the merged image img_comb.

6. A system for the image cloning method based on the PossionGAN network according to any one of claims 1 to 5, characterized in that: It includes image input selection module, image content extraction module, PossionGAN cloning fusion calculation module, parameter and image saving output module; An image input selection module is used to upload a user image, select a background image or video from a database, and provide a starting position of the user image in the background image; The image content extraction module is used to extract the region of interest in the user image to obtain the image img_A to be cloned; PossionGAN cloning fusion calculation module, used to complete the training of PossionGAN network and the fusion of the image to be cloned img_A and the background image; The parameter and image saving output module saves the trained parameters of the PossionGAN network and outputs the image fusion results.

7. The image cloning system based on PossionGAN network according to claim 6, characterized in that: The PossionGAN network includes a generator G network and a discriminator D network. The generator network adopts a U-type network, and the discriminator D network adopts a network with the same depth as the U-type network.

8. A device, characterized in that: comprising a memory and a processor, wherein: A memory for storing computer programs that can be run on the processor; A processor, configured to execute the steps of an image cloning method based on a PossionGAN network as described in any one of claims 1 to 5 when running the computer program.

9. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by at least one processor, the steps of the image cloning method based on the PossionGAN network as described in any one of claims 1 to 5 are implemented.

Images