An image synthesis system

Abstract

The invention belongs to the field of artificial intelligence image processing technology, and specifically relates to an image synthesis system, including an edge enhancement line extraction module, an edge style extraction module, a background style extraction module, an image deformation control module, an edge style transfer module, an edge mask generation module, a foreground style transfer module, a foreground mask generation module, a foreground image fusion module and a target image fusion module; the edge enhancement line extraction module is used to process the foreground image and the background image, and after edge detection and enhancement are performed on the image input to the edge enhancement line extraction module, the edge line is extracted based on the edge feature to obtain the corresponding line image; the target image fusion module fuses the foreground fusion image and the background image based on the foreground mask to obtain a target fusion image, and the target fusion image is the output result of the image synthesis system. The invention takes into account the requirements that the edge line style is consistent with the line style in the background image and that the foreground image and the background image are naturally fused.

Description

Technical Field

[0001] The present invention belongs to the technical field of artificial intelligence image processing, and in particular relates to an image synthesis system. Background Art

[0002] In the prior art, image synthesis needs to consider the style difference between the added foreground image and the background image, so it is necessary to perform style transfer on the foreground image and the synthesized part before image synthesis, so as to ensure that the styles of the synthesized foreground image and the background image are consistent, or have a smooth transition, so as to achieve a better synthesis effect. However, although some means have been adopted in the prior art to achieve the transfer of the background style to the foreground image, if there are stylized effects based on the lines and corresponding lines at the edges of various parts of the image in the background style, such as some painting images that emphasize line effects, special font images, or images with special light and shadow effects at the junction of objects or at the boundaries of concave and convex structures, the style at the edge may be greatly different from that of other parts of the background image. When the prior art performs overall style transfer on the background image, it is easy to cause style distortion in these edge parts or erode the edge line parts with the style features of the non-edge line parts. If the model focuses too much on the style of the edge part, it may cause the contour part between the foreground part and the background part to have a large change in the style of the edge line and the style of the corresponding part of the background image, and lack of transition, resulting in the foreground image being more abrupt relative to the background part in the synthesized image. Moreover, when the foreground image itself does not have edge lines with obvious style, or even when the edge itself is not clear and obvious enough, this model cannot directly perform style transfer on the foreground image, otherwise it is easy to cause edge line distortion after conversion and style transfer errors. Summary of the invention

[0003] The purpose of the present invention is to provide an image synthesis system for solving the technical problem that it is difficult in the prior art to merge images with unclear edges into images with unique edge lines at the edges while ensuring a unified style merging effect, especially it is difficult to ensure that the edge lines of both foreground and background images are clear and consistent in style and that the effect after merging the two is not abrupt.

[0004] The image synthesis system includes: an edge enhancement line extraction module, which is used to process a foreground image and a background image. After edge detection and enhancement are performed on the image input to the edge enhancement line extraction module, edge lines are extracted based on edge features to obtain a corresponding line image; the line image obtained after the background image is processed by the edge enhancement line extraction module is a background edge feature image, and the line image obtained after the foreground image is processed by the edge enhancement line extraction module is a foreground edge feature image.

[0005] The edge style extraction module is used to extract edge style features from the background edge feature image.

[0006] The background style extraction module is used to extract background style features from the background image.

[0007] The image deformation control module is used to deform the foreground image and the foreground edge feature image according to the external input image deformation control information so as to obtain the corresponding foreground deformation feature image and edge deformation feature image.

[0008] The edge style transfer module is used to transfer the style of the edge deformation feature image according to the edge style feature to obtain the style transfer edge feature.

[0009] The edge mask generation module is used to generate the corresponding edge mask according to the style migration edge features.

[0010] The background style transfer module is used to transfer the style of the foreground deformation feature image according to the background style features to obtain the style transferred foreground features.

[0011] The edge mask generation module is used to generate the corresponding foreground mask according to the style migration foreground features.

[0012] The foreground image fusion module is based on the edge mask and fuses the style-transferred foreground features and the style-transferred edge features to form a foreground fusion image.

[0013] The target image fusion module fuses the foreground fusion image and the background image based on the foreground mask to obtain the target fusion image, and the target fusion image is the output result of this image synthesis system.

[0014] Preferably, the edge enhancement line extraction module includes an edge detection module and a line extraction module. The original image input to the edge enhancement line extraction module is first subjected to edge detection by the edge detection module, and then the original image and the edge detection result are input to the line extraction module. The line extraction module uses the edge detection result to enhance the edge features of the original image, and then extracts edge lines consistent with the edge feature trajectory based on the edge features.

[0015] Preferably, the original image input to the edge detection module is P0, and the corresponding multi-scale feature P is extracted by the encoder. i (i=1,2,3…,n), n is the number of encoder layers, all multi-scale features are processed by the receiving domain block, and the smallest multi-scale feature P n After being optimized by the receiving domain block, the corresponding minimum scale multi-scale optimization feature f is obtained n , that is, f n =RFB(P n ), RFB means processing by receiving domain blocks, for other scales of P i(i=1,2,3…,n-1), then we need to combine the multi-scale optimization features f of the next layer (smaller scale) i+1 Generate the corresponding multi-scale optimization feature f i (i=1,2,3…,n-1), the specific expression is as follows:

[0016] f i =CONV(RFB(P i )© UP(f i+1 )),

[0017] Among them, CONV represents a convolution operation with a kernel size of 1×1, RFB represents processing through a receptive field block, © represents a concatenation operation, and UP represents an upsampling operation through bilinear interpolation.

[0018] Then calculate the multi-scale edge feature f' i (i=1,2,3…,n), when i=n, ​​the minimum scale multi-scale edge feature f' n The minimum scale multi-scale optimization feature f is used n , f' n =f n ; When i≠n, by multi-scale optimization feature f i (i=1,2,3…,n-1) and the multi-scale edge features f' of the next layer i+1 These two features are input into the edge selection module to obtain the multi-scale edge features f' of the corresponding scale. i (i=1,2,3…,n-1).

[0019] Preferably, the edge selection module uses a gating mechanism to adaptively select the multi-scale optimization feature f i and multi-scale edge features f' i+1 :First, find the multi-scale optimization feature f i and multi-scale edge features f' i+1 The enhanced feature x obtained by adding element by element i , then enhance the feature x i First, a global average pooling operation is performed, and then a fully connected layer containing a ReLU function and a batch normalization operation is used to process the edge selection index b. The edge selection index b is used to calculate the selection weight matrix of the two features. The calculation formula is as follows:

[0020] , ,

[0021] Among them, w i and w i+1 They are the multi-scale optimized features f i and multi-scale edge features f' i+1The selection weight matrix is, b is the edge selection index, A i and A i+1 are the corresponding multi-scale optimization features f i and multi-scale edge features f' i+1 Adjustable parameters. Get the selection weight matrix w i and w i+1 Then we can optimize the multi-scale features f i and multi-scale edge features f' i+1 Select and obtain the corresponding multi-scale edge feature f' i , the calculation formula is: f' i =w i ⊙f i ⊕w i+1 ⊙f' i+1 .

[0022] Preferably, while extracting the multi-scale edge features, a global feature extractor is used to extract the global feature f0, and the global feature f0 is combined with the multi-scale edge features f' of each scale. i After fusion, the edge detection result f is generated by the convolution operation of the convolution kernel 1×1 E , the expression is: E = CONV(w0f0⊕w'1f'1⊕w'2f'2⊕…⊕w' n f' n ), where w' i Represents the multi-scale edge features f' corresponding to each scale i The weight of i=1,2,3…,n, w0 represents the weight corresponding to the global feature f0; then the edge detection result f E Output.

[0023] Preferably, in the line extraction module, the edge detection result f E Add the original image P0 element by element to obtain the edge enhancement image P containing the edge detection result ES Then the edge enhancement image P ES According to the enhanced edge features, the pixels are classified to extract the corresponding line image P out The line extraction module samples the enhanced edge features and analyzes them pixel by pixel along the edge features. By analyzing the color vector distance between the pixel points in the area around the edge features and the sampling points on the edge features, it can be determined whether the corresponding pixel points belong to the line where the edge features are located. The calculation formula of the color vector distance is: c =min i ‖C-C i ‖, where min i Indicates the position of the currently found minimum element in the image, d crepresents the color vector distance, C represents the color of the analyzed pixel, and C i It represents the i-th color value among several representative colors obtained by the sampling point; by setting the threshold of the color vector distance, it is judged whether the corresponding pixel point belongs to the line where the edge feature is located, thereby extracting the line image P out .

[0024] Preferably, the image deformation control information is input from the outside to the image deformation control module, and the image deformation control module respectively performs the foreground image I F and foreground edge feature image F FEL Processing is performed to obtain the corresponding foreground deformation feature image F FD and edge deformation feature image F ED .

[0025] Preferably, the edge style feature F ES Including background edge feature image F BEL The texture information, width information of the line image and the position information of the line image relative to the edge feature; the edge style feature F ES and edge deformation feature image F ED The input is sent to the edge style transfer module, which first identifies edge features from the edge deformation feature image and then extracts the edge style feature F ES The texture information, width information and position information in the style are transferred to obtain the style transfer edge feature F STE ; The edge mask generation module is based on the style migration edge feature F STE Generate the corresponding edge mask M E ; Edge mask M E , style transfer foreground feature F STF and style transfer edge feature F STE Input the foreground image fusion module and fuse it to obtain the foreground fusion image I FF , the expression is: FF =M E ⊙F STF ⊕F STE .

[0026] Preferably, the background style feature F BS and the foreground deformation feature image F FD Input to the foreground style transfer module, style transfer obtains the style transfer foreground feature F STF ; Then the foreground mask generation module is based on the style migration foreground feature F STF Generate the corresponding foreground mask M F ; Then merge the foreground image I FF , foreground mask M F and background image I BInput the target image fusion module and fuse it to get the target fused image I O , the expression is: O =M F ⊙I B ⊙I FF ⊕I FF .

[0027] The present invention has the following advantages: the present invention first detects and enhances the edge in the image through the edge enhancement line extraction module, and then detects and segments the line image of the edge line in the background image based on the enhanced edge feature, thus ensuring the accurate segmentation of the line image, and can be used for the detection of edge lines with line style in the edge part, and can also be used for the detection and extraction of edge features of the corresponding edge lines. After extracting the foreground edge feature image and the background edge feature image, the edge style features of the edge line part in the background image are first extracted respectively, so that the edge features in the foreground image can be targetedly style-transferred, and the extracted edge features can be deformed under the action of deformation control information alone, avoiding the edge feature extraction effect after the deformation causes distortion to the edge part in the foreground image. Then, through targeted edge mask generation, the area marking and covering of the edge lines in the foreground image after style transfer in the foreground image is realized, and then the fusion of the edge features of the foreground image and the edge features after style transfer is realized. Since the area of ​​the edge line has been covered by the edge mask before fusion, the phenomenon of erosion of the edge line by the background image will not occur. Then, a foreground mask is generated by using the foreground image without the fusion of style transfer edge features. In this way, the relevant position area of ​​the background image is covered based on the foreground mask, and the foreground fusion image is fused with the covered background image. In this way, the foreground image and the background image do not affect each other, but the edge lines at the contour of the foreground image will be fused and transitioned with the background image, thereby compounding the transition effect between the edge lines in the background image and the background style. In this way, the corresponding problems existing in the prior art are overcome, and the requirements of taking into account the consistency of the edge line style with the line style in the background image and the natural fusion of the foreground image and the background image are ensured. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 A schematic diagram of an image synthesis system according to the present invention.

[0029] Figure 2 Schematic diagram of the edge enhancement line extraction module in the present invention. DETAILED DESCRIPTION

[0030] The specific implementation modes of the present invention are further explained in detail below by describing the embodiments with reference to the accompanying drawings, so as to help those skilled in the art to have a more complete, accurate and in-depth understanding of the inventive concept and technical solution of the present invention.

[0031] like Figure 1 , Figure 2 As shown, the present invention provides an image synthesis system, including an edge enhancement line extraction module, an edge style extraction module, a background style extraction module, an image deformation control module, an edge style migration module, an edge mask generation module, a foreground style migration module, a foreground mask generation module, a foreground image fusion module and a target image fusion module.

[0032] The edge enhancement line extraction module is used to process the foreground image and the background image. After edge detection and enhancement are performed on the image input to the edge enhancement line extraction module, edge lines are extracted based on edge features to obtain corresponding line images. The line image obtained after the background image is processed by the edge enhancement line extraction module is the background edge feature image, and the line image obtained after the foreground image is processed by the edge enhancement line extraction module is the foreground edge feature image.

[0033] The edge style extraction module is used to extract edge style features from the background edge feature image. The edge style features include texture information, width information and position information of the line image relative to the edge feature in the background edge feature image. The background style extraction module is used to extract background style features from the background image.

[0034] The image deformation control module is used to deform the foreground image and the foreground edge feature image according to the external input image deformation control information, such as translation, rotation, scaling, flipping, stretching, etc., so as to obtain the corresponding foreground deformation feature image and edge deformation feature image.

[0035] The edge style transfer module is used to transfer the style of the edge deformation feature image according to the edge style feature to obtain the style transfer edge feature. The edge mask generation module is used to generate the corresponding edge mask according to the style transfer edge feature. Similarly, the background style transfer module is used to transfer the style of the foreground deformation feature image according to the background style feature to obtain the style transfer foreground feature. The edge mask generation module is used to generate the corresponding foreground mask according to the style transfer foreground feature.

[0036] The foreground image fusion module fuses the style-transferred foreground features and the style-transferred edge features to form a foreground fusion image based on the edge mask; the target image fusion module fuses the foreground fusion image and the background image based on the foreground mask to obtain a target fusion image, which is the output result of this image synthesis system.

[0037] For the present image synthesis system, the method for synthesizing images includes: obtaining a foreground image and a background image, wherein the edge parts between different parts of the foreground image may have edge lines with a line style or may not have corresponding edge lines, and the edge parts between different parts of the background image all have edge lines with a line style.

[0038] Foreground Image I F and background image I B Input into the edge enhancement line extraction module to obtain the corresponding foreground edge feature image F FEL and background edge feature image F BEL The edge enhancement line extraction module includes an edge detection module and a line extraction module. The original image input to the edge enhancement line extraction module is first subjected to edge detection by the edge detection module, and then the original image and the edge detection result are input to the line extraction module. The line extraction module uses the edge detection result to enhance the edge features of the original image, and then extracts edge lines consistent with the edge feature trajectory based on the edge features.

[0039] The original image input to the edge detection module is P0, and the corresponding multi-scale feature P is extracted through the encoder i (i=1,2,3…,n), n is the number of encoder layers, all multi-scale features are processed by the receptive field block (RFB), and the smallest multi-scale feature P n After being optimized by the receiving domain block, the corresponding minimum scale multi-scale optimization feature f is obtained n , that is, f n =RFB(P n ), RFB means processing by receiving domain block, for other scales of P i (i=1,2,3…,n-1), then we need to combine the multi-scale optimization features f of the next layer (smaller scale) i+1 Generate the corresponding multi-scale optimization feature f i (i=1,2,3…,n-1), the specific expression is as follows:

[0040] f i =CONV(RFB(P i )© UP(f i+1 )),

[0041] Among them, CONV represents a convolution operation with a kernel size of 1×1, RFB represents processing through a receptive field block, © represents a concatenation operation, and UP represents an upsampling operation through bilinear interpolation.

[0042] Then calculate the multi-scale edge feature f' i (i=1,2,3…,n), when i=n, ​​the minimum scale multi-scale edge feature f' n The minimum scale multi-scale optimization feature f is used n , f' n =f n ; When i≠n, by multi-scale optimization feature f i(i=1,2,3…,n-1) and the multi-scale edge features f' of the next layer i+1 These two features are input into the edge selection module to obtain the multi-scale edge features f' of the corresponding scale. i (i=1,2,3…,n-1). The edge selection module uses a gating mechanism to adaptively select multi-scale optimization features f i and multi-scale edge features f' i+1 :First, find the multi-scale optimization feature f i and multi-scale edge features f' i+1 The enhanced feature x obtained by adding element by element i , then enhance the feature x i First, a global average pooling operation is performed, and then a fully connected layer containing a ReLU function and a batch normalization operation is used to process the edge selection index b. The edge selection index b is used to calculate the selection weight matrix of the two features. The calculation formula is as follows:

[0043] , ,

[0044] Among them, w i and w i+1 They are the multi-scale optimized features f i and multi-scale edge features f' i+1 The selection weight matrix is, b is the edge selection index, A i and A i+1 are the corresponding multi-scale optimization features f i and multi-scale edge features f' i+1 Adjustable parameters. Get the selection weight matrix w i and w i+1 Then we can optimize the multi-scale features f i and multi-scale edge features f' i+1 Select and obtain the corresponding multi-scale edge feature f' i , the calculation formula is: f' i =w i ⊙f i ⊕w i+1 ⊙f' i+1 .

[0045] While extracting multi-scale edge features, the global feature f0 is extracted through the global feature extractor, and the global feature f0 is combined with the multi-scale edge features f' of each scale. i After fusion, the edge detection result f is generated by the convolution operation of the convolution kernel 1×1 E , the expression is: E = CONV(w0f0⊕w'1f'1⊕w'2f'2⊕…⊕w' n f'n ), where w' i Represents the multi-scale edge features f' corresponding to each scale i The weight of i=1,2,3…,n, w0 represents the weight corresponding to the global feature f0; then the edge detection result f E Output.

[0046] In the line extraction module, first the edge detection result f E Add the original image P0 element by element to obtain the edge enhancement image P containing the edge detection result ES Then the edge enhancement image P ES According to the enhanced edge features, the pixels are classified to extract the corresponding line image P out The line extraction module samples the enhanced edge features and analyzes them pixel by pixel along the edge features. By analyzing the color vector distance between the pixel points in the area around the edge features and the sampled points on the edge features, it determines whether the corresponding pixel points belong to the line where the edge features are located. The calculation formula for the color vector distance is: c =min i ‖C-C i ‖, where min i Indicates the position of the currently found minimum element in the image, d c represents the color vector distance, C represents the color of the analyzed pixel, and C i It represents the i-th color value among several representative colors obtained by the sampling point. By setting the threshold of the color vector distance, it is judged whether the corresponding pixel point belongs to the line where the edge feature is located, thereby extracting the line image P out .

[0047] Background edge feature image F BEL The edge style feature F is obtained by processing the edge style extraction module ES , edge style feature F ES Including background edge feature image F BEL The texture information, width information and position information of the line image relative to the edge features in the background image I B The edge style feature F is also obtained through the background style extraction module BS .

[0048] The image deformation control information is input from the outside (user) to the image deformation control module, and the image deformation control module performs the foreground image I F and foreground edge feature image F FEL Processing is performed to obtain the corresponding foreground deformation feature image F FD and edge deformation feature image F ED .

[0049] The edge style feature F ES and edge deformation feature image F ED The input is sent to the edge style transfer module, which first identifies edge features from the edge deformation feature image and then extracts the edge style feature F ES The texture information, width information and position information in the style are transferred to obtain the style transfer edge feature F STE ; Then the edge mask generation module is based on the style migration edge feature F STE Generate the corresponding edge mask M E , the pixel value of the area corresponding to the edge is set to 0, and the pixel value of other areas is set to 255. At the same time, the background style feature F BS and the foreground deformation feature image F FD Input to the foreground style transfer module, style transfer obtains the style transfer foreground feature F STF ; Then the foreground mask generation module is based on the style migration foreground feature F STF Generate the corresponding foreground mask M F , the pixel value of the area corresponding to the foreground image is set to 0, and the pixel value of other areas is set to 255.

[0050] Edge Mask M E , style transfer foreground feature F STF And style transfer edge feature F STE Input the foreground image fusion module and fuse it to obtain the foreground fusion image I FF , the expression is: FF =M E ⊙F STF ⊕F STE ; Then merge the foreground image I FF , foreground mask M F and background image I B Input the target image fusion module and fuse it to get the target fused image I O , the expression is: O =M F ⊙I B ⊙I FF ⊕I FF .

[0051] In the above modules, the loss corresponding to the edge style extraction module is the edge style loss L Estyle The loss corresponding to the background style extraction module is the background style loss L Bstyle This type of image-based style loss function is a prior art in this field. The edge detection module outputs the edge detection result L E The loss function combined with the weighted binary cross entropy loss and the weighted intersection-over-union loss is used for optimization, namely ; The line extraction module uses the corresponding line extraction loss L LO For optimization, the loss function is:

[0052] ,

[0053] in, represents the expectation of the line image under the distribution of the corresponding sample data, P out represents the line image generated by the sample, Represents the correct line image corresponding to the sample; the output result of the foreground image fusion module is the foreground fusion image I FF Adopt foreground fusion loss L FF For optimization, the corresponding loss function is:

[0054] ,

[0055] in, represents the expectation of the foreground fusion image under the distribution of the corresponding sample data, I FF represents the foreground fusion image generated by sample fusion, Represents the foreground target image corresponding to the sample; for the output result of the target image fusion module, the target fusion image I O Adopt target fusion loss L TF For optimization, the corresponding loss function is:

[0056] ,

[0057] in, represents the expectation of the target fusion image under the distribution of the corresponding sample data, I O represents the target fused image generated by sample fusion, Indicates the correct target image corresponding to the sample.

[0058] Therefore, the total loss of this method is L total For: L total =λ1L Estyle +λ2L Bstyle +λ3L E +λ4 L LO +λ5L FF +λ6L TF , where λ1, λ2, λ3, λ4, λ5 and λ6 are the weights of the corresponding loss function, which are adjustable hyperparameters. The model is optimized by minimizing the above total loss.

[0059] The present invention is described above by way of example in conjunction with the accompanying drawings. It is obvious that the specific implementation of the present invention is not limited to the above-mentioned method. As long as various non-substantial improvements are made using the inventive concept and technical solution of the present invention, or the inventive concept and technical solution are directly applied to other occasions without improvement, they are all within the protection scope of the present invention.

Claims

1. An image synthesis system, characterized in that: include: The edge enhancement line extraction module is used to process the foreground image and the background image. After edge detection and enhancement are performed on the image input to the edge enhancement line extraction module, edge lines are extracted based on edge features to obtain corresponding line images. The line image obtained after the background image is processed by the edge enhancement line extraction module is the background edge feature image, and the line image obtained after the foreground image is processed by the edge enhancement line extraction module is the foreground edge feature image. The edge style extraction module is used to extract edge style features from the background edge feature image; The background style extraction module is used to extract background style features from the background image; The image deformation control module is used to deform the foreground image and the foreground edge feature image according to the external input image deformation control information to obtain the corresponding foreground deformation feature image and edge deformation feature image; The edge style transfer module is used to transfer the style of the edge deformation feature image according to the edge style feature to obtain the style transfer edge feature; The edge mask generation module is used to generate the corresponding edge mask according to the style migration edge features; The background style transfer module is used to transfer the style of the foreground deformation feature image according to the background style features to obtain the style transferred foreground features; The edge mask generation module is used to generate the corresponding foreground mask according to the style migration foreground features; The foreground image fusion module fuses the style-transferred foreground features and the style-transferred edge features to form a foreground fusion image based on the edge mask; The target image fusion module fuses the foreground fusion image and the background image based on the foreground mask to obtain the target fusion image, and the target fusion image is the output result of this image synthesis system; The edge enhancement line extraction module includes an edge detection module and a line extraction module. The original image input to the edge enhancement line extraction module is first subjected to edge detection by the edge detection module, and then the original image and the edge detection result are input into the line extraction module. The line extraction module uses the edge detection result to enhance the edge features of the original image, and then extracts edge lines consistent with the edge feature trajectory based on the edge features.

2. An image synthesis system according to claim 1, characterized in that: The original image input to the edge detection module is P0, and the corresponding multi-scale feature P is extracted through the encoder i (i=1,2,3…,n), n is the number of encoder layers, all multi-scale features are processed by the receiving domain block, and the smallest multi-scale feature P n After being optimized by the receiving domain block, the corresponding minimum scale multi-scale optimization feature f is obtained n , that is, f n =RFB(P n ), RFB means processing by receiving domain blocks, for other scales of P i (i=1,2,3…,n-1), then we need to combine the multi-scale optimization features f of the next layer with a smaller scale i+1 Generate the corresponding multi-scale optimization feature f i (i=1,2,3…,n-1), the specific expression is as follows: f i =CONV(RFB(P i )© UP(f i+1 )), Among them, CONV represents a convolution operation with a kernel size of 1×1, RFB represents processing through a receptive field block, © represents a concatenation operation, and UP represents an upsampling operation through bilinear interpolation; Then calculate the multi-scale edge feature f' i (i=1,2,3…,n), when i=n, ​​the minimum scale multi-scale edge feature f' n The minimum scale multi-scale optimization feature f is used n , f' n =f n ; When i≠n, by multi-scale optimization feature f i (i=1,2,3…,n-1) and the multi-scale edge features f' of the next layer i+1 These two features are input into the edge selection module to obtain the multi-scale edge features f' of the corresponding scale. i (i=1,2,3…,n-1).

3. An image synthesis system according to claim 2, characterized in that: The edge selection module uses a gating mechanism to adaptively select multi-scale optimization features f i and multi-scale edge features f' i+1 :First, find the multi-scale optimization feature f i and multi-scale edge features f' i+1 The enhanced feature x obtained by adding element by element i , then enhance the feature x i First, a global average pooling operation is performed, and then a fully connected layer containing a ReLU function and a batch normalization operation is used to process the edge selection index b. The edge selection index b is used to calculate the selection weight matrix of the two features. The calculation formula is as follows: ， ， Among them, w i and w i+1 They are the multi-scale optimized features f i and multi-scale edge features f' i+1 The selection weight matrix is, b is the edge selection index, A i and A i+1 are the corresponding multi-scale optimization features f i and multi-scale edge features f' i+1 Adjustable parameters; get the selection weight matrix w i and w i+1 Then we can optimize the multi-scale features f i and multi-scale edge features f' i+1 Select and obtain the corresponding multi-scale edge feature f' i , the calculation formula is: f' i =w i ⊙f i ⊕w i+1 ⊙f' i+1 .

4. An image synthesis system according to claim 2, characterized in that: While extracting multi-scale edge features, the global feature f0 is extracted through the global feature extractor, and the global feature f0 is combined with the multi-scale edge features f' of each scale. i After fusion, the edge detection result f is generated by the convolution operation of the convolution kernel 1×1 E , the expression is: E = CONV(w0f0⊕w'1f'1⊕w'2f'2⊕…⊕w' n f' n ), where w' i Represents the multi-scale edge features f' corresponding to each scale i The weight of i=1,2,3…,n, w0 represents the weight corresponding to the global feature f0; then the edge detection result f E Output.

5. The image synthesis system according to claim 1, characterized in that: In the line extraction module, firstly transform the edge detection result f E Add the original image P0 element by element to obtain the edge enhancement image P containing the edge detection result ES Then the edge enhancement image P ES According to the enhanced edge features, the pixels are classified to extract the corresponding line image P out The line extraction module samples the enhanced edge features and analyzes them pixel by pixel along the edge features. By analyzing the color vector distance between the pixel points in the area around the edge features and the sampling points on the edge features, it can be determined whether the corresponding pixel points belong to the line where the edge features are located. The calculation formula of the color vector distance is: c =min i ‖C-C i ‖, where min i Indicates the position of the currently found minimum element in the image, d c represents the color vector distance, C represents the color of the analyzed pixel, and C i It represents the i-th color value among several representative colors obtained by the sampling point; by setting the threshold of the color vector distance, it is judged whether the corresponding pixel point belongs to the line where the edge feature is located, thereby extracting the line image P out .

6. An image synthesis system according to claim 1, characterized in that: The image deformation control information is input from the outside to the image deformation control module, and the image deformation control module performs the foreground image I F and foreground edge feature image F FEL Processing is performed to obtain the corresponding foreground deformation feature image F FD and edge deformation feature image F ED .

7. An image synthesis system according to claim 6, characterized in that: Edge style feature F ES Including background edge feature image F BEL The texture information, width information of the line image and the position information of the line image relative to the edge feature; the edge style feature F ES and edge deformation feature image F ED The input is sent to the edge style transfer module, which first identifies edge features from the edge deformation feature image and then extracts the edge style feature F ES The texture information, width information and position information in the style are transferred to obtain the style transfer edge feature F STE ; The edge mask generation module is based on the style migration edge feature F STE Generate the corresponding edge mask M E ; Edge mask M E , style transfer foreground feature F STF And style transfer edge feature F STE Input the foreground image fusion module and fuse it to obtain the foreground fusion image I FF , the expression is: FF =M E ⊙F STF ⊕F STE .

8. An image synthesis system according to claim 7, characterized in that: The background style feature F BS and the foreground deformation feature image F FD Input to the foreground style transfer module, style transfer obtains the style transfer foreground feature F STF ; Then the foreground mask generation module is based on the style migration foreground feature F STF Generate the corresponding foreground mask M F ; Then merge the foreground image I FF , foreground mask M F and background image I B Input the target image fusion module and fuse it to get the target fused image I O , the expression is: O =M F ⊙I B ⊙I FF ⊕I FF .

Images