Target mask-based image double-stage smooth fusion generation method and device
By adopting the two-stage smooth fusion generation method based on the target mask in the image generation of training sample, using the progressive smooth growth adversarial network and smooth fusion network, the problems of poor image diversity, fixed targets and single background in the prior art are solved, and higher image diversity and adaptability are achieved.
Patent Information
- Application Number
- CN202411883537.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-19
- Publication Date
- 2025-04-11
AI Technical Summary
The images generated by the existing training sample image generation methods have poor diversity, fixed goals and single background, resulting in serious algorithm overfitting and bias.
Using the two-stage smooth fusion generation method of image based on the target mask, the progressive smooth growth adversarial network is used to smoothly advance learning from low-resolution images to high-resolution images, generate an initial target image, and replace the background of the initial target image through a smooth fusion network based on the target mask, and output the final target image.
It improves the diversity of generated images, solves the problem of fixed goals and single background, and enhances the diversity and adaptability of image generation.
Smart Images
Figure CN119809972A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of deep learning technology, and in particular to a method and device for generating a two-stage smooth fusion image based on a target mask. Background Art
[0002] In the training of target recognition algorithms, the quality of training samples has an increasingly greater impact on the performance of intelligent algorithms, and the size of training samples seriously affects the environmental adaptability of intelligent target recognition methods. In order to solve the problems of too few training samples, unbalanced samples, and the resulting algorithm overfitting and serious bias, generative adversarial networks are currently used to achieve batch generation of training samples. However, traditional generative adversarial networks are mostly trained and learned directly on original images, resulting in poor diversity of generated images, fixed targets, and single backgrounds.
[0003] Therefore, there is an urgent need to provide a new method for generating training sample images. Summary of the invention
[0004] In order to solve the problems that the images generated by the existing training sample image generation method have poor diversity, fixed targets and single background, the embodiments of the present invention provide a method and device for generating an image by two-stage smooth fusion based on a target mask.
[0005] On the one hand, a method for generating a two-stage smooth fusion image based on a target mask is provided, the method comprising:
[0006] Using a progressive smooth growth adversarial network, the original sample image is smoothly advanced from a low-resolution image to a high-resolution image according to a set step size to generate several corresponding initial target images;
[0007] The initial target image is input into a pre-trained target mask-based smooth fusion network to extract the target mask of the initial target image, replace the background of the initial target image, and output the final target image; wherein the smooth fusion network includes a foreground fine-grained segmentation network, a large data set pre-trained skeleton network and an image smooth fusion network connected in series.
[0008] On the other hand, a device for generating a two-stage smooth fusion image based on a target mask is provided, which is used to implement the steps described in any method embodiment of the specification, and the device includes:
[0009] A generation unit is used to use a progressive smooth growth adversarial network to smoothly advance the learning of the original sample image from the low-resolution image to the high-resolution image according to a set step size, and generate a number of corresponding initial target images;
[0010] A fusion unit is used to input the initial target image into a pre-trained smooth fusion network based on a target mask to extract the target mask of the initial target image, replace the background of the initial target image, and output the final target image; wherein the smooth fusion network includes a foreground fine-grained segmentation network, a large data set pre-trained skeleton network and an image smooth fusion network connected in series in sequence.
[0011] On the other hand, a computer device is provided, which includes a memory and a processor, wherein the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory to implement the steps of the above-mentioned method.
[0012] On the other hand, a computer-readable storage medium is provided, wherein a computer program is stored in the storage medium, and when the computer program is executed by a processor, the steps of the above-mentioned method are implemented.
[0013] On the other hand, a computer program product is provided, comprising a computer program, wherein the computer program implements the steps of the above method when executed by a processor.
[0014] The technical solution provided by the present invention can at least bring the following beneficial effects:
[0015] The traditional generative adversarial network is improved, namely the progressive smooth growth adversarial network, which smoothly advances the original sample image from the low-resolution image to the high-resolution image according to the set step size, and ensures the diversity of the generated images on the basis of progressive generative training; after generating the initial target image, the background of the initial target image is replaced by the smooth fusion network based on the target mask, and a smooth target image is output. Therefore, this scheme can solve the problem of fixed targets and single backgrounds, and can improve image diversity. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0017] Figure 1 It is a flow chart of a method for generating a two-stage smooth fusion of an image based on a target mask provided by an embodiment of the present invention;
[0018] Figure 2 It is a structural diagram of a progressive smooth growth adversarial network provided by an embodiment of the present invention;
[0019] Figure 3is a structural diagram of a smooth fusion network based on target mask provided by an embodiment of the present invention;
[0020] Figure 4 It is a structural diagram of a device for generating a two-stage smooth fusion of an image based on a target mask provided by an embodiment of the present invention;
[0021] Figure 5 It is a hardware architecture diagram of a computer device provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0022] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.
[0023] The specific implementation of the above concept is described below.
[0024] Please refer to Figure 1 , an embodiment of the present invention provides a two-stage smooth fusion generation method for an image based on a target mask, the method comprising:
[0025] Step 100: using a progressive smooth growth adversarial network, smoothly advancing the learning of the original sample image from the low-resolution image to the high-resolution image according to a set step size, and generating a number of corresponding initial target images;
[0026] Step 102: Input the initial target image into a pre-trained target mask-based smooth fusion network to extract the target mask of the initial target image, replace the background of the initial target image, and output the final target image; wherein the smooth fusion network includes a foreground fine-grained segmentation network, a large data set pre-trained skeleton network, and an image smooth fusion network that are sequentially connected in series.
[0027] In the embodiment of the present invention, the traditional generative adversarial network is improved, that is, the progressive smooth growth adversarial network, which smoothly advances the learning of the original sample image from the low-resolution image to the high-resolution image according to the set step size, and ensures the diversity of the generated image on the basis of progressive generative training; after the initial target image is generated, the background of the initial target image is replaced by the smooth fusion network based on the target mask, and a smooth target image is output. Therefore, this solution can solve the problem of fixed target and single background, and can improve the diversity of images.
[0028] Described below Figure 1 How the various steps are performed.
[0029] For step 100:
[0030] In the embodiment of the present invention, reference may be made to Figure 2 The structural diagram of the progressive smooth growing adversarial network shown in the figure, the progressive smooth growing adversarial network includes a generator module and a discriminator module, the generator module contains several growing learning sub-modules, and the convolution channels of these growing learning sub-modules are gradually increased until they are increased to the same size as the original sample image; the discriminator module contains several growing discriminant sub-modules corresponding to the growing learning sub-modules one by one, and the growing discriminant sub-module is used to calculate the loss function based on the transition image and the original sample image output by the corresponding growing learning sub-module and feed it back to the corresponding growing learning sub-module.
[0031] In this embodiment, for the sake of clarity, Figure 2 In the figure, (a) the initial state, (b) the 4-8 growth process and (c) the 8×8 final state are the first growth learning submodule and the first growth discrimination submodule, (d) the 8-16 growth process and (e) the 16×16 final state are the second growth learning submodule and the first growth discrimination submodule. It can be seen that Figure 2 Only the final states of the first, second and last growth learning submodules are shown, and the middle growth learning submodules are omitted. It can be seen that the convolution channels of the growth learning submodules of the generator module gradually increase until the transition image output by the last growth learning submodule is equal to the size of the original sample image. This progressive smooth growth adversarial network smoothly advances the learning of the original sample image from the low-resolution image to the high-resolution image according to the set step size. Compared with the traditional generative adversarial network, which mostly directly trains and learns the original image, this scheme can improve the diversity of generated images.
[0032] In some implementations, step 100 may include:
[0033] S1, the network parameters of the initial convolutional block of the first growth learning submodule are initialized using the standard normal distribution;
[0034] S2, based on the small batch standard deviation structure of the initial scale coefficient, trains the network parameters of the high-resolution convolution block of the first growth learning submodule, and outputs the transition image to the corresponding first growth discrimination submodule; wherein the number of convolution channels of the high-resolution convolution block of the first growth learning submodule is greater than that of the initial convolution block;
[0035] S3, the first growth discrimination submodule calculates the loss function based on the transition image and the original sample image and feeds it back to the first growth learning submodule;
[0036] S4, gradually increasing the scale factor of the first growth learning submodule to repeat steps S2-S4 multiple times until the scale factor is 1, completing multiple training of the initial convolution block and the high-resolution convolution block of the first growth learning submodule;
[0037] S5, inherits the network parameters of the high-resolution convolutional block of the first growth learning submodule to the initial convolutional block of the second growth learning submodule;
[0038] S6, based on the small batch standard deviation structure of the initial scale coefficient, trains the network parameters of the high-resolution convolution block of the second growth learning submodule, and outputs the transition image to the corresponding second growth discrimination submodule; wherein the number of convolution channels of the high-resolution convolution block of the second growth learning submodule is greater than that of the initial convolution block of the second growth learning submodule;
[0039] S7, the second growth discrimination submodule calculates the loss function based on the transition image and the original sample image and feeds it back to the second growth learning submodule;
[0040] S8, gradually increase the scale factor of the second growth learning submodule, repeat steps S2-S4 multiple times until the scale factor is 1, and complete multiple training of the initial convolution block and the high-resolution convolution block of the second growth learning submodule;
[0041] S9, and so on, using the same training method as the first growth learning submodule and the second growth learning submodule, train the remaining growth learning submodules in turn. When the loss function of the last growth discrimination submodule is less than the set threshold, the transition image output by the last growth learning submodule is used as the initial target image.
[0042] In this embodiment, reference can be made to Figure 2 , (a) the initial state, (b) the 4-8 growth process and (c) the 8×8 final state are the first growth learning submodule and the first growth discriminant submodule. In the first growth learning submodule of the (b) 4-8 growth process, the initial convolution block is 4×4 and the high-resolution convolution block is 8×8; in the second growth learning submodule of the (d) 8-16 growth process, the initial convolution block is 8×8 and the high-resolution convolution block is 16×16, and so on. Assuming that the original sample image size is 256×256, there are also several growth learning submodules and growth discriminant submodules in the middle. The initial convolution block of the last growth learning submodule is 128×128 and the high-resolution convolution block is 256×256.
[0043] Each growth learning submodule will be trained multiple times, that is, the (b) 4-8 growth process in the first growth learning submodule is repeated several times, the first proportional coefficient α is 0, and it increases according to the step size. For example, 0.2 is the step size, the second execution proportional coefficient α is 0.2, until the sixth execution proportional coefficient α is 1, the (b) 4-8 growth process is equivalent to the (c) 8×8 final state structure. It can be seen that the first growth learning submodule and the first growth discriminator submodule of the generator and the discriminator are repeated 6 times to obtain the final network parameters of the high-resolution convolution block of the first growth learning submodule, and the final network parameters are inherited to the initial convolution block of the second growth learning submodule. By analogy, each growth learning submodule is repeated multiple times until the proportional coefficient is 1, completing the training of the growth learning submodule, and realizing smooth advancement of learning from low-resolution images to high-resolution images according to the set step size. The set step size here refers to the size growth step size of the convolution layer of the growth learning submodule.
[0044] In some embodiments, the growth learning submodule outputs a transition image in the following manner:
[0045] Up-sampling the output of the initial convolutional block of the growth learning submodule to obtain intermediate features;
[0046] The intermediate features are input into two branches respectively, the first branch uses the intermediate features to generate a first image, and the second branch inputs the intermediate features into the high-resolution convolution block of the growth learning submodule for feature extraction to generate a second image;
[0047] The first image and the second image are weightedly summed in a ratio of 1-α:α to obtain a transition image, wherein α is a proportional coefficient.
[0048] In this embodiment, the growth learning submodule uses a small batch standard deviation structure for training to obtain a transition image. The small batch standard deviation structure is the structure of this embodiment, which can be referred to Figure 2 , including an initial convolution block, an upsampling layer, two branches and a weighted summation layer, wherein one branch is used to generate the first image using the intermediate features, and the other branch contains a high-resolution convolution block. The progressive image generation method using a small batch standard deviation structure can ensure the diversity of generated images based on progressive generation training.
[0049] In some embodiments, reference may be made to Figure 2 , the growth discriminant submodule calculates the loss function as follows:
[0050] The transition image is input into two branches respectively, the third branch downsamples the transition image to obtain a third image, and the fourth branch uses a convolution layer of the same size as the high-resolution convolution block of the corresponding growth learning submodule to extract features from the transition image and then downsamples it to obtain a fourth image;
[0051] The third image and the fourth image are weightedly summed in a ratio of 1-α:α, and the weighted summation result is gradually downsampled until it is downsampled to the same size as the initial convolution block of the first growth learning submodule to calculate the loss function with the original sample image; where α is the proportional coefficient.
[0052] In this embodiment, the growth discriminant submodule is also trained using the standard deviation structure. The standard deviation structure in the growth discriminant submodule can refer to Figure 2 , from top to bottom, it includes two branches, a weighted layer, several convolution blocks with gradually decreasing convolution kernels, and a downsampling layer between every two convolution blocks. Among them, the third branch downsamples the transition image to obtain the third image, and the fourth branch contains convolution layers and downsampling layers of the same size as the high-resolution convolution blocks of the corresponding growth learning submodule. The growth discriminant submodule uses the standard deviation structure for decoding until it is downsampled to the same size as the initial convolution block of the first growth learning submodule, so as to calculate the loss function with the original sample image and improve the training accuracy of the corresponding growth learning submodule.
[0053] Regarding step 102:
[0054] In some implementations, step 102 may include:
[0055] Utilize the foreground fine-grained segmentation network to extract the object mask of the initial object image;
[0056] Input the target mask and the initial target image into a skeleton network pre-trained on a large dataset to replace the background of the initial target image and generate a synthetic image;
[0057] The synthesized image and the initial target image are input into the image smoothing fusion network to use the harmonization algorithm to smoothly fuse the target and the background and output the final target image.
[0058] In this embodiment, reference can be made to Figure 3 The smooth fusion network structure diagram uses the foreground fine-grained segmentation network to extract the target mask. The large data set pre-trained skeleton network pre-learns multiple backgrounds. The corresponding background is replaced by artificial or automatic settings to replace the background of the initial target image to generate a synthetic image. Finally, the smooth fusion network uses a harmonization algorithm to smoothly fuse the target and the background to generate the final target image.
[0059] To verify the effectiveness of the embodiment of the present invention, 110 real original sample images of 3 types of foreground targets and 3 types of backgrounds were used as input, and 752 256*256 images were generated using the progressive smooth growth adversarial network and the smooth fusion network in turn. The cross entropy of these generated images was less than 1%, and the cosine similarity with the actual images reached more than 85%, and the average image similarity was 92%. It can be seen that this solution can solve the problem of fixed targets and single backgrounds, and can improve image diversity.
[0060] Please refer to Figure 4 The embodiment of the present invention provides a device for generating a two-stage smooth fusion image based on a target mask, which is used to implement the steps of any method embodiment in the specification, and the device includes:
[0061] A generating unit 401 is used to use a progressive smooth growth adversarial network to smoothly advance the learning original sample image from the low-resolution image to the high-resolution image according to a set step size, and generate a plurality of corresponding initial target images;
[0062] The fusion unit 402 is used to input the initial target image into a pre-trained smooth fusion network based on the target mask to extract the target mask of the initial target image, replace the background of the initial target image, and output the final target image; wherein the smooth fusion network includes a foreground fine-grained segmentation network, a large data set pre-trained skeleton network and an image smooth fusion network connected in series.
[0063] In one embodiment of the present invention, the progressive smooth growing adversarial network in the generation unit 401 includes a generator module and a discriminator module, the generator module contains a number of growing learning sub-modules, and the convolution channels of these growing learning sub-modules are gradually increased until they are increased to the same size as the original sample image; the discriminator module contains a number of growing discriminant sub-modules corresponding one by one to the growing learning sub-modules, and the growing discriminant sub-module is used to calculate the loss function based on the transition image and the original sample image output by the corresponding growing learning sub-module and feed it back to the corresponding growing learning sub-module.
[0064] In one embodiment of the present invention, the generating unit 401 is used to execute:
[0065] S1, the network parameters of the initial convolutional block of the first growth learning submodule are initialized using the standard normal distribution;
[0066] S2, based on the small batch standard deviation structure of the initial scale coefficient, trains the network parameters of the high-resolution convolution block of the first growth learning submodule, and outputs the transition image to the corresponding first growth discrimination submodule; wherein the number of convolution channels of the high-resolution convolution block of the first growth learning submodule is greater than that of the initial convolution block;
[0067] S3, the first growth discrimination submodule calculates the loss function based on the transition image and the original sample image and feeds it back to the first growth learning submodule;
[0068] S4, gradually increasing the scale factor of the first growth learning submodule to repeat steps S2-S4 multiple times until the scale factor is 1, completing multiple training of the initial convolution block and the high-resolution convolution block of the first growth learning submodule;
[0069] S5, inherits the network parameters of the high-resolution convolutional block of the first growth learning submodule to the initial convolutional block of the second growth learning submodule;
[0070] S6, based on the small batch standard deviation structure of the initial scale coefficient, trains the network parameters of the high-resolution convolution block of the second growth learning submodule, and outputs the transition image to the corresponding second growth discrimination submodule; wherein the number of convolution channels of the high-resolution convolution block of the second growth learning submodule is greater than that of the initial convolution block of the second growth learning submodule;
[0071] S7, the second growth discrimination submodule calculates the loss function based on the transition image and the original sample image and feeds it back to the second growth learning submodule;
[0072] S8, gradually increase the scale factor of the second growth learning submodule, repeat steps S2-S4 multiple times until the scale factor is 1, and complete multiple training of the initial convolution block and the high-resolution convolution block of the second growth learning submodule;
[0073] S9, and so on, using the same training method as the first growth learning submodule and the second growth learning submodule, train the remaining growth learning submodules in turn. When the loss function of the last growth discrimination submodule is less than the set threshold, the transition image output by the last growth learning submodule is used as the initial target image.
[0074] In one embodiment of the present invention, the growth learning submodule in the generation unit 401 outputs the transition image in the following manner:
[0075] Up-sampling the output of the initial convolutional block of the growth learning submodule to obtain intermediate features;
[0076] The intermediate features are input into two branches respectively, the first branch uses the intermediate features to generate a first image, and the second branch inputs the intermediate features into the high-resolution convolution block of the growth learning submodule for feature extraction to generate a second image;
[0077] The first image and the second image are weightedly summed in a ratio of 1-α:α to obtain a transition image, wherein α is a proportional coefficient.
[0078] In one embodiment of the present invention, the growth discrimination submodule of the generation unit 401 calculates the loss function in the following manner:
[0079] The transition image is input into two branches respectively, the third branch downsamples the transition image to obtain a third image, and the fourth branch uses a convolution layer of the same size as the high-resolution convolution block of the corresponding growth learning submodule to extract features from the transition image and then downsamples it to obtain a fourth image;
[0080] The third image and the fourth image are weightedly summed in a ratio of 1-α:α, and the weighted summation result is gradually downsampled until it is downsampled to the same size as the initial convolution block of the first growth learning submodule to calculate the loss function with the original sample image; where α is the proportional coefficient.
[0081] In one embodiment of the present invention, the fusion unit 402 is used to perform:
[0082] Utilize the foreground fine-grained segmentation network to extract the object mask of the initial object image;
[0083] Input the target mask and the initial target image into a skeleton network pre-trained on a large dataset to replace the background of the initial target image and generate a synthetic image;
[0084] The synthesized image and the initial target image are input into the image smoothing fusion network to use the harmonization algorithm to smoothly fuse the target and the background and output the final target image.
[0085] It should be noted that the above-mentioned embodiment provides a two-stage smooth fusion generation device for an image based on a target mask, and only uses the division of the above-mentioned functional units as an example. In practical applications, the above-mentioned functions can be assigned to different functional units as needed, that is, the internal structure of the device can be divided into different functional units to complete all or part of the functions described above. In addition, the above-mentioned device embodiment and the method embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.
[0086] The embodiment of the present application also provides a computer device, please refer to Figure 5 The computer device includes a processor and a memory, wherein at least one instruction, at least one program, a code set or an instruction set is stored in the memory, and the at least one instruction, at least one program, a code set or an instruction set is loaded and executed by the processor to implement the two-stage smooth fusion generation method of the image based on the target mask provided by the above-mentioned method embodiments.
[0087] An embodiment of the present application also provides a computer-readable storage medium, on which is stored at least one instruction, at least one program, code set or instruction set, and the at least one instruction, at least one program, code set or instruction set is loaded and executed by a processor to implement the two-stage smooth fusion generation method of an image based on a target mask provided in the above-mentioned method embodiments.
[0088] An embodiment of the present application also provides a computer program product, which includes a computer program. A processor of a computer device reads the computer program from a computer-readable storage medium, and the processor executes the computer program, so that the computer device executes any of the above-mentioned methods for generating a two-stage smooth fusion image based on a target mask.
[0089] For the convenience of description, the above system or device is described as being divided into various modules or units according to their functions. Of course, when implementing the present application, the functions of each unit can be implemented in the same or multiple software and / or hardware.
[0090] It can be known from the description of the above implementation methods that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of each embodiment of the present application or some parts of the embodiments.
[0091] Finally, it should be noted that, in this article, relational terms such as first, second, third and fourth are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.
[0092] The above are only preferred implementations of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.
Claims
1. A two-stage smooth fusion generation method for an image based on a target mask, characterized in that: The method comprises: Using a progressive smooth growth adversarial network, the original sample image is smoothly advanced from a low-resolution image to a high-resolution image according to a set step size to generate several corresponding initial target images; The initial target image is input into a pre-trained target mask-based smooth fusion network to extract the target mask of the initial target image, replace the background of the initial target image, and output the final target image; wherein the smooth fusion network includes a foreground fine-grained segmentation network, a large data set pre-trained skeleton network and an image smooth fusion network connected in series.
2. The method according to claim 1, characterized in that The progressive smooth growing adversarial network includes a generator module and a discriminator module, the generator module contains a number of growing learning sub-modules, and the convolution channels of these growing learning sub-modules are gradually increased until they are increased to the same size as the original sample image; the discriminator module contains a number of growing discriminant sub-modules corresponding one-to-one to the growing learning sub-modules, and the growing discriminant sub-module is used to calculate the loss function based on the transition image output by the corresponding growing learning sub-module and the original sample image, and feed it back to the corresponding growing learning sub-module.
3. The method according to claim 2, characterized in that The progressive smooth growth adversarial network is used to smoothly advance the learning of the original sample image from the low-resolution image to the high-resolution image according to the set step size, and generate several corresponding initial target images, including: S1, the network parameters of the initial convolutional block of the first growth learning submodule are initialized using the standard normal distribution; S2, based on the small batch standard deviation structure of the initial scale coefficient, trains the network parameters of the high-resolution convolution block of the first growth learning submodule, and outputs the transition image to the corresponding first growth discrimination submodule; wherein the number of convolution channels of the high-resolution convolution block of the first growth learning submodule is greater than that of the initial convolution block; S3, the first growth discrimination submodule calculates the loss function based on the transition image and the original sample image and feeds it back to the first growth learning submodule; S4, gradually increasing the scale factor of the first growth learning submodule to repeat steps S2-S4 multiple times until the scale factor is 1, completing multiple training of the initial convolution block and the high-resolution convolution block of the first growth learning submodule; S5, inheriting the network parameters of the high-resolution convolution block of the first growth learning submodule to the initial convolution block of the second growth learning submodule; S6, based on the small batch standard deviation structure of the initial scale coefficient, trains the network parameters of the high-resolution convolution block of the second growth learning submodule, and outputs the transition image to the corresponding second growth discrimination submodule; wherein the number of convolution channels of the high-resolution convolution block of the second growth learning submodule is greater than that of the initial convolution block of the second growth learning submodule; S7, the second growth discrimination submodule calculates a loss function based on the transition image and the original sample image and feeds the loss function back to the second growth learning submodule; S8, gradually increase the scale factor of the second growth learning submodule, repeat steps S2-S4 multiple times until the scale factor is 1, and complete multiple training of the initial convolution block and the high-resolution convolution block of the second growth learning submodule; S9, and so on, using the same training method as the first growth learning submodule and the second growth learning submodule, train the remaining growth learning submodules in turn. When the loss function of the last growth discrimination submodule is less than the set threshold, the transition image output by the last growth learning submodule is used as the initial target image.
4. The method according to claim 3, characterized in that The growth learning submodule outputs the transition image in the following way: Up-sampling the output of the initial convolutional block of the growth learning submodule to obtain intermediate features; The intermediate features are input into two branches respectively, the first branch uses the intermediate features to generate a first image, and the second branch inputs the intermediate features into the high-resolution convolution block of the growth learning submodule for feature extraction to generate a second image; The first image and the second image are weightedly summed in a ratio of 1-α:α to obtain a transition image; wherein α is the ratio coefficient.
5. The method according to claim 3, characterized in that: The growth discriminant module calculates the loss function as follows: The transition image is input into two branches respectively, the third branch downsamples the transition image to obtain a third image, and the fourth branch uses a convolution layer of the same size as the high-resolution convolution block of the corresponding growth learning submodule to extract features from the transition image and then downsamples it to obtain a fourth image; The third image and the fourth image are weightedly summed in a ratio of 1-α:α, and the weighted sum result is gradually downsampled until it is downsampled to the same size as the initial convolution block of the first growth learning submodule, so as to calculate the loss function with the original sample image; wherein α is the proportional coefficient.
6. The method according to any one of claims 1 to 5, characterized in that The step of inputting the initial target image into a pre-trained smooth fusion network based on a target mask to extract a target mask of the initial target image, replacing the background of the initial target image, and outputting a final target image comprises: Extracting a target mask of the initial target image using the foreground fine-grained segmentation network; Inputting the target mask and the initial target image into the large data set pre-trained skeleton network to replace the background of the initial target image and generate a synthetic image; The synthesized image and the initial target image are input into the image smoothing fusion network to smoothly fuse the target and the background using a harmonization algorithm, and a final target image is output.
7. A device for generating a two-stage smooth fusion image based on a target mask, used to implement the steps of any of the methods described in claims 1 to 6, characterized in that: The device comprises: A generation unit is used to use a progressive smooth growth adversarial network to smoothly advance the learning of the original sample image from the low-resolution image to the high-resolution image according to a set step size, and generate a number of corresponding initial target images; A fusion unit is used to input the initial target image into a pre-trained smooth fusion network based on a target mask to extract the target mask of the initial target image, replace the background of the initial target image, and output the final target image; wherein the smooth fusion network includes a foreground fine-grained segmentation network, a large data set pre-trained skeleton network and an image smooth fusion network connected in series in sequence.
8. A computer device, characterized in that: The computer device includes a memory and a processor, the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory to implement the steps of any one of the methods described in claims 1-6.
9. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method described in any one of claims 1 to 6 are implemented.
10. A computer program product, characterized in that The method comprises a computer program, wherein when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.