Method for training an image generation model, method for executing a service, device, and medium

Abstract

The present invention provides a method for training an image generation model, a service execution method, an apparatus, and a medium that meet the need for training a model that meets service requirements with a limited training set and improves the overall training efficiency of the model. [Solution] A method for training an image generation model includes acquiring an image, performing a noise addition process on the acquired original image to acquire a noise-added image, inputting the noise-added image to a first image generation model, performing noise removal on the noise-added image using the first image generation model to obtain a restored image, determining image foreground features extracted from the restored image, and training the first image generation model with an optimization goal of minimizing the difference between image foreground features corresponding to the original image and image foreground features extracted from the restored image.

Description

[Technical field]

[0001] The present invention relates to the field of computer vision, and in particular to a method, an apparatus, and a medium for training an image generation model. [Background technology]

[0002] With the rapid development of the field of computer vision, models based on deep learning technology are being used in various fields, such as the image recognition field, which judges the content of an image according to the user's needs, the medical field, which generates images of lesions based on the obtained lesion information of a patient, and the computer game field, which generates game images according to the player's exploration status.

[0003] However, in the existing model training technology, in order to train a model that meets the service requirements, it is necessary to use a sufficient amount of image data as training data in the training process of the model. If the amount of image data is insufficient, it is difficult for the existing technology to train a model that meets the service requirements. For example, the higher the expressiveness and realism of the artistic image generated by the artistic image generation model, the more image data is required in the training process, and if the image data is insufficient, the expressiveness and realism of the artistic image generated by the trained artistic image generation model cannot meet the requirements.

[0004] Therefore, how to train a model that meets service requirements based on image data that is insufficient is an urgent issue. Summary of the Invention [Problem to be solved by the invention]

[0005] SUMMARY OF THE PRESENT DISCLOSURE In order to partially solve the above problems existing in the prior art, the present invention provides a method for training an image generation model, a service execution method, an apparatus, and a medium. [Means for solving the problem]

[0006] The technical solutions adopted in the present invention are as follows:

[0007] The method for training an image generation model provided by the present invention includes the steps of: acquiring an original image; performing a noise addition process on the original image to obtain a noise-added image; inputting the noise-added image and a number of times the noise-added image is noise-added to a first image generation model, predicting a superimposed noise signal used when performing a noise addition process of the number of times on the original image to convert it into the noise-added image using the first image generation model until a restored image is obtained, predicting a k-1th transition image before performing the k-th noise addition process based on the superimposed noise signal, predicting a k-2th transition image before performing the k-1th noise addition process based on the superimposed noise signal and the k-1th transition image, and determining image foreground features extracted from the restored image, where k is a positive integer not exceeding the number of times, the image foreground features are for representing morphological features of a target object in an image, and the image foreground features do not include detailed physical features for representing the target object; training the first image generation model with an optimization goal of minimizing a difference between image foreground features corresponding to the original image and image foreground features extracted from the restored image.

[0008] Optionally, the step of performing a noise addition process on the original image to obtain a noise-added image includes: The method includes a step of inputting the original image and the number of noise signals into a second image generation model constructed in advance, and causing the second image generation model to output a noise-added image of the original image in which noise addition processing has been performed a number of times corresponding to the number of noise signals.

[0009] Optionally, constructing the second image generation model comprises: obtaining a sample image; a step of adding noise to a noise-added image to which noise has been added with an (N-1)th noise signal using an Nth noise signal to obtain a noise-added image to which noise has been added with the Nth noise signal, where N is a positive integer of 1 or more, and the noise-added image to which noise has been added with a 0th noise signal is the sample image; a step of determining a conversion relationship from the noise-added image to which noise has been added with the Nm-th noise signal to the noise-added image to which noise has been added with the N-th noise signal, based on the noise-added image to which noise has been added with the Nm-th noise signal, the N-th noise signal, and the N-m+1-th noise signal, where m is a positive integer smaller than N; and constructing the second image generation model based on the transformation relationship.

[0010] The service execution method provided by the present invention comprises: acquiring an initial image; inputting the initial image into a pre-trained image generation model and outputting a target image, the image generation model being a model obtained by training using the training method described above; constructing a training set based on the initial image and the target image, training a predetermined designated model using the training set, and executing a service using the trained designated model.

[0011] The image generation model training device provided by the present invention comprises: an acquisition module for acquiring an original image; a noise addition module for performing a noise addition process on the original image to obtain a noise-added image; an input module for inputting the noise-added image and a number of times the noise-added image is noise-added to a first image generation model, predicting a superimposed noise signal used when performing a noise addition process of the number of times on the original image to convert it into the noise-added image using the first image generation model until a restored image is obtained, predicting a k-1th transition image before performing a k-th noise addition process based on the superimposed noise signal, predicting a k-2th transition image before performing the k-1th noise addition process based on the superimposed noise signal and the k-1th transition image, and determining an image foreground feature extracted from the restored image; and a training module for training the first image generation model with an optimization goal of minimizing a difference between image foreground features corresponding to the original image and image foreground features extracted from the restored image.

[0012] Optionally, the noise addition module specifically comprises: The original image and the number of noise signals are input to a pre-constructed second image generation model, and the second image generation model is used to output a noise-added image of the original image in which noise addition processing has been performed a number of times corresponding to the number of noise signals.

[0013] Optionally, the noise addition module specifically comprises: Get a sample image, using the N-th noise signal, noise is added to a noise-added image to which noise has been added with the (N-1)-th noise signal, to obtain a noise-added image to which noise has been added with the N-th noise signal, N being a positive integer of 1 or more, and the noise-added image to which noise has been added with the 0-th noise signal is the sample image; determining a conversion relationship from the noise-added image to which noise has been added with the Nm-th noise signal to the noise-added image to which noise has been added with the N-th noise signal, based on the noise-added image to which noise has been added with the Nm-th noise signal, the N-th noise signal, and the N-m+1-th noise signal, where m is a positive integer less than N; It is used to construct the second image generation model based on the transformation relationship.

[0014] The service execution device provided by the present invention comprises: an acquisition module for acquiring an initial image; an input module for inputting the initial image into a pre-trained image generation model and outputting a target image, the image generation model being a model obtained by training using the training method; and a training module for constructing a training set based on the initial image and the target image, training a predetermined designated model using the training set, and executing a service using the trained designated model.

[0015] The present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the above-described method for training an image generation model.

[0016] The present invention provides an electronic device including a processor and a computer program stored in a memory and operable on the processor, the electronic device performing the above-mentioned method for training an image generation model when the processor executes the computer program. Effect of the Invention

[0017] At least one of the above technical solutions adopted by the present invention can achieve the following beneficial effects:

[0018] In the method for training an image generation model provided by the present invention, a dedicated device performs a noise addition process on an acquired original image using a second image generation model, inputs the noise-added image into a first image generation model, performs noise removal on the noise-added image using the first image generation model to obtain a restored image, determines image foreground features extracted from the restored image, and trains the first image generation model with an optimization goal of minimizing the difference between the image foreground features corresponding to the original image and the image foreground features extracted from the restored image. The trained first image generation model outputs a restored image having image foreground features similar to those of the original image based on the input image after noise addition.

[0019] In the service execution method provided by the present invention, after obtaining an initial image, the initial image is input into an image generation model pre-trained using the above-mentioned training method, a target image is output, a training set is constructed based on the initial image and the target image, a predetermined designated model is trained using the training set, and the service is executed using the trained designated model.

[0020] As can be seen from the above method, by adding noise to the image data of a training set and inputting it into a pre-trained image generation model to remove the noise, an image can be generated that resembles the foreground features of the image before the noise was added but has different other details. The generated image can be used as image data to extend the training set, thereby meeting the need to train a model that meets service requirements with a limited training set, and improving the overall training efficiency of the model. [Brief description of the drawings]

[0021] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention, and the schematic embodiments of the present invention and their explanations are used to interpret the present invention and do not constitute undue limitations on the present invention. [Figure 1] FIG. 1 is a schematic diagram showing the flow of a method for training an image generation model provided by the present invention. [Diagram 2] FIG. 2 is a schematic diagram illustrating the flow of a service execution method provided by the present invention. [Diagram 3] FIG. 1 is a schematic diagram showing the structure of an image generation model training device provided by the present invention; [Figure 4] 1 is a schematic diagram showing the structure of a service execution device provided by the present invention; [Diagram 5] 1 is a schematic diagram showing the structure of an electronic device provided by the present invention; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0022] In order to make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be described clearly and completely below in conjunction with specific embodiments of the present invention and corresponding drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments that can be obtained by those skilled in the art without creative efforts are all within the protection scope of the present invention.

[0023] The technical solutions provided by the embodiments of the present invention are described in detail below.

[0024] FIG. 1 is a schematic diagram showing the flow of the method for training an image generation model provided by the present invention, which includes the following steps:

[0025] In step S101, an original image is acquired.

[0026] The execution entity of the training method for an image generation model provided by the present invention may be a terminal device such as a notebook computer or a desktop computer, a client or server installed in a terminal device, or a dedicated device for training a model. For the sake of convenience, the training method for an image generation model provided by the present invention will be described below using only examples in which the execution entity is a dedicated device.

[0027] In the current field of computer vision, models based on deep learning technology often require a sufficient amount of image data for training, and if the amount of image data is insufficient, the trained model has difficulty meeting the service requirements. Currently, the mainstream method for dealing with the problem of insufficient image data is to perform processes such as rotation, inversion, translation, and filling on the initial image to generate additional image data for model training. However, the image data generated by the above processing method may have problems such as the image structure or content being too similar to the initial image, or the image foreground features in the image being too different from the original image, i.e., the realism is too low, and the generalization ability of the model trained using image data that may have such problems as a training set is affected, and as a result, the performance of the model cannot meet the service requirements.

[0028] Based on this, the present invention provides a method for training an image generation model, in which a dedicated device acquires an original image, and then the dedicated device inputs the noise-added original image into a first image generation model to remove noise and obtain a restored image, and trains the model based on image foreground features extracted from the original image and image foreground features extracted from the acquired restored image, thereby obtaining an image generation model that can determine a restored image from the original image.

[0029] In the training process of an image generation model, the dedicated device first needs to acquire original images to be used as training samples, where the original images may be acquired from a predetermined image set or may be acquired by a predetermined acquisition device.

[0030] In step S102, a noise addition process is performed on the original image to obtain a noise-added image.

[0031] After the dedicated device acquires the original image, it is necessary to perform a noise addition process on the original image to obtain a noise-added image. When adding noise to an original image using multiple noise signals, it is necessary to add the noise signals to the original image sequentially. That is, after adding noise to the original image with the first noise signal, noise is added using the second noise signal to the original image that has already had noise added with the first noise signal, and so on. After all the noise signals have been added to the original image, a noise-added image is obtained.

[0032] In addition, the scale of calculations required to sequentially add noise signals is large, and when the number of image data is large, the calculation load increases if each calculation is performed multiple times. Therefore, the present invention provides a method for adding noise to an original image based on a second image generation model constructed in advance. The second image generation model constructed in advance can output a noise-added image of the original image that has been subjected to noise addition processing a number of times corresponding to the number of noise signals based on the input original image and the number of noise signals.

[0033] Taking the example where the noise signals added to the original image are multiple Gaussian distribution signals, in the process of constructing the second image generation model, a sample image needs to be obtained first, and then the Nth Gaussian distribution signal is used to add noise to the noise-added image to which noise has been added with the (N-1)th Gaussian distribution signal, to obtain a noise-added image to which noise has been added with the Nth Gaussian distribution signal, where N is a positive integer equal to or greater than 1, and the noise-added image to which noise has been added with the 0th noise signal is the sample image.

[0034] Based on the noise-added image to which noise has been added with the N-th noise signal, the noise-added image to which noise has been added with the Nm-th noise signal, the N-th noise signal, and the N-m+1-th noise signal, a transformation relationship from the noise-added image to which noise has been added with the Nm-th noise signal to the noise-added image to which noise has been added with the N-th noise signal is determined, and the second image generation model is constructed based on the transformation relationship from the noise-added image to which noise has been added with the Nm-th noise signal to the noise-added image to which noise has been added with the N-th noise signal, where m is a positive integer smaller than N.

[0035] Specifically, in the process of constructing the second image generation model, the dedicated device inquires and obtains Gaussian distribution signal additive equation information according to the received fitting command, and calculates a Gaussian distribution signal additive equation for noise addition according to the acquired Gaussian distribution signal additive equation information.

number

[0036] where ε is a Gaussian distributed signal, x is the pixel value of the sample image, and β is the variance of the Gaussian distributed signal, a coefficient with a value between [0.0,1.0].

[0037] Then, the dedicated device repeats the addition formula of the input Gaussian distribution signal to realize adding the Gaussian distribution signal to the sample image multiple times.

[0038] Specifically, if x0 is the initial sample image, the dedicated device will generate a convolution formula for adding the Gaussian distribution signal of the first repetition based on the above Gaussian distribution signal addition formula:

number

[0039] In this way, the dedicated device repeats the addition equation of the Gaussian distribution signal multiple times based on the fitting command, and finally obtains the following convolution equations:

number

[0040] Here, ε in each iteration is a random number obtained by resampling according to the Standard Normal Distribution, and 0<β1<β2<β3……<β N-3 <β N-2 <β N-1 <β N <1.

[0041] If α=1-β, then

number

[0042] Dedicated devices are x0 to x N After determining the calculation process up to, let x0 be the Nth noise-added image x N It is also possible to construct a conversion relationship that converts in one go.

[0043] Specifically, x N-2 , x N-1、 x Nの relationship

number

number

[0044] The above formula can be simplified to:

number

[0045] When two normal distributions are convolved, the probability density function after the convolution remains normal, so

number

[0046] In the formula, ε N-1 and ε N are two independent random numbers, both of which satisfy the normal distribution, so based on the above formula, the two samplings can be combined into one sampling and sampling can be performed using the superimposed probability distribution.

[0047]

number

number

number

number

[0048] With μ=0 and σ=1,

number

number

[0049] ε N Similarly,

number

[0050] By overlaying two normal distributions, a new normal distribution N´(0,1-α N α N-1 ) can be obtained.

[0051] The dedicated device randomly samples the new distribution N', which is equivalent to sampling the two original distributions one over the other, and each sampling produces x N-2 From x N The conversion was completed up to

number

[0052] This conversion method allows dedicated devices to N-3 From x N Conversion relationship up to

number

[0053] In this way, the dedicated device will eventually go from x0 to x N Conversion relationship up to

number

[0054]

number

number

[0055] Based on the determined conversion relationship, the dedicated device can construct a second image generation model, and can add noise to an image using the second image generation model. That is, the second image generation model can output a noise-added image of the original image in which noise addition processing has been performed a number of times corresponding to the number of Gaussian distribution signals based on the input original image and the number of Gaussian distribution signals to be superimposed.

[0056] The dedicated device inputs the original image and the number of noise signals into a second image generation model constructed in advance by the above method, and causes the second image generation model to output a noise-added image of the original image that has been subjected to noise addition processing a number of times corresponding to the number of noise signals. That is, the original image and the number N of noise signals are input into the second image generation model, and the noise-added image output by the second image generation model corresponds to an original image that has been subjected to noise addition N times (that is, noise is added to the original image with the first noise signal to obtain an original image with noise added with the first noise signal, then noise is added to the original image with noise added with the first noise signal using the second noise signal, and so on), thereby significantly improving the efficiency of adding noise to an image. Thereafter, a restored image can be obtained by performing noise removal on the noise-added image.

[0057] In step S103, the noise-added image is input to a first image generation model, and the first image generation model is used to perform noise removal on the noise-added image to obtain a restored image, and image foreground features extracted from the restored image are determined.

[0058] After obtaining the noise-added image output by the second image generation model, the dedicated device inputs the noise-added image output by the second image generation model and the number of times noise is added to the noise-added image into a first image generation model, and uses the first image generation model to perform noise removal on the noise-added image to obtain a restored image, and then determines image foreground features extracted from the restored image.

[0059] In the process of removing noise from a noise-added image by a first image generation model, a superimposed noise signal used when performing a noise addition process corresponding to the number of times on an original image corresponding to the noise-added image is predicted based on a number of times the noise-added image is added, and a transition image before the number of times the noise addition process is performed on the input noise-added image is predicted based on the predicted superimposed noise signal. The above step of predicting a transition image can be used for the noise-added image and all transition images of the noise-added image. That is, the first image generation model can predict a superimposed noise signal used when performing a k-th noise addition process on an original image corresponding to an input image based on an input image and a number of times k of noise addition corresponding to the image until a restored image corresponding to the input image is obtained, then predict a k-1-th transition image before the k-th noise addition process based on the superimposed noise signal, and predict a k-2-th transition image before the k-1-th noise addition process based on the superimposed noise signal and the k-1-th transition image. k is a positive integer not exceeding the number of times corresponding to the input image.

[0060] For removal of Gaussian distribution signals, a conversion model may be constructed such that the first image generation model achieves the function of predicting a restored image corresponding to a noise-added image based on the frequency value of noise addition corresponding to the noise-added image, as follows:

[0061] x t Let x be the noise-added image with the predicted noise-addition number value t, and then, in the reverse direction, t The first image generation model obtains the restored image from x t From x t-1 Construct a conversion relationship up to x at any point t From x t-1 Based on the conversion relationship up to x at any point t Determine the transformation relationship from to x0.

[0062] Here, the first image generation model is x t From x t-1 That is, we construct a transformation relationship from t-1 │x t The procedure for determining

[0063] Transformation relationship P(x t │x t-1 ) in P(x t-1 │x t ) is expressed as the conversion relationship P(x t │x t-1 ) can be determined in the process of constructing the second image generation model.

[0064] x t From x t-1 Since is a random process, Bayes' theorem

number

number

[0065] Here, P(x t ) and P(x t-1 ) represent the probability of obtaining them from x0, so

number

[0066] Adding the condition x0 to everything means that under the same x0, x0 can actually be ignored.

[0067] The dedicated device is a transformation model that removes Gaussian distribution signals from noise-added images.

number

[0068] P(x t │x t-1 ) is x t-1 If x occurs t represents the probability of occurrence.

[0069] In the process of constructing the second image generation model,

number

number

[0070] where ε t satisfies the distribution N(0,1), and the constant

number

number

number

[0071] P(x t │x0) is the probability that x occurs t represents the probability of occurrence.

number

[0072] The dedicated device is P(x t │x0) is the probability distribution

number

[0073] P(x t-1 │x0) is the probability that x occurs t-1 represents the probability of occurrence.

number

[0074] The dedicated device is P(x t-1 │x0) is the probability distribution

number

[0075] The right-hand side of the equation for the transformation model to remove Gaussian distribution signals is all normal distribution, so the parameters are expressed as a probability density function.

number

[0076] Here, x is the random number value, μ is the mean, and σ is the standard deviation.

[0077]

number

[0078] The dedicated device is a transformation model that removes Gaussian distribution signals from the above three probability density functions and noise-added images.

number

number

[0079] Finally, the shape that fits the normal distribution equation

number

[0080] Here, the dedicated device is x t Given the condition x t-1 Probability density function and its distribution

number

[0081] Next, we need to remove the last term of the distribution, x0. t Relationship

number

number

number

[0082] Above x t Given the condition x t-1 Based on the probability distribution of x, the dedicated device uses a first image generation model to generate t Given the condition that a Gaussian distribution signal ε is added once, x t-1 can be determined, and a prediction of the transition image is realized by the first image generation model, thereby obtaining a restored image.

[0083] In step S104, the first image generation model is trained with the optimization goal of minimizing the difference between image foreground features corresponding to the original image and image foreground features extracted from the restored image.

[0084] After obtaining a restored image corresponding to the noise-added image using a first image generation model, the first image generation model is trained with an optimization goal of minimizing the difference between image foreground features extracted from an original image corresponding to the noise-added image and image foreground features extracted from the restored image, so that the trained first image generation model is used to generate a restored image having image foreground features similar to those of the original image, and the restored image can be used to construct a training set for training other models that require image data for training.

[0085] The trained first image generation model can output a restored image corresponding to the noise-added image based on the input noise-added image, and the image foreground features in the restored image have a high similarity with the image foreground features in the corresponding original image, but other parts have certain differences, achieving the effect of similar but dissimilar. Therefore, the restored image can be used as different image data to build a training set for training the model.

[0086] As can be seen from the above method, the above model training method can obtain an image generation model that can obtain differentiated images, and the image generation model is used to generate training samples that are used to construct a training set for training the model to be trained. The process is as follows:

[0087] FIG. 2 is a schematic diagram showing the flow of a service execution method provided by the present invention, which includes the following steps.

[0088] In step S201, an initial image is acquired.

[0089] The entity that executes the service execution method provided by the present invention may be a terminal device such as a notebook computer or a desktop computer, a client installed on a terminal device, or a server. For ease of explanation, however, the service execution method provided by the present invention will be explained using only examples in which the execution entity is a server.

[0090] For a model required for service execution, a certain amount of training data needs to be obtained to train the model. Here, the more the number and quality of the training data, the higher the ability of the trained model. If the number of training data is insufficient, the effect of the trained model cannot meet the needs of service execution. In response to this, the service execution method provided by the present invention can use the image generation model trained by the above-mentioned image generation model training method to additionally obtain target images from the initial image, which can be used to construct a training set.

[0091] Taking an artistic image generation service as an example, the required artistic image generation model (i.e., the specified model to be trained) requires a large amount of artistic image data as learning data, and when the number of image data is insufficient, for example, when there are few images of a certain artistic style, a small number of images of this artistic style can be used as initial images to obtain additional target images for training the artistic image generation model.

[0092] The server obtains image data that can be used for training the model as an initial image, which may be a noise-added image or an unnoised image. Here, for the noise-added image, the number of times of noise addition corresponding to the noise-added image must be carried, otherwise it will affect the quality of the restored image output by the pre-trained image generation model. For the unnoised image, noise addition may be performed using a second image generation model trained using the training method for the image generation model provided by the present invention.

[0093] In step S202, the initial image is input to a pre-trained image generation model, and a target image is output, where the image generation model is a model obtained by training using the above-mentioned training method.

[0094] The server inputs an initial image into a pre-trained first image generation model to obtain a target image, and the target image can be used to construct a training set for training a predetermined designated model required for service execution. Here, the pre-trained image generation model is a first image generation model trained using the image generation model training method provided by the present invention, and a restored image that can be used for model training can be obtained by removing noise from the noise-added image.

[0095] For example, before training an artistic image generation model (i.e., a designated model to be trained), an image in the training set can be used as an initial image, noise can be added to the initial image, and then the initial image along with the noise addition frequency value can be input to a pre-trained first image generation model to perform noise removal, thereby obtaining images whose image foreground features are similar to those of the initial image as additional image data, and expanding the training set of the artistic image generation model.

[0096] In addition, by not limiting the number of times noise is added to the initial image, it is possible to obtain more target images. For example, for an image to which noise has been added once, it is possible to obtain not only a target image with one restored original, but also a target image with two restored originals; there is no particular limitation on this.

[0097] In step S203, a training set is constructed based on the initial image and the target image, a predetermined designated model is trained using the training set, and a service is executed using the trained designated model.

[0098] After the server obtains a target image whose image foreground features are similar to those of the initial image, it constructs a training set based on the initial image and the target image, trains a predetermined designated model required for executing the service, and executes the service using the trained designated model.

[0099] For example, when the server trains an artistic image generation model (i.e., a designated model to be trained), it can construct a training set for the artistic image generation model based on an initial image and a target image, and train an artistic image generation model that meets the service requirements with a small amount of initial image data, thereby improving the training efficiency of the artistic image generation model.

[0100] In the present invention, the initial image input to the first image generation model may be an image with noise added or an image without noise added, but in either case, the first image generation model performs noise removal on the input image as an image with noise added, and obtains a target image whose image foreground features are similar to those of the initial image but have significant differences in other parts. For an initial image without noise added, the first image generation model performs noise removal by regarding a part of the image data included in the initial image as noise according to the logic of noise removal learned in the training process.

[0101] The image generation model training method and service execution method of the present invention have been described above. Based on the same concept, the present invention also provides corresponding apparatus, storage medium and electronic device.

[0102] FIG. 3 is a schematic diagram showing the structure of the training device for the image generation model provided by the present invention. The apparatus comprises: an acquisition module 301 for acquiring an original image; a noise addition module 302 for performing a noise addition process on the original image to obtain a noise-added image; an input module 303 for inputting the noise-added image and a number of times the noise-added image is noise-added to a first image generation model, predicting a superimposed noise signal used when performing a noise addition process of the number of times on the original image to convert it into the noise-added image using the first image generation model until a restored image is obtained, predicting a k-1th transition image before performing the k-th noise addition process based on the superimposed noise signal, predicting a k-2th transition image before performing the k-1th noise addition process based on the superimposed noise signal and the k-1th transition image, and determining an image foreground feature extracted from the restored image; and a training module 304 for training the first image generation model with an optimization objective of minimizing the difference between image foreground features corresponding to the original image and image foreground features extracted from the restored image.

[0103] Optionally, the noise addition module 302 specifically: The original image and the number of noise signals are input to a pre-constructed second image generation model, and the second image generation model is used to output a noise-added image of the original image in which noise addition processing has been performed a number of times corresponding to the number of noise signals.

[0104] Optionally, the noise addition module 302 specifically: Get a sample image, using the N-th noise signal, noise is added to a noise-added image to which noise has been added with the (N-1)-th noise signal, to obtain a noise-added image to which noise has been added with the N-th noise signal, N being a positive integer of 1 or more, and the noise-added image to which noise has been added with the 0-th noise signal is the sample image; determining a conversion relationship from the noise-added image to which noise has been added with the Nm-th noise signal to the noise-added image to which noise has been added with the N-th noise signal, based on the noise-added image to which noise has been added with the Nm-th noise signal, the N-th noise signal, and the N-m+1-th noise signal, where m is a positive integer less than N; It is used to construct the second image generation model based on the transformation relationship.

[0105] FIG. 4 is a schematic diagram showing the structure of a service execution device provided by the present invention. The device includes: an acquisition module 401 for acquiring an initial image; an input module 402 for inputting the initial image into a pre-trained image generation model and outputting a target image, the image generation model being a model obtained by training using the training method; and a training module 403 for constructing a training set based on the initial image and the target image, training a predetermined designated model using the training set, and executing a service using the trained designated model.

[0106] The present invention further provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the training method for an image generation model provided in FIG. 1 above or the service execution method provided in FIG. 2 above.

[0107] Based on the training method for an image generation model shown in Fig. 1 and the service execution method shown in Fig. 2, the embodiment of the present invention further provides a schematic diagram showing the structure of an electronic device shown in Fig. 5. As shown in Fig. 5, at the hardware level, the electronic device includes a processor, an internal bus, a network interface, an internal memory, and a non-volatile memory, and may of course include other hardware required for operation. The processor loads the corresponding computer program from the non-volatile memory into the internal memory and executes it to realize the training method for an image generation model shown in Fig. 1 or the service execution method shown in Fig. 2.

[0108] Of course, in addition to realization by software, the present invention does not exclude other realization methods, such as logical devices or a combination of hardware and software. In other words, the entity that executes the following processing steps is not limited to each logical unit, but may be hardware or a logical device.

[0109] In the 1990s, improvements in a technology could be clearly divided into hardware improvements (such as improvements in circuit structures such as diodes, transistors, and switches) and software improvements (improvements in method flows). However, with the development of technology, many of the current method flow improvements can be considered as direct improvements to hardware circuit structures. Designers can mostly obtain the corresponding hardware circuit structures by programming the improved method flows into the hardware circuits. Therefore, it cannot be said that method flow improvements cannot be realized by hardware physical modules. For example, Programmable Logic Devices (PLDs) (e.g. Field Programmable Gate Arrays (FPGAs)) are such integrated circuits, whose logical functions are determined by the user's programming of the device. Instead of chip manufacturers designing and manufacturing dedicated integrated circuit chips, designers program and "integrate" digital systems onto a single PLD.Nowadays, instead of handcrafting integrated circuit chips, this programming is mostly realized using software called a "logic compiler", which is similar to a software compiler used to write a program, and in order to compile the original code, it needs to be written in a specific programming language, which is called a Hardware Description Language (HDL). There is not just one type of HDL, but many types such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc., and the most commonly used at present are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. It will be clear to those skilled in the art that a hardware circuit that realizes a logical method flow can be easily obtained by simply programming a method flow logically in some of the above hardware description languages ​​and programming it into an integrated circuit.

[0110] The controller may be implemented in any suitable manner, for example, the controller may take the form of a microprocessor or processor, a computer readable storage medium storing computer readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application specific integrated circuits (ASICs), programmable logic controllers and embedded microcontrollers, examples of which include, but are not limited to, microcontrollers such as ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, Silicone Labs C8051F320, etc., and the memory controller may also be implemented as part of the control logic of the memory. It will be clear to those skilled in the art that in addition to implementing the controller purely with computer readable program code, it is entirely possible to logically program method steps to cause the controller to perform the same functions in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers and embedded microcontrollers, etc. Thus, such controllers may be considered as hardware components, and the devices included therein for implementing various functions may also be considered as structures within the hardware components. Alternatively, the apparatus for implementing various functions may be considered to be software modules implementing a method, or structures within hardware components.

[0111] The system, device, module or unit described in the above embodiments may be specifically implemented by a computer chip, an entity, or a product having some function. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a mobile phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet, a wearable device, or any combination of these devices.

[0112] For the sake of convenience, the above-mentioned device will be described by dividing it into various units according to their functions. Of course, when implementing the present invention, the functions of each unit can be realized by the same or multiple pieces of software and / or hardware.

[0113] As will be appreciated by those skilled in the art, embodiments of the present invention may be provided as a method, a system, or a computer program product. Thus, the present invention may employ an entirely hardware embodiment, an entirely software embodiment, or an embodiment that combines software and hardware. Furthermore, the present invention may take the form of a computer program product embodied in one or more computer usable storage media (including, but not limited to, magnetic disk memory, CD-ROM, optical memory, etc.) that contains computer usable program code.

[0114] The present invention will be described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and combinations of flows and / or blocks in the flowcharts and / or block diagrams, may be realized by computer program instructions. These computer program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, whereby the instructions executed by the processor of the computer or other programmable data processing device generate an apparatus for implementing the functions specified in one or more flows of the flowcharts and / or one or more blocks of the block diagrams.

[0115] These computer program instructions may be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable memory result in an article of manufacture that includes an instruction apparatus that implements the functions specified in one or more flows of the flowcharts and / or one or more blocks of the block diagrams.

[0116] These computer program instructions may be loaded into a computer or other programmable data processing device such that a series of operational steps are executed on the computer or other programmable device to generate a computer-implemented process, whereby the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more flows of the flowcharts and / or one or more blocks of the block diagrams.

[0117] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0118] The memory may include any form of computer-readable storage medium, such as volatile memory, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash memory (flash RAM). The memory is one example of a computer-readable storage medium.

[0119] Computer readable storage media include non-volatile and volatile media, removable and non-removable media, and may implement information storage by any method or technology. Information may be computer readable instructions, data structures, program modules, or other data. Computer storage media include, but are not limited to, Phase Change Memory (PRAM), Static Random-Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, Compact Disc Read Only Memory (CD-ROM), Digital Versatile Disc (DVD) or other optical storage, magnetic cassette tape, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that may be used to store information accessible by a computing device. As defined herein, computer readable storage media does not include transitory computer readable storage media, such as modulated data signals and carriers.

[0120] Additionally, the terms "comprise," "contain," or any other variations thereof are intended to include a non-exclusive inclusion, whereby a process, method, article, or device that comprises a set of elements not only includes those elements, but also includes other elements not expressly listed, or includes the inherent elements of such process, method, article, or device. Absent more limitations, an element qualified by the phrase "comprises a..." does not exclude the presence of further identical elements in a process, method, article, or device that includes said element.

[0121] As will be appreciated by those skilled in the art, embodiments of the present invention may be provided as a method, a system, or a computer program product. Thus, the present invention may employ an entirely hardware embodiment, an entirely software embodiment, or an embodiment that combines software and hardware. Furthermore, the present invention may take the form of a computer program product embodied in one or more computer usable storage media (including, but not limited to, magnetic disk memory, CD-ROM, optical memory, etc.) that contains computer usable program code.

[0122] The invention may be described in the general context of computer-executable instructions being executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The invention may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media, including storage devices.

[0123] Each embodiment of the present invention will be described in a stepwise manner, and the same or similar parts between each embodiment may be referred to each other, and the emphasis of each embodiment is on the differences from other embodiments. In particular, the system embodiment is basically similar to the method embodiment, so it will be briefly described, and the relevant parts may be referred to the description of some of the method embodiment.

[0124] The above is merely an embodiment of the present invention, and is not used to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modifications, equivalent replacements, improvements, etc. made without departing from the spirit and principle of the present invention should be included in the scope of the claims of the present invention.

Claims

1. acquiring an original image; performing a noise addition process on the original image to obtain a noise-added image; a step of inputting the noise-added image and a number of times the noise-added image is noise-added to a first image generation model, predicting a superimposed noise signal used when performing a noise addition process of the number of times on the original image to convert it into the noise-added image using the first image generation model until a restored image is obtained, predicting a k-1th transition image before performing the k-th noise addition process based on the superimposed noise signal, predicting a k-2th transition image before performing the k-1th noise addition process based on the superimposed noise signal and the k-1th transition image, and determining an image foreground feature extracted from the restored image, where k is a positive integer not exceeding the number of times, the image foreground feature is for representing a morphological feature of a target object in an image, and the image foreground feature does not include detailed physical features for representing the target object; training the first image generation model with an optimization goal of minimizing a difference between image foreground features corresponding to the original image and image foreground features extracted from the restored image; The step of performing a noise addition process on the original image to obtain a noise-added image includes: a step of inputting the original image and the number of noise signals into a second image generation model constructed in advance, and causing the second image generation model to output a noise-added image of the original image in which noise addition processing has been performed a number of times corresponding to the number of noise signals; The construction of the second image generation model includes: obtaining a sample image; a step of adding noise to a noise-added image to which noise has been added with an (N-1)th noise signal using an Nth noise signal to obtain a noise-added image to which noise has been added with the Nth noise signal, where N is a positive integer of 1 or more, and the noise-added image to which noise has been added with a 0th noise signal is the sample image; a step of determining a conversion relationship from the noise-added image to which noise has been added with the N-mth noise signal to the noise-added image to which noise has been added with the Nth noise signal, based on the noise-added image to which noise has been added with the N-mth noise signal, the Nth noise signal, and the N-m+1th noise signal, where m is a positive integer smaller than N; and constructing the second image generation model based on the transformation relationship. A method for training an image generation model, comprising:

2. acquiring an initial image; a step of inputting the initial image into a pre-trained image generation model and outputting a target image, the image generation model being a model obtained by training using the training method according to claim 1; constructing a training set based on the initial image and the target image, training a predetermined designated model using the training set, and executing a service using the trained designated model. A service execution method comprising:

3. an acquisition module for acquiring an original image; a noise addition module for performing a noise addition process on the original image to obtain a noise-added image; an input module for inputting the noise-added image and a number of times the noise-added image is noise-added to a first image generation model, predicting a superimposed noise signal used when performing a noise-adding process of the number of times on the original image to convert it into the noise-added image using the first image generation model until a restored image is obtained, predicting a k-1th transition image before performing a k-th noise-adding process based on the superimposed noise signal, predicting a k-2th transition image before performing a k-1th noise-adding process based on the superimposed noise signal and the k-1th transition image, and determining image foreground features extracted from the restored image, where k is a positive integer not exceeding the number of times, the image foreground features are for representing morphological features of a target object in an image, and the image foreground features do not include detailed physical features for representing the target object; a training module for training the first image generation model with an optimization objective of minimizing a difference between image foreground features corresponding to the original image and image foreground features extracted from the restored image; The noise addition module specifically includes: The original image and the number of noise signals are input to a second image generation model constructed in advance, and the second image generation model is used to output a noise-added image of the original image having been subjected to noise addition processing a number of times corresponding to the number of noise signals; The noise addition module specifically includes: Get a sample image, Using the N-th noise signal, noise is added to a noise-added image to which noise has been added with the (N-1)-th noise signal, thereby obtaining a noise-added image to which noise has been added with the N-th noise signal; determining a conversion relationship from the noise-added image to which noise has been added with the N-mth noise signal to the noise-added image to which noise has been added with the Nth noise signal, based on the noise-added image to which noise has been added with the N-mth noise signal, the Nth noise signal, and the N-m+1th noise signal; used to construct the second image generation model based on the transformation relationship; N is a positive integer equal to or greater than 1, a noise-added image to which noise has been added with the 0th noise signal is the sample image, and m is a positive integer smaller than N. An apparatus for training an image generation model, comprising:

4. an acquisition module for acquiring an initial image; an input module for inputting the initial image into a pre-trained image generation model and outputting a target image, the image generation model being a model obtained by training using the training method according to claim 1; a training module for constructing a training set based on the initial image and the target image, training a predetermined designated model using the training set, and executing a service using the trained designated model; A service execution device comprising:

5. A computer-readable storage medium storing a computer program, the computer program being executed by a processor to perform the method of claim 1. A computer-readable storage medium comprising:

6. 11. An electronic device comprising a processor and a computer program stored in a memory and operable on the processor, the electronic device performing the method of claim 1 when the processor executes the computer program.

1. An electronic device comprising:

Images