Method, device and medium for training virtual image generation model

Abstract

The application provides a training method and device of a virtual image generation model, equipment and a medium. The method comprises: selecting a standard template image from a standard template image library, which needs to generate a virtual image, and extracting a real image from a data set corresponding to the standard template image; generating a first noise image corresponding to the real image based on the real image; merging the standard template image and the first noise image to obtain a merged image; inputting the merged image into a pre-set generator network to obtain a candidate image output by the generator network; inputting a first image pair and a second image pair into a pre-set discriminator network to train the generator network by the discriminator network in an adversarial manner, and obtaining a trained generator network as a virtual image generation model; wherein the first image pair comprises the merged image and the real image, and the second image pair comprises the merged image and the candidate image. The virtual image generation model is trained by a generative adversarial network, and the authenticity of the generated virtual image is improved.

Description

Technical Field

[0001] This invention relates to the field of image generation technology, and in particular to a training method, apparatus, device, and medium for a virtual image generation model. Background Technology

[0002] As image generation technology matures, its applications can be found in various image recognition data augmentation processes, everyday life, entertainment, and artistic creation. Especially in the training of deep learning-based artificial intelligence perception models, the data-driven nature of these models inevitably places greater demands on the diversity and scale of data. Often, to further improve the accuracy of perception models, it is necessary to generate large amounts of diverse virtual data through data augmentation methods to expand the sample library.

[0003] However, some existing image generation methods suffer from insufficient diversity or unrealistic image generation, which greatly limits the improvement of the accuracy of subsequent image perception models. Summary of the Invention

[0004] This invention provides a training method, apparatus, device, and medium for a virtual image generation model, which addresses the problems of insufficient diversity or unrealistic nature of generated images in existing technologies.

[0005] This invention provides a training method for a virtual image generation model, comprising:

[0006] Select a standard template image from the standard template image library to generate the virtual image, and extract the real image from the dataset corresponding to the standard template image;

[0007] Based on the real image, a first noise map corresponding to the real image is generated; wherein, the first noise map is used to characterize the difference between the real image and the standard template image;

[0008] The standard template image and the first noise image are merged to obtain a merged image;

[0009] The merged image is input into a pre-set generator network to obtain candidate images output by the generator network; wherein, the generator network is used to generate virtual images as candidate images based on the merged image;

[0010] The first image pair and the second image pair are input into a pre-set discriminator network to train the generator network adversarially, resulting in a trained generator network, which serves as a virtual image generation model; wherein the first image pair includes the merged image and the real image, and the second image pair includes the merged image and the candidate image.

[0011] According to a training method for a virtual image generation model provided by the present invention, after inputting a first image pair and a second image pair into a pre-set discriminator network to train the generator network adversarially by the discriminator network, and obtaining the trained generator network as a virtual image generation model, the method further includes:

[0012] The standard template image and the randomly generated second noise image are merged and then input into the virtual image generation model to obtain the virtual image corresponding to the standard template image output by the virtual image generation model.

[0013] According to a training method for a virtual image generation model provided by the present invention, after merging the standard template image and the randomly generated second noise image and inputting them into the virtual image generation model to obtain the virtual image corresponding to the standard template image output by the virtual image generation model, the method further includes:

[0014] The image perception model is trained based on the virtual image corresponding to the standard template image output by the virtual image generation model.

[0015] According to a training method for a virtual image generation model provided by the present invention, the step of selecting a standard template image from a standard template image library to generate a virtual image, and extracting a real image from the dataset corresponding to the standard template image, includes:

[0016] Select the standard template image from the standard template image library;

[0017] The real image is extracted from the dataset corresponding to the standard template image by random sampling or non-random sampling.

[0018] According to a training method for a virtual image generation model provided by the present invention, the step of generating a first noise map corresponding to the real image based on the real image includes:

[0019] Extract image features from the real image as noise seeds;

[0020] Based on the noise seed, a first noise map corresponding to the real image is generated;

[0021] The size of the first noise map is the same as the size of the standard template map.

[0022] According to a training method for a virtual image generation model provided by the present invention, the step of merging the standard template image and the first noise image to obtain a merged image includes:

[0023] The merged image is obtained by merging the standard template image and the first noise image through channel-level image stitching and / or matrix calculation.

[0024] According to a training method for a virtual image generation model provided by the present invention, before inputting a first image pair and a second image pair into a pre-set discriminator network to train the generator network adversarially by the discriminator network to obtain a trained generator network as a virtual image generation model, the method further includes:

[0025] The merged image and the real image are taken as the first image pair, and the label value of the first image pair is set to represent true.

[0026] The merged image and the candidate image are used as the second image pair, and the label value of the second image pair is set to represent false.

[0027] The step of inputting the first image pair and the second image pair into a pre-set discriminator network, so that the discriminator network can be used to train the generator network adversarially, and the trained generator network is obtained as a virtual image generation model, includes:

[0028] The first image pair and the second image pair are input into the discriminator network, which then adversarially trains the generator network based on the label values ​​of the first image pair and the second image pair, resulting in a trained generator network that serves as a virtual image generation model.

[0029] The present invention also provides a training apparatus for a virtual image generation model, comprising:

[0030] The extraction module is used to select a standard template image from the standard template image library to generate a virtual image, and to extract a real image from the dataset corresponding to the standard template image.

[0031] A generation module is configured to generate a first noise map corresponding to the real image based on the real image; wherein the first noise map is used to characterize the difference between the real image and the standard template image;

[0032] The merging module is used to merge the standard template image and the first noise image to obtain a merged image;

[0033] The processing module is used to input the merged image into a pre-set generator network to obtain candidate images output by the generator network; wherein, the generator network is used to generate virtual images as candidate images based on the merged image;

[0034] The training module is used to input a first image pair and a second image pair into a pre-set discriminator network, so that the discriminator network can train the generator network adversarially to obtain a trained generator network as a virtual image generation model; wherein, the first image pair includes the merged image and the real image, and the second image pair includes the merged image and the candidate image.

[0035] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement a training method for the virtual image generation model as described above.

[0036] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a training method for a virtual image generation model as described above.

[0037] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements a training method for a virtual image generation model as described above.

[0038] The present invention provides a training method, apparatus, device, and medium for a virtual image generation model. First, a standard template image for generating virtual images is selected. Then, the standard template image and its corresponding real image's first noise image are merged. The merged image is then paired with both the real image and a candidate image output by the generator network, forming image pairs. These pairs are input into a discriminator network, which then trains the generator network adversarially to obtain the trained generator, which serves as the virtual image generation model. The virtual image generation model trained using a generative adversarial network effectively improves the realism of the generated virtual images. Furthermore, different standard template images can be used to train the generator network, allowing the trained virtual image generation model to generate virtual images corresponding to different standard template images. This increases the diversity of subsequently generated virtual images and facilitates the expansion of the training dataset or sample library for the image perception model using more realistic and diverse virtual images, thereby improving the accuracy of the image perception model. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0040] Figure 1 This is one of the flowcharts illustrating the training method for the virtual image generation model provided by the present invention;

[0041] Figure 2 This is a schematic diagram of the standard template image in the training method of the virtual image generation model provided by the present invention;

[0042] Figure 3 This is a schematic diagram of the virtual image in the training method of the virtual image generation model provided by the present invention;

[0043] Figure 4 This is the second flowchart illustrating the training method for the virtual image generation model provided by this invention;

[0044] Figure 5 This is the third flowchart illustrating the training method for the virtual image generation model provided by this invention;

[0045] Figure 6 This is a schematic diagram of the structure of the training device for the virtual image generation model provided by the present invention;

[0046] Figure 7 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0048] The training method, apparatus, device, and medium of the virtual image generation model of the present invention are described below with reference to the accompanying drawings.

[0049] Figure 1 This is one of the flowcharts illustrating the training method for the virtual image generation model provided by the present invention, such as... Figure 1 As shown, the method includes steps 101 to 105; wherein:

[0050] Step 101: Select a standard template image from the standard template image library to generate a virtual image, and extract a real image from the dataset corresponding to the standard template image;

[0051] Step 102: Based on the real image, generate a first noise map corresponding to the real image;

[0052] The first noise map is used to characterize the difference between the real image and the standard template map;

[0053] Step 103: Merge the standard template image and the first noise image to obtain a merged image;

[0054] Step 104: Input the merged image into a pre-set generator network to obtain the candidate image output by the generator network;

[0055] The generator network is used to generate virtual images as candidate images based on the merged images;

[0056] Step 105: Input the first image pair and the second image pair into a pre-set discriminator network to train the generator network adversarially, thereby obtaining the trained generator network as a virtual image generation model; wherein, the first image pair includes the merged image and the real image, and the second image pair includes the merged image and the candidate image.

[0057] Specifically, some existing image generation methods suffer from insufficient diversity or unrealistic image generation, which greatly limits the improvement of the accuracy of subsequent image perception models.

[0058] Unlike conventional image generation methods, this invention utilizes standard template images to obtain more realistic virtual images while ensuring diversity.

[0059] Optionally, the training method for the virtual image generation model provided in this embodiment of the invention can be applied to the field of traffic sign recognition to generate virtual images of traffic signs to expand the dataset or sample library, and then use the expanded dataset or sample library to train the image perception model to improve the accuracy of the image perception model.

[0060] Optionally, Figure 2 This is a schematic diagram of the standard template image in the training method of the virtual image generation model provided by the present invention, such as... Figure 2 As shown, standard template diagrams in the field of traffic signs can include instruction signs, prohibition signs, and warning signs.

[0061] In this embodiment of the invention, a standard template image for generating virtual images is first selected from the standard template image library, and a real image is extracted from the dataset corresponding to the standard template image. Here, the dataset can be the dataset required for subsequent training of the image perception model.

[0062] It is understandable that the standard template image does not contain noise, while the actual captured real image does contain noise. Therefore, a first noise image corresponding to the real image can be generated based on the real image. This first noise image is used to characterize the difference between the real image and the standard template image.

[0063] The standard template image and the first noise image are then merged to obtain a merged image. The merged image is then input into a pre-set generator network to obtain candidate images output by the generator network. A first image pair including the merged image and the real image and a second image pair including the merged image and the candidate image are then input into a discriminator network to train the generator network against the discriminator network. The trained generator network is then obtained as a virtual image generation model.

[0064] It should be noted that in step 105, steps 101 to 104 can be repeated iteratively to train the generator network in order to reduce the difference between the virtual images generated by the virtual image generation model and the real images.

[0065] Different standard template images can also be selected, and steps 101 to 105 above can be repeated iteratively to train the generator network, so that the trained virtual image generation model can generate virtual images corresponding to different standard template images, further improving the diversity of generated virtual images.

[0066] Optionally, corresponding convolutional neural networks, Transformer networks, etc., can be designed based on deep learning, machine learning, and other methods to serve as generator networks.

[0067] In the training method of the virtual image generation model provided in this embodiment of the invention, a standard template image for generating virtual images is first selected. The standard template image and its corresponding real image's first noise image are then merged. The merged image is then paired with the real image and the candidate image output by the generator network to form image pairs, which are input into a discriminator network. The discriminator network then trains the generator network adversarially, resulting in a trained generator network, which serves as the virtual image generation model. The virtual image generation model trained through a generative adversarial network can effectively improve the realism of the generated virtual images. Furthermore, different standard template images can be used to train the generator network, allowing the trained virtual image generation model to generate virtual images corresponding to different standard template images. This increases the diversity of subsequently generated virtual images and facilitates the use of more realistic and diverse virtual images to expand the dataset or sample library for training the image perception model, thereby improving the accuracy of the image perception model.

[0068] Optionally, after inputting the first image pair and the second image pair into a pre-set discriminator network to train the generator network adversarially and obtain the trained generator network as a virtual image generation model, the standard template image and the randomly generated second noise image can be merged and input into the virtual image generation model to obtain the virtual image corresponding to the standard template image output by the virtual image generation model.

[0069] Specifically, by inputting the standard template image that needs data augmentation and the randomly generated second noise image into the trained virtual image generation model, diverse and realistic virtual images can be generated.

[0070] Optionally, a random noise map generator can be designed to generate several random noise maps as second noise maps; then, the generated second noise maps are combined with the standard template map that needs to be expanded and input into the trained generator network, that is, into the virtual image generation model, so as to generate diverse and realistic virtual images through the virtual image generation model.

[0071] Figure 3 This is a schematic diagram of the virtual image in the training method of the virtual image generation model provided by the present invention, as shown below. Figure 3 As shown in the figure, the first row represents five different standard template images, and the second, third and fourth rows represent generated images using different random noises, that is, different randomly generated second noise images, corresponding to the generated virtual images.

[0072] In this embodiment of the invention, after generating the virtual image generation model, the standard template image and the randomly generated second noise image can be merged and input into the virtual image generation model to generate the virtual image corresponding to the standard template image. Using the randomly generated second noise image and the standard template image to generate the virtual image effectively improves the diversity of the generated virtual images, which is beneficial for subsequently training the image perception model with more diverse virtual images, thereby improving the accuracy of the image perception model.

[0073] Optionally, after merging the standard template image and the randomly generated second noise image and inputting them into the virtual image generation model to obtain the virtual image corresponding to the standard template image output by the virtual image generation model, an image perception model can be trained based on the virtual image corresponding to the standard template image output by the virtual image generation model.

[0074] Specifically, after generating the virtual image corresponding to the standard template image that needs to be expanded, the image perception model can be trained based on the virtual image to improve the perception accuracy of the image perception model.

[0075] In this embodiment of the invention, training the image perception model based on more diverse and realistic virtual images can effectively improve the accuracy of the image perception model.

[0076] Optionally, the method for selecting a standard template image from a standard template image library to generate a virtual image, and extracting a real image from the dataset corresponding to the standard template image, may include:

[0077] Select the standard template image from the standard template image library;

[0078] The real image is extracted from the dataset corresponding to the standard template image by random sampling or non-random sampling.

[0079] Specifically, after selecting a standard template image from the standard template image library, the corresponding real image can be extracted from the dataset corresponding to the standard template image through various types of random or non-random sampling.

[0080] Optionally, the method for generating the first noise map corresponding to the real image based on the real image may include:

[0081] Extract image features from the real image as noise seeds;

[0082] Based on the noise seed, a first noise map corresponding to the real image is generated;

[0083] The size of the first noise map is the same as the size of the standard template map.

[0084] Specifically, the real image can be serialized, or a certain stable image feature can be extracted using certain existing fixed algorithms. Then, the serialized real image or the extracted feature can be used as a noise seed to generate a noise image of a specific size as the first noise image. Here, the specific size means that the size of the first noise image needs to be the same as the size of the standard template image to ensure that when merging the first noise image and the standard template image, there will be no distortion due to the lack of correspondence between the points in the first noise image and the standard template image.

[0085] In this embodiment of the invention, image features extracted from a real image are used as noise seeds to generate a first noise map, which can ensure that there is a one-to-one correspondence between the real image and the first noise map generated based on the real image.

[0086] Optionally, the method of merging the standard template image and the first noise image to obtain the merged image may include:

[0087] The merged image is obtained by merging the standard template image and the first noise image through channel-level image stitching and / or matrix calculation.

[0088] Specifically, the generated first noise map and the corresponding standard template map can be combined using methods such as channel-level image stitching and various matrix multiplication and addition; the above matrix calculations can include matrix multiplication or addition.

[0089] Optionally, before inputting the first image pair and the second image pair into a pre-set discriminator network to train the generator network adversarially and obtain the trained generator network as a virtual image generation model, the merged image and the real image can be used as the first image pair, and the label value of the first image pair can be set to represent true; then the merged image and the candidate image can be used as the second image pair, and the label value of the second image pair can be set to represent false.

[0090] The method of inputting the first image pair and the second image pair into a pre-set discriminator network, so that the discriminator network can be used to train the generator network adversarially, to obtain a trained generator network, can be implemented as a virtual image generation model as follows:

[0091] The first image pair and the second image pair are input into the discriminator network, which then adversarially trains the generator network based on the label values ​​of the first image pair and the second image pair, resulting in a trained generator network that serves as a virtual image generation model.

[0092] Specifically, a discriminator network can be designed based on methods such as deep learning and machine learning;

[0093] The standard template image (i.e., the merged image) combined with the first noise image and the sampled real image are used as an image pair, which are then input into the discriminator network and their label value is set to true.

[0094] The standard template image (i.e., the merged image) combined with the first noise image and the candidate image generated by the generator network are used as an image pair and input into the discriminator network, with the discriminator label value set to false.

[0095] Then, the discriminator network, based on the two image pairs and their corresponding label values, adversarially trains the generator network to obtain the trained generator network, which serves as the virtual image generation model.

[0096] The following example illustrates the training method for the virtual image generation model provided in this embodiment of the invention.

[0097] Figure 4 This is the second flowchart illustrating the training method for the virtual image generation model provided by this invention, as shown below. Figure 4 As shown, the method includes steps 401 to 406; wherein:

[0098] Step 401: Select a standard template image and sample the real images required for training the network from the corresponding dataset;

[0099] Step 402: Generate the corresponding first noise map using the sampled real image;

[0100] Step 403: Combine the first noise image with the selected standard template image and input it into the generator network to generate candidate images;

[0101] Step 404: Combine the standard template image with the first noise image to form image pairs with the sampled real image and the generated candidate image, respectively, input them into the discriminator network, and label the two image pairs as real and false respectively to train the generator network against the adversarial training.

[0102] Step 405: Iterate and repeat the above steps to obtain the trained generator network, which serves as the virtual image generation model.

[0103] Specifically, the steps of selecting a standard template image, generating a first noise image, generating candidate images, and using a discriminator to distinguish between real and fake generated images are repeated. After multiple iterations, the generator network and the discriminator network are trained adversarially to obtain the final generator network, which serves as the virtual image generation model.

[0104] Step 406: Input the standard template image that needs to be augmented and the randomly generated second noise image into the virtual image generation model to generate a variety of realistic virtual images.

[0105] Figure 5 This is the third flowchart illustrating the training method for the virtual image generation model provided by this invention, as shown below. Figure 5 As shown, the embodiments of the present invention are mainly divided into training process and inference process.

[0106] (I) Regarding the training process

[0107] First, a standard template image x is selected. Then, one or more real images y are randomly sampled from the corresponding training set of the standard template image x. Next, a corresponding noise image z is generated based on the real image y, which serves as the first noise image. Finally, the standard template image and the first noise image are merged to obtain the merged image. Merge images The input is fed into the generator network G to obtain the candidate image output by the generator network G.

[0108] Combine the real image y and the merged image As the first image pair, and marking the first image pair as true, the candidate image... and merged images The first image pair and the second image pair are used as the second image pair and marked as false. The first image pair and the second image pair are then input into the discriminator network D, so that the discriminator network D can be used to train the generator network G against the first image pair. The trained generator network G is then used as the virtual image generation model.

[0109] (II) Regarding the reasoning process

[0110] First, a random noise graph z is generated using a random noise generator. i The second noise map is then merged with the standard template map x to obtain the merged image. Merge images The input is fed into the trained virtual image generation model to obtain the virtual image output by the model.

[0111] The training method for the virtual image generation model provided in this invention can be used, but is not limited to, in the data augmentation process of many image recognition tasks. It can also be applied to various virtual image generation tasks with standard template images.

[0112] The training apparatus for the virtual image generation model provided by the present invention will be described below. The training apparatus for the virtual image generation model described below and the training method for the virtual image generation model described above can be referred to in correspondence.

[0113] Figure 6 This is a schematic diagram of the structure of the training device for the virtual image generation model provided by the present invention, as shown below. Figure 6 As shown, the training device 600 for the virtual image generation model includes:

[0114] The extraction module 601 is used to select a standard template image from the standard template image library to generate a virtual image, and extract a real image from the dataset corresponding to the standard template image.

[0115] The generation module 602 is used to generate a first noise map corresponding to the real image based on the real image; wherein the first noise map is used to characterize the difference between the real image and the standard template image;

[0116] Merging module 603 is used to merge the standard template image and the first noise image to obtain a merged image;

[0117] The processing module 604 is used to input the merged image into a pre-set generator network to obtain candidate images output by the generator network; wherein, the generator network is used to generate virtual images as candidate images based on the merged image;

[0118] The training module 605 is used to input the first image pair and the second image pair into a pre-set discriminator network, so that the discriminator network can train the generator network against the generator network to obtain the trained generator network as a virtual image generation model; wherein, the first image pair includes the merged image and the real image, and the second image pair includes the merged image and the candidate image.

[0119] In the training apparatus for the virtual image generation model provided in this embodiment of the invention, the extraction module first selects a standard template image to be generated as the virtual image, and extracts a real image from the dataset corresponding to the standard template image. Then, the generation module generates a first noise image corresponding to the real image. The merging module merges the standard template image and the first noise image. The training module then pairs the merged image with the real image and the candidate image output by the generator network, and inputs them into a discriminator network. The discriminator network then trains the generator network adversarially, resulting in a trained generator network, which serves as the virtual image generation model. The virtual image generation model trained through generative adversarial networks can effectively improve the realism of the generated virtual images. In addition, different standard template images can be used to train the generator network, so that the trained virtual image generation model can also generate virtual images corresponding to different standard template images, increasing the diversity of subsequently generated virtual images. This is beneficial for expanding the dataset or sample library for training the image perception model with more realistic and diverse virtual images, thereby improving the accuracy of the image perception model.

[0120] Optionally, the training device 600 for the virtual image generation model also includes:

[0121] The inference module is used to merge the standard template image and the randomly generated second noise image, and then input them into the virtual image generation model to obtain the virtual image corresponding to the standard template image output by the virtual image generation model.

[0122] Optionally, the training module 605 is further used to train an image perception model based on the virtual image corresponding to the standard template image output by the virtual image generation model.

[0123] Optionally, the extraction module 601 is specifically used for:

[0124] Select the standard template image from the standard template image library;

[0125] The real image is extracted from the dataset corresponding to the standard template image by random sampling or non-random sampling.

[0126] Optionally, the generation module 602 is specifically used for:

[0127] Extract image features from the real image as noise seeds;

[0128] Based on the noise seed, a first noise map corresponding to the real image is generated;

[0129] The size of the first noise map is the same as the size of the standard template map.

[0130] Optionally, the merging module 603 is specifically used to: merge the standard template image and the first noise image through channel-level image stitching and / or matrix calculation to obtain the merged image.

[0131] Optionally, the training device 600 for the virtual image generation model also includes:

[0132] The setting module is used to set the label value of the first image pair as true, and the combined image and the candidate image as false, and set the label value of the second image pair as false.

[0133] The training module 605 is specifically used to input the first image pair and the second image pair into the discriminator network, so that the discriminator network can adversarially train the generator network based on the label values ​​of the first image pair and the label values ​​of the second image pair, and obtain the trained generator network as a virtual image generation model.

[0134] Figure 7 This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 7 As shown, the electronic device 700 may include: a processor 710, a communication interface 720, a memory 730, and a communication bus 740, wherein the processor 710, the communication interface 720, and the memory 730 communicate with each other through the communication bus 740. The processor 710 can call logical instructions in the memory 730 to execute a training method for a virtual image generation model, the method including:

[0135] Select a standard template image from the standard template image library to generate the virtual image, and extract the real image from the dataset corresponding to the standard template image;

[0136] Based on the real image, a first noise map corresponding to the real image is generated; wherein, the first noise map is used to characterize the difference between the real image and the standard template image;

[0137] The standard template image and the first noise image are merged to obtain a merged image;

[0138] The merged image is input into a pre-set generator network to obtain candidate images output by the generator network; wherein, the generator network is used to generate virtual images as candidate images based on the merged image;

[0139] The first image pair and the second image pair are input into a pre-set discriminator network to train the generator network adversarially, resulting in a trained generator network, which serves as a virtual image generation model; wherein the first image pair includes the merged image and the real image, and the second image pair includes the merged image and the candidate image.

[0140] Furthermore, the logical instructions in the aforementioned memory 730 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0141] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program that can be stored on a non-transitory computer-readable storage medium, wherein when the computer program is executed by a processor, the computer is able to execute the training method for the virtual image generation model provided by the above methods, the method comprising:

[0142] Select a standard template image from the standard template image library to generate the virtual image, and extract the real image from the dataset corresponding to the standard template image;

[0143] Based on the real image, a first noise map corresponding to the real image is generated; wherein, the first noise map is used to characterize the difference between the real image and the standard template image;

[0144] The standard template image and the first noise image are merged to obtain a merged image;

[0145] The merged image is input into a pre-set generator network to obtain candidate images output by the generator network; wherein, the generator network is used to generate virtual images as candidate images based on the merged image;

[0146] The first image pair and the second image pair are input into a pre-set discriminator network to train the generator network adversarially, resulting in a trained generator network, which serves as a virtual image generation model; wherein the first image pair includes the merged image and the real image, and the second image pair includes the merged image and the candidate image.

[0147] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a training method for the virtual image generation model provided by the methods described above, the method comprising:

[0148] Select a standard template image from the standard template image library to generate the virtual image, and extract the real image from the dataset corresponding to the standard template image;

[0149] Based on the real image, a first noise map corresponding to the real image is generated; wherein, the first noise map is used to characterize the difference between the real image and the standard template image;

[0150] The standard template image and the first noise image are merged to obtain a merged image;

[0151] The merged image is input into a pre-set generator network to obtain candidate images output by the generator network; wherein, the generator network is used to generate virtual images as candidate images based on the merged image;

[0152] The first image pair and the second image pair are input into a pre-set discriminator network to train the generator network adversarially, resulting in a trained generator network, which serves as a virtual image generation model; wherein the first image pair includes the merged image and the real image, and the second image pair includes the merged image and the candidate image.

[0153] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0154] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0155] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A training method for a virtual image generation model, characterized in that, include: Select a standard template image from the standard template image library to generate the virtual image, and extract the real image from the dataset corresponding to the standard template image; Extract image features from the real image as noise seeds; Based on the noise seed, a first noise map corresponding to the real image is generated; wherein, the first noise map is used to characterize the difference between the real image and the standard template image; the size of the first noise map is the same as the size of the standard template image; The standard template image and the first noise image are merged to obtain a merged image; The merged image is input into a pre-set generator network to obtain candidate images output by the generator network; wherein, the generator network is used to generate virtual images as candidate images based on the merged image; The merged image and the real image are taken as a first image pair, and the label value of the first image pair is set to represent true. The merged image and the candidate image are used as a second image pair, and the label value of the second image pair is set to represent false. The first image pair and the second image pair are input into the discriminator network, which then adversarially trains the generator network based on the label values ​​of the first image pair and the second image pair, resulting in a trained generator network that serves as a virtual image generation model.

2. The training method for the virtual image generation model according to claim 1, characterized in that, After inputting the first image pair and the second image pair into a discriminator network to adversarially train the generator network and obtain the trained generator network as a virtual image generation model, the method further includes: The standard template image and the randomly generated second noise image are merged and then input into the virtual image generation model to obtain the virtual image corresponding to the standard template image output by the virtual image generation model.

3. The training method for the virtual image generation model according to claim 2, characterized in that, After merging the standard template image and the randomly generated second noise image and inputting them into the virtual image generation model to obtain the virtual image corresponding to the standard template image output by the virtual image generation model, the method further includes: The image perception model is trained based on the virtual image corresponding to the standard template image output by the virtual image generation model.

4. The training method for the virtual image generation model according to any one of claims 1 to 3, characterized in that, The step of selecting a standard template image from a standard template image library to generate a virtual image, and extracting a real image from the dataset corresponding to the standard template image, includes: Select the standard template image from the standard template image library; The real image is extracted from the dataset corresponding to the standard template image by random sampling or non-random sampling.

5. The training method for the virtual image generation model according to any one of claims 1 to 3, characterized in that, The process of merging the standard template image and the first noise image to obtain a merged image includes: The merged image is obtained by merging the standard template image and the first noise image through channel-level image stitching and / or matrix calculation.

6. A training device for a virtual image generation model, characterized in that, include: The extraction module is used to select a standard template image from the standard template image library to generate a virtual image, and to extract a real image from the dataset corresponding to the standard template image. The generation module is used to extract image features from the real image as noise seeds; Based on the noise seed, a first noise map corresponding to the real image is generated; wherein, the first noise map is used to characterize the difference between the real image and the standard template image; the size of the first noise map is the same as the size of the standard template image; The merging module is used to merge the standard template image and the first noise image to obtain a merged image; A processing module is used to input the merged image into a pre-set generator network to obtain candidate images output by the generator network; wherein, the generator network is used to generate virtual images as candidate images based on the merged image; The training module is used to take the merged image and the real image as a first image pair and set the label value of the first image pair to represent true; take the merged image and the candidate image as a second image pair and set the label value of the second image pair to represent false; input the first image pair and the second image pair into a discriminator network, so that the discriminator network can adversarially train the generator network based on the label values ​​of the first image pair and the label values ​​of the second image pair, and obtain the trained generator network as a virtual image generation model.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the training method for the virtual image generation model as described in any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the training method of the virtual image generation model as described in any one of claims 1 to 5.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the training method of the virtual image generation model as described in any one of claims 1 to 5.

Images