An image reality enhancement method and system based on a generative adversarial network
By constructing an image realism enhancement model based on adversarial generative networks, and using the rendering features of real and simulated samples for semantic alignment and feature fusion, the problem of insufficient style transfer effect and efficiency in existing technologies is solved, and efficient realism style transfer of simulated images is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHONGQING CHANGAN TECH CO LTD
- Filing Date
- 2023-09-26
- Publication Date
- 2026-06-26
AI Technical Summary
Existing technologies cannot balance the effectiveness and efficiency of style transfer, especially in multi-semantic images, where the style transfer effect of simulated images is not realistic.
By constructing an image realism enhancement model based on adversarial generative networks, semantic alignment and feature fusion are performed using rendering features of real and simulated sample images. An image enhancement network and a perception discrimination network are constructed, and adversarial training is conducted to optimize the image realism enhancement model.
This method improves the realism of style transfer in simulated images by enhancing the quality of style transfer across multi-semantic images, ensuring both efficiency and effectiveness.
Smart Images
Figure CN117078560B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image processing technology, and specifically to an image realism enhancement method and system based on generative adversarial networks. Background Technology
[0002] With the development of artificial intelligence technology, research on its application in the field of autonomous driving has emerged continuously. Object detection, as a key task in autonomous driving technology, requires deep learning. However, the training process of deep learning models requires a large number of labeled images, and image acquisition and labeling are extremely labor-intensive tasks. To save on the manual costs of acquiring and labeling real-world images, simulation software can be used to generate labeled images to supplement the real-world image dataset for training deep learning models for object detection. Considering the stylistic inconsistencies between the content in simulation scenes and real-world scenes, to improve the object detection accuracy of simulation images, it is necessary to transfer the style of real-world images to the simulation images.
[0003] Most current style transfer methods can only achieve artistic style transfer, using global statistics from deep feature maps generated in convolutional neural networks to describe the artistic style texture of images, such as using Gram matrices and covariance matrices. However, most simulated images often contain multiple different semantic parts. In this case, style transfer based on global statistics is insufficient to perform style texture transfer based on semantic matching between content and style images, making the overall style texture of the transfer result look like a style map, but failing to maintain the local semantic structure of the image content well.
[0004] Some researchers have proposed style transfer methods based on local semantic alignment. These methods find semantically similar or corresponding regions between content and style images, achieving good results in preserving the content structure of similar semantic regions and transferring details of style texture. Existing techniques propose a method for image-realistic style transfer. This method constructs a neural network model based on a flow model and determines style and content losses based on style and content features to obtain an image-realistic style transfer model. This model can then be used to stylize style and content maps, improving the processing speed and realism of image-realistic style transfer. However, this approach involves several loss terms that require trade-offs. In the process of image-realistic style transfer, for images with multiple semantic layers, it cannot simultaneously achieve both style transfer effectiveness and efficiency, thus reducing the overall effect of image-realistic style transfer. Summary of the Invention
[0005] One objective of this invention is to provide an image realism enhancement method based on adversarial generative networks (GCNs) to solve the problem that existing technologies cannot simultaneously achieve both style transfer effectiveness and efficiency; another objective is to provide an image realism enhancement system based on GCNs.
[0006] To achieve the above-mentioned objectives of this invention, the technical solution adopted is as follows:
[0007] An image realism enhancement method based on adversarial generative networks includes:
[0008] Render real sample images into simulated sample images and obtain rendering information generated during the rendering process;
[0009] Real sample images and simulated sample images are respectively segmented into labeled real image fragments and simulated image fragments, and then matched.
[0010] The rendering information is encoded to obtain rendering features;
[0011] An image realism enhancement model is constructed, which includes an image enhancement network and a perception discrimination network. The image enhancement network is trained using simulated image fragments and their corresponding rendering features to achieve the fusion of feature information of simulated image fragments and rendering features, resulting in enhanced simulated image fragments. Then, the image enhancement network and the perception discrimination network are trained adversarially using real image fragments that match the simulated image fragments and the enhanced simulated image fragments, resulting in a well-trained image realism enhancement model.
[0012] The simulated image to be enhanced is input into the trained image realism enhancement model to obtain a realistic simulation image.
[0013] This technical solution constructs an image realism enhancement model to enhance the realism of simulated images, utilizes the rendering features of simulated sample images to improve the quality of simulated images, and optimizes and trains the image realism enhancement model to improve the image quality of simulated image realism style transfer while ensuring the efficiency and effectiveness of image realism style transfer.
[0014] Furthermore, the real sample images and simulated sample images are respectively segmented into labeled real image fragments and simulated image fragments, and then matched, including the following steps:
[0015] A semantic segmentation network is used to perform category label mapping on real sample images and simulated sample images respectively, resulting in labeled real sample images and labeled simulated sample images. The real sample images are used as the target domain for category label mapping, and the simulated sample images are used as the source domain for category label mapping.
[0016] The labeled real sample images and simulated sample images are cropped into labeled real image fragments and simulated image fragments of the same size;
[0017] Labeled real image fragments and simulated image fragments are grouped according to category labels;
[0018] Calculate the feature distance between all simulated image fragments and real image fragments within each group;
[0019] Set a feature distance threshold to match simulated image fragments within a group with real image fragments whose feature distance is less than the feature distance threshold.
[0020] Based on the above technical means, the category label mapping between real sample images and simulated sample images and the semantic alignment between real image fragments and simulated image fragments can be realized. By grouping and matching image fragments with multiple category labels, it is beneficial to realize the style transfer of multi-semantic simulated images, which can adapt to the detailed image realism style requirements.
[0021] Furthermore, the rendering method is G-buffer rendering, which obtains the G-buffer geometric buffer information generated during the rendering process; a G-buffer encoder consisting of multiple residual networks connected in sequence is constructed, and the G-buffer encoder is used to encode the G-buffer geometric buffer information generated during the rendering of the simulation sample image to obtain the G-buffer geometric buffer features.
[0022] Based on the above technical means, it is beneficial to compile G-buffer geometric buffer features through G-buffer encoder, optimize the training process of image realism enhancement model using G-buffer geometric buffer features, avoid artifacts in the simulation image output by the model, and after the image realism enhancement model is trained, G-buffer encoder does not participate in the realism style transfer process of the simulation image to be enhanced.
[0023] Furthermore, the total loss function of the image augmentation network is constructed. Train the image enhancement network until... The convergence stops, resulting in a well-trained image enhancement network. The training objective formula for this image enhancement network is as follows:
[0024]
[0025] in, To reduce the loss of realism in the enhanced simulated image fragments, Let G represent the feature structure loss of the image augmentation network. This represents the total loss function of the image enhancement network. This represents the data distribution of the images generated by the image enhancement network. The data distribution representing real sample images, This represents the enhanced simulated image fragments generated by the image enhancement network. This represents the input real sample image, and MSSIM is the average structural similarity function.
[0026] Based on the above technical means, the parameters of the image enhancement network are adjusted by the loss of realism between the generated image and the real sample image and the loss of feature structure between the generated image and the simulated sample image.
[0027] Furthermore, in step S4, the enhanced simulated image fragments and real image fragments are input into the perception and discrimination network to obtain the realism loss of the enhanced simulated image fragments, and the parameters of the perception and discrimination network are adjusted according to the obtained realism loss.
[0028] Furthermore, the perception and discrimination network includes a VGG-16 network, a CGL network, and a CLC network connected in sequence;
[0029] The VGG-16 network is used to extract the perceptual features of the enhanced simulated image fragments generated by the image enhancement network. The CGL network converts the extracted perceptual features into feature vectors. The CLC network reduces the dimensionality of the converted feature vectors to a single-channel map. The realism loss of the enhanced simulated image fragments is calculated based on the feature distance between the single-channel map of the simulated fragments and the real image fragments.
[0030] Furthermore, the CLC network includes a boundary expansion layer Pad, a first convolutional layer Cov1, an activation function ReLU, and a second convolutional layer Cov2 connected in sequence. The CGL network includes four sets of recurrent structures connected in sequence. Each set of recurrent structures includes a boundary expansion layer Pad, a first convolutional layer Cov1, a group normalization layer GroupNormal, and an activation function ReLU connected in sequence.
[0031] Furthermore, the total loss function of the perceptual discrimination network is constructed. Train the perception and discrimination network until... The convergence stops, resulting in a well-trained perceptual discrimination network. The training objective formula for the perceptual discrimination network is as follows:
[0032]
[0033] Where D represents the perceptual discrimination network, This represents the data distribution of the simulated sample images. This represents the total loss function of the perception and discrimination network.
[0034] Based on the aforementioned technical means, the parameters of the perceptual discrimination network are adjusted according to the loss of realism between the generated image and the real sample image.
[0035] Furthermore, both the image enhancement network and the perception discrimination network are trained simultaneously, with the overall training objective as follows:
[0036] .
[0037] Based on the above technical means, the training objectives of both the image enhancement network and the perception discrimination network can be achieved simultaneously, so that the image enhancement network and the perception discrimination network can eventually reach Nash equilibrium.
[0038] This technical solution also proposes an image realism enhancement system based on generative adversarial networks, including:
[0039] The rendering information acquisition module is used to acquire rendering information generated during the process of rendering real sample images into simulated sample images;
[0040] The image segmentation module is used to segment real sample images and simulated sample images into labeled real image fragments and simulated image fragments, respectively, and then match them.
[0041] The feature extraction module is used to encode the rendering information generated during the rendering process to obtain rendering features;
[0042] The image realism enhancement model module is used to construct an image realism enhancement model, which includes an image enhancement network and a perceptual discrimination network.
[0043] The training module is used to train the image enhancement network using simulated image fragments and their corresponding rendering features to achieve the fusion of feature information of simulated image fragments and rendering features, so as to obtain enhanced simulated image fragments. The image enhancement network and the perception discrimination network are adversarially trained using real image fragments that match the simulated image fragments and the enhanced simulated image fragments to obtain a trained image realism enhancement model.
[0044] The simulation image enhancement module is used to input the simulation image to be enhanced into the trained image realism enhancement model to obtain a realistic simulation image.
[0045] Compared with the prior art, the beneficial effects of the technical solution of the present invention are:
[0046] This invention proposes an image realism enhancement method and system based on generative adversarial networks. First, real sample images and simulated sample images are paired using category label mappings obtained from semantic segmentation, resulting in semantically aligned real and simulated image fragments. The rendering information of the simulated sample images is compiled into rendering features, and then an image realism enhancement model is constructed. This model is trained using real image fragments and simulated image fragments incorporating rendering features, resulting in a well-trained image realism enhancement model. Finally, the trained model is used to generate more realistic simulated images. This approach achieves realistic style transfer for multiple targets in complex scenes while ensuring the efficiency and effectiveness of the style transfer process. Attached Figure Description
[0047] Figure 1 This is a flowchart illustrating the image realism enhancement method based on adversarial generative networks proposed in this invention.
[0048] Figure 2 This is a flowchart illustrating the semantic alignment process between real and simulated sample images in this invention.
[0049] Figure 3 This is a schematic diagram of the G-buffer encoder structure in this invention;
[0050] Figure 4 This is a schematic diagram of the structure of the first residual network in this invention;
[0051] Figure 5 This is a schematic diagram of the structure of the second residual network in this invention;
[0052] Figure 6 This is a flowchart illustrating the overall process of the image realism enhancement method based on adversarial generative networks proposed in this invention.
[0053] Figure 7 This is a schematic diagram of the image enhancement network structure in this invention;
[0054] Figure 8 A schematic diagram of the known structure of the denormalized layer RAD for rendering;
[0055] Figure 9 This is a schematic diagram of the perceptual discrimination network structure in this invention;
[0056] Figure 10 This is a schematic diagram of the CLC network structure;
[0057] Figure 11 This is a schematic diagram of the CGL network structure;
[0058] Figure 12 This is a schematic diagram of the image realism enhancement system based on adversarial generative networks proposed in this invention. Detailed Implementation
[0059] The embodiments of the present invention will be described below with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for illustrating the present invention and not for limiting the scope of protection of the present invention.
[0060] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.
[0061] like Figure 1 As shown, this embodiment proposes an image realism enhancement method based on generative adversarial networks, including the following steps:
[0062] S1: Render the real sample image into a simulated sample image, wherein the rendering method is G-buffer rendering; collect the G-buffer (Geometric Buffer) geometric buffer information generated during the rendering process of the simulated sample image, wherein the G-buffer geometric buffer information refers to the texture information containing color, normal, and world space coordinates during the rendering process; in this embodiment, the G-buffer geometric buffer information generated during the rendering process of four kinds of simulated sample images is obtained.
[0063] S2: Segment the real sample image and the simulated sample image into labeled real image fragments and simulated image fragments, respectively. Match the labeled real image fragments and simulated image fragments. See [link / reference] Figure 2 This includes the following steps:
[0064] S21: Using real sample images as the target domain and simulated sample images as the source domain, a semantic segmentation network is used to process and obtain the category label mapping between real sample images and simulated sample images. In specific implementation, the category labels include people, roads, vehicles, plants, lane lines, etc.
[0065] S22: Crops the labeled real sample image and simulated sample image into labeled real image fragments and simulated image fragments of the same size;
[0066] S23: Group the labeled real image fragments and simulated image fragments according to category;
[0067] S24: Calculate the feature distance between all simulated image fragments and real image fragments in each group. In this embodiment, the VGG-16 (Visual Geometry Group) network is used to extract image fragment features.
[0068] S25: Set a feature distance threshold to match simulated image fragments within a group with real image fragments whose feature distance is less than the feature distance threshold.
[0069] S3: Construct a G-buffer encoder and use the G-buffer encoder to encode the G-buffer geometric buffer information generated during the rendering of the simulation sample image to obtain the G-buffer geometric buffer features of the simulation sample image.
[0070] In this embodiment, as Figure 3 As shown, the G-buffer encoder includes eight residual network layers connected in sequence. Layers 1, 2, 4, 6, and 8 form the first residual network, and layers 3, 5, and 7 form the second residual network. Figure 4 As shown, the first residual network contains a first convolutional layer Cov1, an activation function ReLU, a second convolutional layer Cov2, and an activation function ReLU connected in sequence, as follows: Figure 5 As shown, the second residual network adds a convolutional branch CovP to the first residual network.
[0071] S4: Construct an image realism enhancement model, see [link / reference] Figure 6 The image realism enhancement model includes an image enhancement network and a perception discrimination network;
[0072] S41: The image enhancement network is trained using labeled simulated image fragments and the G-buffer geometric buffer features of simulated sample images to obtain enhanced simulated image fragments.
[0073] In this embodiment, as Figure 7As shown, the image enhancement network comprises four branches, each containing 4, 8, 16, and 32 RB (Residual Block) network blocks, respectively. Each RB residual network block contains two branches. The first branch consists of a first convolutional layer (Cov1), a render-aware denormalization layer (RAD), an activation function (ReLU), and a second convolutional layer (Cov2), connected sequentially. The second branch consists of a third convolutional layer (CovP) and a group normalization layer (GroupNormal), connected sequentially. The two branches are stacked and activated by the ReLU activation function. The render-aware denormalization layer (RAD) is mainly used to modulate the features of the simulated image, and its structure is as follows: Figure 8 As shown, it contains three branches. The first branch only has a GroupNormal normalization layer. The second and third branches have the same structure, each including three first residual networks and one convolutional layer connected in sequence. Finally, the feature information of the three branches is fused to obtain the modulated simulated image features.
[0074] In practice, different image evaluation metrics can be selected based on different target detection algorithms to calculate the feature structure distance between the enhanced and unenhanced simulated image fragments. Examples include LPIPS (Learned Perceptual Image Patch Similarity) and PSNR (Peak Signal-to-Noise Ratio). In this embodiment, MSSIM (Mean Structural Similarity) is used to calculate the feature structure distance between the enhanced and unenhanced simulated image fragments. The training objective of the image enhancement network is as follows:
[0075]
[0076] in, To reduce the loss of realism in the enhanced simulated image fragments, Let G represent the feature structure loss of the image augmentation network. This represents the total loss function of the image enhancement network. This represents the data distribution of the images generated by the image enhancement network. The data distribution representing real sample images, This represents the enhanced simulated image fragments generated by the image enhancement network. Represents the input real sample image, where MSSIM is the average structural similarity function;
[0077] S42: Train the perception and discrimination network by inputting the enhanced simulated image fragments and real image fragments into the perception and discrimination network to obtain the realism loss of the enhanced simulated image fragments, and adjust the parameters of the perception and discrimination network based on the obtained realism loss.
[0078] In this embodiment, as Figure 9 As shown, the perceptual discrimination network includes a VGG-16 network, a CGL (Convolution-GroupNorm-LeakyReLU) network, and a CLC (Convolution-LeakyReLU-Convolution) network; see [link to documentation]. Figure 10 The CLC network consists of a boundary expansion layer Pad, a first convolutional layer Cov1, an activation function ReLU, and a second convolutional layer Cov2, connected in sequence. See [link to documentation]. Figure 11 The CGL network consists of four sets of sequentially connected recurrent structures. Each set of recurrent structures includes a boundary expansion layer Pad, a first convolutional layer Cov1, a group normalization layer GroupNormal, and an activation function ReLU, which are connected in sequence.
[0079] The VGG-16 network is used to extract the perceptual features of the enhanced simulated image fragments generated by the image enhancement network. The CGL network converts the extracted perceptual features into feature vectors. The CLC network reduces the dimensionality of the converted feature vectors to a single-channel map. The realism loss of the enhanced simulated image fragments is calculated based on the feature distance between the single-channel map of the simulated fragments and the real image fragments.
[0080] The training objectives of the perceptual discrimination network are as follows:
[0081]
[0082] Where D represents the generator network, This represents the data distribution of the simulated sample images. This represents the total loss of the perception and discrimination network.
[0083] After the image augmentation network and the perception discrimination network are trained, a trained image realism enhancement model is obtained. In this embodiment, in order to achieve the above two training objectives simultaneously and enable the image augmentation network and the perception discrimination network to eventually reach Nash equilibrium, the number of training iterations can be selected as 600,000-1,000,000 according to actual needs. The overall training objective of the perception discrimination network is set as follows:
[0084] .
[0085] S5: Input the simulated image to be enhanced into the trained image realism enhancement model to obtain a realistic simulation image; in this process, the G-buffer encoder does not participate in the style transfer process.
[0086] Based on the above embodiments, the present invention also proposes an image realism enhancement system based on generative adversarial networks, such as... Figure 12 As shown, it includes:
[0087] The rendering information acquisition module 101 is used to acquire the G-buffer geometric buffer information generated during the process of rendering a real sample image into a simulated sample image.
[0088] The image segmentation module 102 is used to segment real sample images and simulated sample images into labeled real image fragments and simulated image fragments, respectively, and then match them.
[0089] The feature extraction module 103 is used to encode the G-buffer geometric buffer information generated during the rendering process to obtain G-buffer geometric buffer features;
[0090] Image realism enhancement model module 104 is used to construct an image realism enhancement model, which includes an image enhancement network and a perceptual discrimination network;
[0091] Training module 105 is used to train the image enhancement network using simulated image fragments and their corresponding G-buffer geometric buffer features, so as to achieve the fusion of feature information of simulated image fragments and G-buffer geometric buffer features to obtain enhanced simulated image fragments; using real image fragments that match the simulated image fragments and enhanced simulated image fragments to perform adversarial training on the image enhancement network and the perception discrimination network to obtain a trained image realism enhancement model.
[0092] The simulation image enhancement module 106 is used to input the simulation image to be enhanced into the trained image realism enhancement model to obtain a realistic simulation image.
[0093] The above embodiments are merely preferred embodiments provided to fully illustrate the present invention, and the scope of protection of the present invention is not limited thereto. Equivalent substitutions or modifications made by those skilled in the art based on the present invention are all within the scope of protection of the present invention.
Claims
1. A method for enhancing image realism based on generative adversarial networks, characterized in that, include: Render real sample images into simulated sample images and obtain rendering information generated during the rendering process; Real sample images and simulated sample images are respectively segmented into labeled real image fragments and simulated image fragments, and then matched. The rendering information is encoded to obtain rendering features; An image realism enhancement model is constructed, which includes an image enhancement network and a perception discrimination network. The image enhancement network is trained using labeled simulated image fragments and their corresponding rendering features to achieve the fusion of feature information of simulated image fragments and rendering features, thereby obtaining enhanced simulated image fragments. Then, real image fragments that match the simulated image fragments and enhanced simulated image fragments are used to perform adversarial training on the image enhancement network and the perception discrimination network to obtain a well-trained image realism enhancement model. The simulated image to be enhanced is input into the trained image realism enhancement model to obtain a realistic simulation image.
2. The image realism enhancement method based on generative adversarial networks according to claim 1, characterized in that, The real sample image and the simulated sample image are respectively segmented into labeled real image fragments and simulated image fragments, and then matched, including the following steps: A semantic segmentation network is used to perform category label mapping on real sample images and simulated sample images respectively, resulting in labeled real sample images and labeled simulated sample images. The real sample images are used as the target domain for category label mapping, and the simulated sample images are used as the source domain for category label mapping. The labeled real sample images and labeled simulated sample images are cropped into labeled real image fragments and labeled simulated image fragments of the same size; Labeled real image fragments and labeled simulated image fragments are grouped according to category labels; Calculate the feature distance between all simulated image fragments and real image fragments within each group; Set a feature distance threshold to match simulated image fragments within a group with real image fragments whose feature distance is less than the feature distance threshold.
3. The image realism enhancement method based on generative adversarial networks according to claim 1, characterized in that, The rendering method is G-buffer rendering, which acquires G-buffer geometric buffer information generated during the rendering process; the G-buffer geometric buffer information is encoded, wherein a G-buffer encoder consisting of multiple residual networks connected in sequence is constructed, and the G-buffer encoder is used to encode the G-buffer geometric buffer information generated during the rendering of the simulation sample image to obtain G-buffer geometric buffer features.
4. The image realism enhancement method based on generative adversarial networks according to claim 3, characterized in that, Construct the total loss function for the image augmentation network Train the image enhancement network until... The convergence stops, resulting in a well-trained image enhancement network. The training objective formula for this image enhancement network is as follows: in, To reduce the loss of realism in the enhanced simulated image fragments, Let G represent the feature structure loss of the image augmentation network. This represents the total loss function of the image enhancement network. This represents the data distribution of the images generated by the image enhancement network. The data distribution representing real sample images, This represents the enhanced simulated image fragments generated by the image enhancement network. Let D(x) represent the input real sample image, MSSIM be the average structural similarity function, and D(x) be the simulated image fragments input by the perceptual discriminative network. x The output value.
5. The image realism enhancement method based on generative adversarial networks according to claim 4, characterized in that, In step S4, the enhanced simulated image fragments and real image fragments are input into the perception and discrimination network to obtain the realism loss of the enhanced simulated image fragments. The parameters of the perception and discrimination network are adjusted based on the obtained realism loss.
6. The image realism enhancement method based on generative adversarial networks according to claim 5, characterized in that, The perception and discrimination network includes a VGG-16 network, a CGL network, and a CLC network connected in sequence. The VGG-16 network is used to extract the perceptual features of the simulated image fragments after realism enhancement generated by the image enhancement network. The CGL network converts the extracted perceptual features into feature vectors. The CLC network reduces the dimensionality of the converted feature vectors to a single-channel map. The realism loss of the simulated image fragments after realism enhancement is calculated based on the feature distance between the single-channel map of the simulated fragments and the real image fragments.
7. The image realism enhancement method based on generative adversarial networks according to claim 6, characterized in that, The CLC network includes a boundary expansion layer Pad, a first convolutional layer Cov1, an activation function ReLU, and a second convolutional layer Cov2 connected in sequence. The CGL network includes four sets of recurrent structures connected in sequence. Each set of recurrent structures includes a boundary expansion layer Pad, a first convolutional layer Cov1, a group normalization layer GroupNormal, and an activation function ReLU connected in sequence.
8. The image realism enhancement method based on generative adversarial networks according to claim 7, characterized in that, Construct the total loss function of the perceptual discrimination network Train the perception and discrimination network until... The convergence stops, resulting in a well-trained perceptual discrimination network. The training objective formula for the perceptual discrimination network is as follows: Where D represents the perceptual discrimination network, This represents the data distribution of the simulated sample images. This represents the total loss function of the perception and discrimination network.
9. The image realism enhancement method based on generative adversarial networks according to claim 8, characterized in that, Simultaneously, the image enhancement network and the perception discrimination network are trained, with the overall training objective as follows: 。 10. An image realism enhancement system based on generative adversarial networks, characterized in that, include: The rendering information acquisition module is used to acquire rendering information generated during the process of rendering real sample images into simulated sample images; The image segmentation module is used to segment real sample images and simulated sample images into labeled real image fragments and simulated image fragments, respectively, and then match them. The feature extraction module is used to encode the rendering information generated during the rendering process to obtain rendering features; The image realism enhancement model module is used to construct an image realism enhancement model, which includes an image enhancement network and a perceptual discrimination network. The training module is used to train the image enhancement network using simulated image fragments and their corresponding rendering features, so as to achieve the fusion of feature information of simulated image fragments and rendering features to obtain enhanced simulated image fragments. By using real image fragments that match simulated image fragments and enhanced simulated image fragments, adversarial training is performed on the image enhancement network and the perception discrimination network to obtain a well-trained image realism enhancement model. The simulation image enhancement module is used to input the simulation image to be enhanced into the trained image realism enhancement model to obtain a realistic simulation image.