High-fidelity simulation image synthesis method and system based on domain migration

Abstract

The invention discloses a high-fidelity simulation image synthesis method and system based on domain migration, and belongs to the field of simulation image synthesis. The method comprises the following steps: segmenting an object foreground and a background from a real image, and complementing the background; generating a simulation image and an object mask in the simulation scene by using the 3D model, and extracting a simulation object foreground; constructing a two-stage domain migration neural network, taking a real object foreground as a reference, carrying out style migration on the simulation foreground, and generating a high-fidelity synthesis foreground; meanwhile, the real background is expanded to generate a diversified high-fidelity synthetic background; and fusing the foreground after style migration and the background after expansion to obtain a final high-fidelity simulation image. According to the method, on the premise of ensuring the authenticity of the geometric structure, the visual authenticity of the image is remarkably improved, high-quality simulation images can be efficiently synthesized in batches at low cost, and rich data support is provided for computer vision tasks.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of simulation image synthesis, and in particular to a high-fidelity simulation image synthesis method and system based on domain migration. BACKGROUND

[0002] Simulation image synthesis refers to a technical means of using computer-generated virtual images to replace or supplement real images. This method generates image data that can be used to train, verify and evaluate various algorithm models by constructing scenes, objects and interactive processes similar to reality in a virtual environment. Compared with directly collecting real images, simulation image synthesis has significant advantages in terms of controllable cost, efficiency improvement and data scalability. In recent years, with the rapid development of artificial intelligence and computer vision, simulation image synthesis technology has been widely used in automatic driving, robot operation, medical imaging, remote sensing identification and industrial quality inspection, providing massive and diversified data support for related algorithm models.

[0003] Existing simulation image synthesis methods can be mainly divided into two categories: synthesis methods based on physical engines and synthesis methods based on generative models.

[0004] Among them, the synthesis method based on physical engine relies on three-dimensional modeling and rendering technology to simulate the geometric structure, lighting conditions and physical interaction rules of the real world in virtual software. This method usually uses physical engines, optical engines and material renderers to generate simulation data, and can achieve the diversification of scenes and objects through large-scale parameterized modeling. Because it can generate annotated accurate data in large quantities at low cost, it has become the most widely used synthesis method. At the same time, in order to reduce the difference between virtual images and real images, researchers often introduce domain randomization technology to disturb the light, texture, perspective and background elements in a large range to improve the cross-domain generalization ability of the model.

[0005] However, this method still faces many challenges in practical application. On the one hand, the geometric parameters and physical properties of the simulation model often do not match the real environment, making it difficult to fully match the real world; on the other hand, due to the difference between the rendering mechanism and the real imaging process, the generated images have large domain differences with real images in terms of texture details, noise distribution and imaging reality. These problems lead to limited effectiveness and applicability of simulation images generated based on physical engines in some high-precision tasks.

[0006] On the other hand, the synthesis method based on the generative model relies on the deep generative model technology that has emerged in recent years, and realizes the generation of new images by learning the distribution characteristics of real images. The basic idea of this kind of method is to use generative mechanisms such as generative adversarial networks, variational autoencoders, and diffusion models to directly synthesize realistic images without relying on accurate physical modeling. This kind of method can realize relatively realistic style transfer, that is, under the premise of keeping the scene geometry unchanged, the style of the target image data domain is mapped to the original image data domain, thereby enhancing the realism and domain consistency of the synthesized image. The advantage of this kind of method is that the generated results usually have higher visual fidelity, which can effectively reduce the perceptual difference from the real data, and therefore has good development prospects in automatic driving simulation, virtual reality content generation and other aspects.

[0007] However, the synthesis method based on the generative model also has certain limitations. Since it mainly relies on statistical distribution learning and lacks explicit modeling of geometric constraints and physical laws, the generated images often have deficiencies in spatial consistency and physical reality. For example, unreasonable occlusion relationships may occur between objects, the light distribution does not conform to the real physical laws, and even the generated geometric shapes are distorted. These problems will significantly weaken the effectiveness and usability of the synthesized images in tasks that require strict physical interpretability and geometric consistency, such as robot grasping, structure reconstruction, and mechanical simulation.

[0008] In summary, the existing methods have problems such as poor realism of the synthesized simulation images or inability to guarantee the reality of the geometric structure, and there is an urgent need to develop a simulation image generation method that can greatly improve the realism of the images while guaranteeing the reality of the geometric structure in the images. SUMMARY

[0009] To solve the above technical problems, the present application provides a high-fidelity simulation image synthesis method and system based on domain transfer, which can greatly improve the realism of the images while guaranteeing the reality of the geometric structure in the images, thereby realizing the efficient and low-cost synthesis of a large number of high-fidelity simulation images and can be widely applied in the fields of artificial intelligence, autonomous driving, robots, etc.

[0010] To achieve the above-mentioned application purposes, the high-fidelity simulation image synthesis method based on domain transfer provided by the embodiments comprises the following steps: Obtain a plurality of real scene images, obtain real scene object foreground and real scene background images through image segmentation, complete the real scene background image to obtain a complete real scene background image; Step 2: Build a simulation scene, generate a plurality of simulation scene images and corresponding simulation scene object masks in the simulation scene using object 3D models and interference object 3D models, and segment using the simulation scene object masks to obtain simulation scene object foreground; Step 3: Establish a two-stage domain transfer neural network with the real scene object foreground as the reference, input the real scene object foreground and the simulated scene object foreground, and perform style transfer on the simulated scene object foreground to output a high-fidelity synthesized object foreground; Step 4: Perform background extension on the real scene background image to obtain multiple high-fidelity synthesized background images, and fuse the high-fidelity synthesized object foreground and the high-fidelity synthesized background image to obtain a high-fidelity simulation image.

[0011] In one embodiment, in step 1, the real scene object foreground and the real scene background image obtained by image segmentation include: performing image segmentation by a large model or manual annotation to obtain an object mask in the real scene image, and segmenting the real scene image using the object mask to obtain the real scene object foreground and the real scene background image.

[0012] In one embodiment, in step 1, the real scene background image is completed to obtain a complete real scene background image, including: using a pre-trained diffusion model to complete the real scene background image to obtain a complete real scene background image.

[0013] In one embodiment, in step 2, the multiple simulated scene images and the corresponding simulated scene object masks generated in the simulated scene using the object 3D model and the interference object 3D model include: The simulated scene is built based on the Blender simulation software, the object 3D model and the interference object 3D model are set and domain randomization is performed, and multiple simulated scenes are generated, wherein the domain randomization includes randomization of background texture, background material, position and pose of objects and interference objects, surface optical parameters of objects and interference objects, light source position, and light source parameters in the simulated scene; Based on the multiple simulated scenes, an image is rendered using a simulation physics engine and an optical engine to obtain multiple simulated scene images and corresponding simulated scene object masks.

[0014] In one embodiment, in step 3, the two-stage domain transfer neural network includes a first-stage encoding and decoding neural network and a second-stage encoding and decoding neural network. The first-stage encoding and decoding neural network is configured to input the real scene object foreground and the simulated scene object foreground, extract features step by step through a first extraction module in an encoding stage to align low-frequency features and obtain first extraction features, decode and reconstruct features step by step through a first restoration module in a decoding stage, and output a synthesized object foreground with consistent artistic style. The second stage is a coding and decoding form of neural network, which is used to input the synthetic object foreground in artistic style consistent with the real scene object foreground, and to output a high-fidelity synthetic object foreground by gradually extracting features through a second extraction module in the coding stage and gradually decoding features through a second restoration module in the decoding stage.

[0015] In an embodiment, in the coding and decoding form of neural network of the first stage and the second stage, the coding stage and the decoding stage are sequentially connected through an encoding layer, a whitening-coloring transformation layer, and a decoding layer; In the coding and decoding form of neural network of the first stage, the first extracted features output by the coding stage are subjected to feature extraction through the encoding layer, standardization and alignment of cross-domain feature distribution through the whitening-coloring transformation layer, dimension reduction of the aligned features through the decoding layer, and output of high-resolution features, which are used as input of the first restoration module. In the coding and decoding form of neural network of the second stage, the second extracted features output by the coding stage are subjected to feature compression and distribution reorganization through the encoding layer, cross-domain feature standardization through the whitening-coloring transformation layer, and semantic information decoding and feature restoration through the decoding layer, and output of new features, which are used as input of the second restoration module.

[0016] In an embodiment, the first extraction module is used to input the real scene object foreground and the simulated scene object foreground, extract image features through the encoding layer, select the extracted image features at the pixel level through the foreground perception layer, output pixel-screened features, output low-frequency features through a low-pass filter, and obtain the first extracted features. The first restoration module is used to input the high-resolution features, recover the spatial resolution and compensate the feature details through the up-sampling layer and the decoding layer, output higher-resolution features, realize cross-domain style consistency constraint on the higher-resolution features through the whitening-coloring transformation layer, decode and reconstruct the features subjected to the style consistency constraint through the decoding layer, and output the synthetic object foreground in artistic style consistent.

[0017] In an embodiment, there is a high-pass module for jump connection between the first extraction module and the first restoration module; the high-pass module is used to input the pixel-screened features in the first extraction module, sequentially pass through the whitening-coloring transformation layer, the high-pass filter, and the Gaussian blur layer, realize cross-domain alignment of high-frequency features, retain high-frequency information, and smooth and suppress high-frequency noise and redundancy, and output features after high-frequency feature conversion.

[0018] In an embodiment, the second extraction module is configured to take the synthetic object foreground in artistic style and the real scene object foreground as input, perform preliminary feature extraction through an encoding layer, complete cross-domain feature distribution alignment through a whitening-coloring transformation layer, compress and enhance deep semantic information through an encoding layer, obtain pixel-selected features through a foreground perception layer, retain low-frequency components and suppress noise interference through a low-pass filter, and output second extraction features. The second restoration module is configured to take the new features as input, implement style conversion of low-frequency features through an upsampling layer, a decoding layer, a whitening-coloring transformation layer, and a decoding layer, and output a high-fidelity synthetic object foreground.

[0019] In an embodiment, a high-pass filter is arranged between the second extraction module and the second restoration module, and is configured to take the pixel-selected features in the second extraction module as input to perform skip connection, so as to realize invariant alignment of high-frequency features.

[0020] In an embodiment, the method further includes: performing image processing on the complete real scene background image obtained in step 1 to obtain an enhanced real scene background image, taking the enhanced real scene background image as input, and outputting a synthetic background image after background extension through a pre-trained diffusion model combined with a scene-related text description driven by domain knowledge.

[0021] In an embodiment, the high-fidelity synthetic object foreground and the synthetic background image after background extension are fused through an Alpha blending method to obtain a high-fidelity simulation image.

[0022] In another aspect, the application also provides a high-fidelity simulation image synthesis system based on domain migration, which is configured to implement the high-fidelity simulation image synthesis method based on domain migration. The real scene processing unit is configured to obtain a plurality of real scene images, obtain real scene object foreground and real scene background images through image segmentation, and complete the real scene background image to obtain a complete real scene background image. The simulation scene generation unit is configured to build a simulation scene, generate a plurality of simulation scene images and corresponding simulation scene object masks in the simulation scene by using object 3D models and interference object 3D models, and obtain simulation scene object foreground by using the simulation scene object masks for segmentation. The domain migration neural network unit is configured to take the real scene object foreground as a reference, establish a two-stage domain migration neural network, take the real scene object foreground and the simulation scene object foreground as input, perform style migration on the simulation scene object foreground, and output a high-fidelity synthetic object foreground. The background extension and synthesis unit is used for background extension on a real scene background image, obtaining a plurality of high-fidelity synthesized background images, fusing a high-fidelity synthesized object foreground and the high-fidelity synthesized background images to obtain a high-fidelity simulation image.

[0023] Compared with the prior art, the present application has at least the following beneficial effects: (1) The present application proposes a high-fidelity simulation image synthesis method based on domain migration, which generates simulation images in a simulation environment by combining a physical engine and domain randomization technology, ensuring the geometric consistency and authenticity of the key foreground; a two-stage domain migration neural network is used to realize high-fidelity restoration of the key foreground and the target image domain; a generative neural network is used for background extension to realize the authenticity and diversity of the image background.

[0024] (2) Using the method proposed in the present application, any number of similar synthesized images can be generated from a reference target domain image, and the fidelity of the synthesized images is very high. Compared with the comparative method, the present application can improve the similarity to the target domain by nearly 100 times. The images synthesized by the present method can achieve similar or even better training effect as real images. On the international public data set of image segmentation task, the baseline method uses 113198 real images for training to achieve a segmentation accuracy of 0.763, while the method proposed in the present application only needs to use 48 images as a reference, and the 3000 synthesized images are used for training to achieve a segmentation accuracy of 0.762. And because the images generated by the present method have self-labeling, a lot of labeling cost is saved. Therefore, the images generated by the present application can effectively replace real images, greatly reducing the demand for real images which are difficult to collect and label.

[0025] (3) The high-fidelity simulation image synthesis method based on domain migration provided by the present application has good application prospects and can be used in various application scenarios that require a large amount of diverse data for training, such as artificial intelligence, autonomous driving, robotics, etc. BRIEF DESCRIPTION OF DRAWINGS

[0026] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiment or prior art description.

[0027] Figure 1 is a flowchart of the high-fidelity simulation image synthesis method based on domain migration provided by the present application.

[0028] Figure 2is a structural schematic diagram of a two-stage domain migration neural network provided by the present application.

[0029] Figure 3 is a structural schematic diagram of a high-fidelity simulation image synthesis system based on domain migration provided by the present application. DETAILED DESCRIPTION

[0030] To make the objects, technical solutions and advantages of the present application clearer, the following further describes the present application in detail with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely intended to explain the present application and do not limit the protection scope of the present application.

[0031] For the problems that the simulation images synthesized by the existing method are poor in fidelity or cannot guarantee the authenticity of the geometric structure, the high-fidelity simulation image synthesis method based on domain migration provided by the present application can greatly improve the authenticity of the image while guaranteeing the authenticity of the geometric structure in the image, thereby realizing efficient and low-cost synthesis of a large number of high-fidelity simulation images, as shown in the following Figure 1 As shown in the following, it is a flowchart of the high-fidelity simulation image synthesis method based on domain migration, including the following steps: Step 1: Obtain a plurality of real scene images, obtain real scene object foreground and real scene background images through image segmentation, and complete the real scene background image to obtain a complete real scene background image. Specifically as follows: A camera collects a real scene image of a target domain, image segmentation is performed by using a segmentation large model or a manual labeling method to obtain an object mask in the real scene image. The real scene image is segmented by using the object mask to obtain real scene object foreground and real scene background images.

[0032] When the real scene object mask is used to remove the foreground object, there are multiple empty parts in the real scene image, resulting in an incomplete image. For the real scene background image with empty parts after removing the foreground object, a pre-trained diffusion model is used in combination with the real scene object mask to repair and complete the incomplete real scene image to obtain a complete and real real scene background image.

[0033] Step 2: Build a simulation scene, generate a plurality of simulation scene images and corresponding simulation scene object masks in the simulation scene by using object 3D models and interference object 3D models, and segment the simulation scene object masks to obtain simulation scene object foreground. Specifically as follows: A simulation scene is built in the Blender simulation software. First, a blank room is set, including a floor and four walls, then object 3D models and interference 3D models are placed in the simulation scene, and then a light source is set. The background texture, background material, position and posture of the object and the interference, the surface optical parameters of the object and the interference, the light source position, the light source parameters and other parameters in the simulation scene are set to be domain randomized, so as to generate a plurality of different simulation scenes.

[0034] After the simulation scene construction and the domain randomization setting are completed, the simulation physical engine and the optical engine in the Blender simulation software are used for image rendering to obtain a plurality of simulation scene images. In the rendering process, the corresponding simulation scene object mask can be obtained at the same time, and the simulation scene images are segmented by using the masks to obtain the simulation scene object foreground.

[0035] Step 3: Taking the real scene object foreground as a reference, a two-stage domain migration neural network is established, the real scene object foreground and the simulation scene object foreground are taken as inputs, the style migration of the simulation scene object foreground is performed, and a high-fidelity synthesized object foreground is output. A two-stage domain migration neural network is constructed, the real scene object foreground is taken as a reference, and the style migration of the simulation scene object foreground is performed. The specific structure of the two-stage domain migration neural network is shown in the accompanying Figure 2

[0036] ​The first-stage neural network is an encoder-decoder type network. The network input consists of a simulated foreground image and a real foreground image, and the output is a synthetic foreground image of an object with a consistent artistic style. In the encoding stage, a first extraction module performs step-by-step feature extraction to align low-frequency features. This first extraction module consists of two encoding layers, a foreground perception layer, and a low-pass filter in sequence. The encoding layers extract image features layer by layer; the foreground perception layer performs pixel-level selection on key foreground elements, outputting the filtered features; the low-pass filter preserves low-frequency information, outputting the low-frequency feature representation, which is the first extracted feature. The encoding and decoding stages are connected by an encoding layer, a whitening-coloring transformation layer, and a decoding layer. The first extracted features output from the encoding stage are processed by the encoding layer for feature extraction; the whitening-coloring transformation layer standardizes and aligns the cross-domain feature distribution, outputting aligned features; and the decoding layer restores the dimensions of the aligned features, outputting high-resolution features. In the decoding stage, the high-resolution features are used as input, and a first restoration module performs step-by-step image restoration. Each first restoration module consists of an upsampling layer, a decoding layer, a whitening-coloring transformation layer, and another decoding layer in sequence. The upsampling layer and the decoding layer restore spatial resolution and compensate for feature details, outputting higher-resolution features. The whitening-coloring transformation layer applies cross-domain style consistency constraints to the higher-resolution features. The decoding layer then decodes and reconstructs the style-consistent features, outputting a synthetic object foreground with a consistent artistic style.

[0037] In addition, a high-pass module provides a skip connection between each first extraction module and each first restoration module. This high-pass module takes the pixel-filtered features from the first extraction module as input and performs high-frequency feature transformation. Each high-pass module consists of a whitening-coloring transformation layer, a high-pass filter, and a Gaussian blur layer. The whitening-coloring transformation layer achieves cross-domain alignment of high-frequency features; the high-pass filter preserves high-frequency information and outputs the aligned high-frequency feature map; the Gaussian blur layer smooths high-frequency noise and suppresses redundancy, outputting the transformed high-frequency features.

[0038] The second-stage neural network is also an encoder-decoder type. The network input consists of a synthetic foreground image with a consistent artistic style and a real foreground image. In the encoding stage, a second extraction module performs step-by-step feature extraction. Each second extraction module consists of an encoding layer, a whitening-coloring transformation layer, an encoding layer, a foreground perception layer, and a low-pass filter in sequence. Specifically, the first encoding layer performs preliminary feature extraction, the whitening-coloring transformation layer aligns cross-domain feature distributions, the second encoding layer further compresses and enhances deep semantic information, and the foreground perception layer performs pixel-level selection of key foreground elements to obtain pixel-selected features. These features are then filtered by a low-pass filter to retain low-frequency components and suppress noise interference, outputting the second extracted features.

[0039] The encoding and decoding stages are connected by an encoding layer, a whitening-coloring transformation layer, and a decoding layer. Specifically, the second extracted feature output from the encoding stage undergoes feature compression and distribution reshaping in the encoding layer, cross-domain feature standardization is achieved in the whitening-coloring transformation layer, and semantic information decoding and feature restoration are performed in the decoding layer to output new features.

[0040] The network composition in the decoding stage is the same as that in the encoding and decoding neural network of the first stage. Style transfer of low-frequency features is achieved by the aforementioned main encoding / decoding network. Between each second extraction module and the second restoration module, a high-pass filter is used as input to skip connections with the pixel-selected features from the second extraction module, achieving invariant alignment of high-frequency features and outputting a high-fidelity synthetic object foreground.

[0041] Step 4: Perform background expansion on the real scene background image to obtain multiple high-fidelity composite background images. Then, merge the high-fidelity composite foreground image with the high-fidelity composite background image to obtain a high-fidelity simulation image.

[0042] Specifically, the complete real-scene background image obtained in step 1 is processed using conventional image processing methods such as random cropping, rotation, scaling, affine transformation, and translation to obtain an enhanced real-scene background image.

[0043] Using an enhanced real-world background image as input, a pre-trained diffusion model is used in conjunction with manually provided scene-related text descriptions to output a richer, more faithful synthetic background image after expansion and completion.

[0044] The high-fidelity foreground and background images of the synthesized object are fused using the Alpha fusion method to obtain a complete high-fidelity simulation image.

[0045] On the other hand, the present invention also provides a high-fidelity simulation image synthesis system based on domain migration, for implementing the high-fidelity simulation image synthesis method based on domain migration, as shown in the appendix.Figure 3 As shown, the high-fidelity simulation image synthesis system includes: The real scene processing unit is used to acquire several real scene images, obtain real scene object foreground and real scene background images through image segmentation, and complete the real scene background image to obtain a complete real scene background image. The simulation scene generation unit is used to build a simulation scene. In the simulation scene, multiple simulation scene images and corresponding simulation scene object masks are generated using the 3D models of objects and 3D models of interference objects, respectively. The simulation scene object masks are then used for segmentation to obtain the foreground of the simulation scene objects. The domain transfer neural network unit is used to establish a two-stage domain transfer neural network with real scene object foreground as reference. It takes real scene object foreground and simulated scene object foreground as input, performs style transfer on simulated scene object foreground, and outputs high-fidelity synthetic object foreground. The background expansion and compositing unit is used to expand the background image of a real scene to obtain multiple high-fidelity composite background images. The high-fidelity composite foreground image and the high-fidelity composite background image are then fused together to obtain a high-fidelity simulation image.

[0046] To verify the effectiveness of the proposed domain-transfer-based high-fidelity simulated image synthesis and system, experimental verification was conducted. Using the method proposed in this invention, any number of similar synthesized images can be generated from a single reference target domain image, and the fidelity of the synthesized images is very high. Evaluations on multiple international public datasets show that, compared to comparative methods, this invention achieves nearly 100 times the similarity to the target domain. The synthesized images produced by this method can achieve training results similar to or even better than real images. On international public datasets for image segmentation tasks, the baseline method uses 113,198 real images for training to achieve a segmentation accuracy of 0.763, while the method proposed in this invention only needs 48 images as references, and the synthesized 3,000 images are used for training to achieve a segmentation accuracy of 0.762. Furthermore, since the images generated by this method are self-annotated, significant annotation costs are saved. Therefore, the images generated by this invention can effectively replace real images, greatly reducing the need for real images that are difficult to collect and annotate.

[0047] The specific embodiments described above illustrate the technical solution and beneficial effects of the present invention in detail. It should be understood that the above description is only the most preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, additions, and equivalent substitutions made within the scope of the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A high-fidelity simulation image synthesis method based on domain migration, characterized by, The method comprises the following steps: Step 1: Obtain a plurality of real scene images, obtain a real scene object foreground and a real scene background image through image segmentation, and complete the real scene background image to obtain a complete real scene background image; Step 2: Build a simulation scene, generate a plurality of simulation scene images and corresponding simulation scene object masks in the simulation scene by using an object 3D model and an interference object 3D model, and segment the simulation scene object masks to obtain a simulation scene object foreground; Step 3: Take the real scene object foreground as a reference, establish a two-stage domain migration neural network, input the real scene object foreground and the simulation scene object foreground, perform style migration on the simulation scene object foreground, and output a high-fidelity synthesized object foreground; Step 4: Perform background extension on the real scene background image to obtain a plurality of high-fidelity synthesized background images, and fuse the high-fidelity synthesized object foreground and the high-fidelity synthesized background images to obtain a high-fidelity simulation image.

2. The domain migration based high-fidelity simulation image synthesis method according to claim 1, characterized in that, In step 2, the simulation scene is built based on a Blender simulation software, the object 3D model and the interference object 3D model are set and domain randomization is performed, a plurality of simulation scenes are generated, and the domain randomization includes randomization of background texture, background material, positions and postures of the object and the interference object, surface optical parameters of the object and the interference object, light source position, and light source parameters in the simulation scene. Based on the plurality of simulation scenes, an image is rendered by using a simulation physics engine and an optical engine to obtain a plurality of simulation scene images and corresponding simulation scene object masks. In step 3, the two-stage domain migration neural network comprises a first-stage coding and decoding form neural network and a second-stage coding and decoding form neural network.

3. The domain migration based high-fidelity simulation image synthesis method of claim 1, wherein, The first-stage coding and decoding form neural network is used for inputting the real scene object foreground and the simulation scene object foreground, extracting features by a first extraction module in a coding stage to realize low-frequency feature alignment and obtain first extraction features, decoding and reconstructing features by a first restoration module in a decoding stage, and outputting a synthesized object foreground with consistent artistic style. The second-stage coding and decoding form neural network is used for inputting the synthesized object foreground with consistent artistic style and the real scene object foreground, extracting features by a second extraction module in a coding stage to obtain second extraction features, decoding and reconstructing features by a second restoration module in a decoding stage, and outputting a high-fidelity synthesized object foreground. In the first-stage and second-stage coding and decoding form neural networks, the coding stage and the decoding stage are connected in sequence by a coding layer, a whitening-coloring transformation layer, and a decoding layer.

4. The domain migration based high-fidelity simulation image synthesis method according to claim 3, characterized in that, ​ In the first stage of the encoding and decoding form of the neural network, the first extracted feature output by the encoding stage is subjected to feature extraction by the encoding layer, standardization and alignment of cross-domain feature distribution are completed by the whitening-coloring transformation layer, the aligned feature is output, and the dimension of the aligned feature is reduced by the decoding layer, and the high-resolution feature is output, which is used as the input of the first restoration module; In the second stage of the encoding and decoding form of the neural network, the second extracted feature output by the encoding stage is subjected to feature compression and distribution reorganization by the encoding layer, cross-domain feature standardization is realized by the whitening-coloring transformation layer, and semantic information decoding and feature restoration are realized by the decoding layer, and the new feature is output, which is used as the input of the second restoration module.

5. The domain migration based high-fidelity simulation image synthesis method according to claim 3, characterized in that, The first extraction module is used to take the real scene object foreground and the simulated scene object foreground as input, extract image features layer by layer through the encoding layer, select the extracted image features at the pixel level through the foreground perception layer, output the pixel-selected features, output the low-frequency features through the low-pass filter, and obtain the first extracted features. The first restoration module is used to take the high-resolution feature as input, restore the spatial resolution and compensate the feature details through the up-sampling layer and the decoding layer, output the higher-resolution feature, realize the cross-domain style consistency constraint on the higher-resolution feature through the whitening-coloring transformation layer, and decode and reconstruct the feature subjected to the style consistency constraint through the decoding layer, and output the synthesized object foreground with artistic style consistency.

6. The domain migration based high-fidelity simulation image synthesis method according to claim 5, characterized in that, There is a high-pass module between the first extraction module and the first restoration module for skip connection; the high-pass module is used to take the pixel-selected features in the first extraction module as input, realize cross-domain alignment of high-frequency features, retain high-frequency information, and smooth and suppress high-frequency noise through the whitening-coloring transformation layer, the high-pass filter, and the Gaussian blur layer in sequence, and output the feature after high-frequency feature conversion.

7. The domain migration based high-fidelity simulation image synthesis method according to claim 3, characterized in that, The second extraction module is used to take the synthesized object foreground with artistic style consistency and the real scene object foreground as input, perform preliminary feature extraction through the encoding layer, complete cross-domain feature distribution alignment through the whitening-coloring transformation layer, compress and enhance deep semantic information through the encoding layer, obtain the pixel-selected features through the foreground perception layer, retain the low-frequency components and suppress noise interference through the low-pass filter, and output the second extracted features. The second restoration module is used to take the new feature as input, realize style conversion of the low-frequency feature through the up-sampling layer, the decoding layer, the whitening-coloring transformation layer, and the decoding layer, and output the high-fidelity synthesized object foreground.

8. The domain migration based high-fidelity simulation image synthesis method according to claim 7, characterized in that, There is a high-pass filter between the second extraction module and the second restoration module for skip connection, which takes the pixel-selected features in the second extraction module as input to realize invariant alignment of high-frequency features.

9. The domain migration based high-fidelity simulation image synthesis method according to claim 1, characterized in that, The background extension of the real scene background image obtains multiple high-fidelity synthesized background images, comprising: performing image processing on the complete real scene background image obtained in step 1 to obtain an enhanced real scene background image, taking the enhanced real scene background image as input, combining the scene-related text description driven by the domain knowledge through the pre-trained diffusion model, and outputting the synthesized background image after background extension.

10. A high-fidelity simulation image synthesis system based on domain migration, characterized in that, The high-fidelity simulation image synthesis system is used to implement the domain migration-based high-fidelity simulation image synthesis method of any one of claims 1-9, and the high-fidelity simulation image synthesis system comprises: A real scene processing unit is configured to obtain a plurality of real scene images, obtain real scene object foreground and real scene background images through image segmentation, complete the real scene background image, and obtain a complete real scene background image; A simulation scene generation unit is configured to build a simulation scene, generate a plurality of simulation scene images and corresponding simulation scene object masks in the simulation scene using object 3D models and interference object 3D models, and obtain simulation scene object foreground by using the simulation scene object masks for segmentation; A domain migration neural network unit is configured to take the real scene object foreground as a reference, establish a two-stage domain migration neural network, take the real scene object foreground and the simulation scene object foreground as input, perform style migration on the simulation scene object foreground, and output high-fidelity synthesized object foreground; A background extension and synthesis unit is configured to perform background extension on the real scene background image to obtain multiple high-fidelity synthesized background images, and fuse the high-fidelity synthesized object foreground and the high-fidelity synthesized background images to obtain high-fidelity simulation images.

Images