A small dataset craniofacial translation method based on gan

Abstract

The application discloses a small data set craniomaxillofacial translation method based on GAN, which comprises the following steps: 1, collecting skull and facial CT image data; 2, performing image preprocessing, three-dimensional reconstruction and fairing treatment on the skull and facial CT image data to obtain complete three-dimensional models of the skull and the face; 3, placing the three-dimensional models of the skull and the face in the Frankfurt coordinate system to perform normalization operation; 4, performing vertical mapping of the three-dimensional models of the skull and the face on the XOZ plane in the Frankfurt coordinate system to obtain the front view images of the skull and the face; 5, introducing a Gaussian pyramid into a GAN network to construct a network model PCC-GAN for skull and facial translation; 6, training network parameters of the pyramid cycle consistency generative adversarial network model PCC-GAN; and 7, placing the skull and facial images into the craniomaxillofacial translation model PCC-GAN to generate two-dimensional skull and facial images, and more accurate and real facial images can be generated under the condition of less point cloud data.

Description

Technical Field

[0001] This invention belongs to the field of cranial reconstruction technology, specifically a craniofacial translation method based on a small dataset using GAN. Background Technology

[0002] The face is a unique feature of human visual appearance, and facial reconstruction from the skull is an important technology for skull identification. For example, when encountering an unidentified skull, craniofacial reconstruction technology can be used to restore the appearance and compare it with a database to identify the skull, which is of great reference value to the police in solving cases. Early three-dimensional craniofacial reconstruction predicted the geometric structure of the face from the skull based on the three-dimensional relationship between the skull and the face. It has been widely used in archaeology, criminal investigation, forensic medicine, and cosmetic medicine, and is playing an increasingly important role.

[0003] In recent years, computer-aided craniofacial reconstruction has attracted widespread attention. By utilizing computer technology, it can significantly reduce working time and difficulty, offering a more flexible and efficient reconstruction method. Currently, computer-aided craniofacial reconstruction mainly includes two methods: knowledge analysis models and statistical learning models. However, these methods suffer from drawbacks such as large point cloud data volumes, inaccurate feature point calibration methods, insufficient number of points to describe craniofacial morphology, incomplete description of craniofacial morphological changes, and long processing times.

[0004] Currently, image-to-image translation technology is gradually being widely used. It is implemented using generative adversarial networks (GANs), which aim to learn mapping functions between domains to transform the content or style of an image from the original domain X to another image domain Y. A generative adversarial network (GAN) is a combination of a generator and a discriminator. The generator's goal is to learn to generate images by mapping from latent codes, while the discriminator learns to distinguish between real images and images generated by the generator. In this process, the generator and the discriminator compete with each other, thereby generating images that are difficult to distinguish from real images.

[0005] Therefore, it is hoped that an image translation technology based on generative adversarial networks (GANs) can be proposed to achieve craniofacial reconstruction, thereby solving the problems of large point cloud data volume, long completion time and low model accuracy in the existing three-dimensional craniofacial reconstruction process. Summary of the Invention

[0006] To address the shortcomings of existing technologies, the purpose of this invention is to provide a GAN-based craniofacial translation method for small datasets, which can generate more accurate and realistic craniofacial images with less point cloud data.

[0007] To achieve the above objectives, the present invention employs the following technical solution:

[0008] A craniofacial translation method based on a small dataset using GANs, characterized by the following steps:

[0009] Step 1: Acquire CT images of the skull and face;

[0010] Step 2: Perform image preprocessing, 3D reconstruction, and smoothing on the CT image data of the skull and face to obtain a complete 3D model of the skull and face;

[0011] Step 3: Place the 3D model of the skull and face in the Frankfurt coordinate system and perform normalization.

[0012] Step 4: In the Frankfurt coordinate system, the three-dimensional models of the skull and face are vertically mapped to the XOZ plane to obtain the front view images of the skull and face.

[0013] Step 5: Introduce a Gaussian pyramid into the GAN network to construct a pyramid cycle consistency generative adversarial network model PCC-GAN for skull and face translation;

[0014] Step 6: Train the network parameters of the Pyramid Cyclic Consistency Generative Adversarial Network (PCC-GAN) model;

[0015] Step 7: Input the skull and facial images into the craniofacial translation model PCC-GAN to generate two-dimensional skull and facial images.

[0016] Furthermore, the specific process of acquiring skull and facial CT data in step 1 is as follows: using computed tomography (CT) technology, a fine scan of the thick slices of the skull is performed using X-ray beams, gamma rays, or ultrasound to obtain several consecutive skull and facial CT slice images.

[0017] Furthermore, the process of training the Pyramid Cyclic Consistency Generative Adversarial Network (PCC-GAN) model in step 6 is as follows:

[0018] Step 6.1: Train the PCC-GAN network training model on the skull and face paired datasets to learn the mapping between the image translation domain X and domain Y;

[0019] Step 6.2: Use the adversarial loss function L GAN Cyclic consistency loss function L cyc (G,F) and multi-scale loss function L multi-scale The overall loss function of the network structure for optimizing the image translation domain transformation problem is expressed as:

[0020] Lall=λ1L GAN +λ2L cyc (G,F)+λ3L multi-scale (2)

[0021] In the formula, λ1, λ2, and λ3 are respectively L GAN L cyc (,F) and L multi-scale The hyperparameter values ​​are: G is the generator for domain X -> domain Y, and F is the generator for domain Y -> domain X.

[0022] Furthermore, step 6.1 specifically includes the following steps:

[0023] Step 6.1.1: Extract feature maps from skull and facial images and input them into the PCC-GAN network. After learning, the generator G generates skull and facial images.

[0024] Step 6.1.2: Apply Gaussian convolution to scale the generated skull and facial images, then downsample the last layer of the obtained feature map, and then apply Gaussian convolution to scale four times to obtain the first octave. After downsampling the last layer of the first octave and applying Gaussian convolution to scale four times, the second octave is obtained. After downsampling the last layer of the second octave and applying Gaussian convolution to scale four times, the third octave is obtained.

[0025] Step 6.1.3: Perform transformation enhancement on the image generated by generator G that is about to be input into discriminator D. The transformed and enhanced image and the target image are then input into discriminator D.

[0026] Furthermore, the specific process of scale transformation in step 6.1.2 is as follows:

[0027] Step 6.1.2.1: Smooth the generated image and target image using a low-pass filter. The low-pass filter uses a Gaussian kernel G. σ G σ It is a two-dimensional Gaussian kernel with standard deviation σ = 1, expressed as:

[0028]

[0029] In the formula, x is the x-coordinate of the pixel and y is the y-coordinate of the pixel;

[0030] Step 6.1.2.2: Downsample the smoothed generated image and the target image to remove redundant pixels and reduce the image resolution;

[0031] Step 6.1.2.3: Perform multiple Gaussian convolutions to achieve scale transformation and obtain feature maps of the same size.

[0032] Furthermore, the transformation enhancement in step 6.1.3 includes pixel transformation, geometric transformation, color transformation, image spatial filtering, additive noise, and cropping;

[0033] The pixel transformations include x-flip, 90° rotation, and integer translation.

[0034] Furthermore, the intensity P value of the transformation enhancement in step 6.1.3 is dynamically adjusted according to the degree of overfitting, where overfitting is expressed as:

[0035]

[0036] In the formula, D train It is the output of discriminator D. It is the average of N consecutive mini-batches, N=4. When r=0, it means there is no overfitting, and when r=1, it means complete overfitting.

[0037] By initializing the P value to zero and incrementing or decrementing it by a fixed amount, enabling it to rapidly increase from 0 to 1, an adaptive discriminator is implemented to enhance ADA.

[0038] Furthermore, the target value of r is 0.6, and the P value is controlled below 0.8.

[0039] Furthermore, step 6.2 specifically includes the following steps:

[0040] Step 6.2.1, the adversarial loss function for the generator G is expressed as:

[0041]

[0042] In the formula, D Y It is the discriminator of the generator G from domain X to domain Y;

[0043] The adversarial loss function for generator F is expressed as:

[0044]

[0045] In the formula, D X It is the discriminator of the generator D from domain Y to domain X;

[0046] Therefore, the total adversarial loss function L GAN Represented as:

[0047] L GAN =L GAN (G,D Y ,X,U)+L GAN (F,D X ,X,U) (5)

[0048] Step 6.2.2: To constrain the consistency of content information between the generated image and the target image, a cycle consistency loss function L is introduced. cyc(G,F) ensures that the image in domain X, after passing through generator G, is input into generator F, and the generated image remains consistent with the original image x in domain X, i.e., x->G(x)->F(G(x))≈x. Similarly, the image y in domain Y also satisfies the inverse cycle consistency loss, i.e., y->F(y)->G(F(y))≈y; the cycle consistency loss function L cyc (G,F) is represented as:

[0049]

[0050] In the Gaussian pyramid, the loss is calculated by extracting the first layer of the image in each octave. The loss function is expressed as:

[0051]

[0052] In formulas (6) and (7), F i (y) and G represent the Gaussian filtering operation and the generator, respectively; i is the octave number, and each octave represents a different scale; ||||smoot L1 represents the value of calculating the smoothL1 loss; x, y, and z represent the input image, target image, and random noise in the target domain, respectively; x ~ p data (x) represents an image x, y ~ p from the real domain X. data (y) represents the image y from the real domain Y;

[0053] Step 6.2.3, Multi-scale loss function L multi-scale Represented as:

[0054]

[0055] In the formula, λ i This represents the weight of scale i.

[0056] Compared with the prior art, the present invention has the following technical effects:

[0057] This invention transforms the craniofacial reconstruction problem into the translation of skull images to generate facial images, referred to as craniofacial translation, to complete the identification of unknown skulls. Since most of the facial features with high discriminative power are concentrated in the front view, the generated two-dimensional facial images can be used for the identification of unknown skulls. Compared with three-dimensional craniofacial reconstruction technology, the two-dimensional craniofacial image generation technology using PCC-GAN network has two main advantages: (1) It has a great ability to capture the complex relationship between skull and facial features; (2) It does not require three-dimensional scanning data of the entire head. During the training phase, only pairs of skull and facial front views are needed, and during the application phase, only one skull front view is needed, which greatly simplifies the facial generation steps. Moreover, this invention constrains the generated images at multiple scales. The scale transformation learns image details in the early stage and image contours in the later stage, extracting more comprehensive skull morphological information and generating facial images that are closer to the target.

[0058] To address the problem of discriminator overfitting that easily occurs with small datasets, this invention introduces an adaptive discriminator enhancement mechanism. This mechanism significantly stabilizes training under limited data conditions, and the generated facial images are superior in both visual quality and image quality evaluation metrics, making them more accurate and realistic. Attached Figure Description

[0059] Figure 1 This is a flowchart of the present invention;

[0060] Figure 2 The training process of the PCC-GAN model of this invention;

[0061] Figure 3 Facial images generated for this invention;

[0062] Figure 4 This is a flowchart of the adaptive discriminator mechanism of the present invention;

[0063] Figure 5 This is a comparison image of craniofacial images generated by the present invention and existing models. Detailed Implementation

[0064] The specific content of the present invention will be further explained in detail below with reference to the embodiments.

[0065] The three-dimensional reconstruction process of craniofacial data in this embodiment can be referenced in: Lin Pengyue. Research on cranial and facial reconstruction and realism processing based on generative adversarial networks [D]. Northwest University, 2022. DOI:10.27405 / d.cnki.gxbdu.2022.000244).

[0066] like Figure 1 As shown, a PCC-GAN method for craniofacial translation on a small dataset based on GANs includes the following steps:

[0067] Step 1: Acquire CT image data of the skull and face with a size of 256*256: First, use computed tomography (CT) technology to perform a detailed scan of the thick layer of the skull using X-ray beam, gamma rays, and ultrasound. Then, input the obtained CT data of the skull and face into the computer to obtain cross-sectional images of the head. In order to represent the shape of the entire craniofacial region, many continuous cross-sectional images of the skull and face are needed. The most common of these is the cross-sectional image. Since CT slices can describe the specific morphological structure of the craniofacial region through three-dimensional reconstruction technology, this embodiment uses CT scanning to obtain skull and face image data.

[0068] Step 2, 3D Reconstruction of Craniofacial Data: In order to obtain a more intuitive and complete craniofacial morphology and structure, it is necessary to perform image preprocessing, 3D reconstruction and smoothing on the collected skull and facial CT image data to obtain a complete 3D model of the skull and face, which will lay the foundation for subsequent craniofacial restoration and realistic processing.

[0069] Step 3: Place the skull and facial data in the Frankfurt coordinate system and perform normalization to remove factors that are irrelevant to the restoration task, such as pose and scale, to ensure that the experimental results are not interfered with.

[0070] Step 4: Acquisition and processing of craniofacial front view image: In the Frankfurt coordinate system, the three-dimensional model of the skull and face is vertically mapped to the XOZ plane to obtain the front view image of the skull and face.

[0071] Step 5: Introduce a Gaussian pyramid into the GAN network to construct the Pyramid Cyclic Consistency Generative Adversarial Network (PCC-GAN) model for skull and face translation. Its structure is as follows: Figure 2 As shown;

[0072] Step 6: Train the network parameters of the Pyramid Cyclic Consistency Generative Adversarial Network (PCC-GAN) model, which includes the following steps:

[0073] Step 6.1: Input the paired skull and facial images into the PCC-GAN network training model for learning and training, learning the mapping between the image translation domain X and domain Y. The specific process is as follows:

[0074] Step 6.1.1: Extract feature maps from skull and facial images and input them into the PCC-GAN network. After learning, the generator G generates skull and facial images.

[0075] Step 6.1.2: Scale the generated skull and facial images using Gaussian convolution. Then, downsample the last layer of the resulting feature map, followed by four scale transformations using Gaussian convolution to obtain the first octave. Downsample the last layer of the first octave and perform four scale transformations using Gaussian convolution to obtain the second octave. Downsample the last layer of the second octave and perform four scale transformations using Gaussian convolution to obtain the third octave. The specific scale transformation process is as follows:

[0076] Step 6.1.2.1: Smooth the generated image and the target image using a low-pass filter. Since the Gaussian kernel is the only linear kernel that can achieve image scaling, using this linear kernel will not introduce other noise. Therefore, the low-pass filter in this embodiment uses a 3×3 Gaussian kernel G. σ G σ It is a two-dimensional Gaussian kernel with standard deviation σ = 1, expressed as:

[0077]

[0078] In the formula, x is the x-coordinate of the pixel and y is the y-coordinate of the pixel;

[0079] Step 6.1.2.2: Downsample the smoothed generated image and the target image to remove redundant pixels and reduce the image resolution;

[0080] Step 6.1.2.3: Perform multiple Gaussian convolutions to achieve scale transformation and obtain feature maps of the same size;

[0081] Step 6.1.3: Perform transformation enhancement on the image generated by generator G that is about to be input into discriminator D. Input the transformed and enhanced image and the target image into discriminator D.

[0082] The transform enhancement includes six categories: pixel transformation, geometric transformation, color transformation, image spatial filtering, additive noise, and shearing. Pixel transformation includes x-flip, 90° rotation, and integer translation. The image evaluation discriminator D is based solely on transform enhancement. Figure 4 As shown, during training, a set of predefined transformations are used to enhance the image to be input to the discriminator D in a fixed order. The enhancement strength is controlled by the probability p∈[0,1]. Each transformation is applied with probability p or skipped with probability 1-p. All transformations use the same p value. Figure 4 The blue elements highlight operations related to amplification, the green boxes represent the network being trained, the orange elements represent the loss function, and the remaining elements perform standard GAN training using the generator G and discriminator D, dynamically adjusting the enhancement strength P value based on the degree of overfitting to avoid manual adjustment. Overfitting is represented as:

[0083]

[0084] In the formula, D train It is the output of discriminator D. It is the average of N consecutive mini-batches. In this example, N=4. When r=0, it means there is no overfitting, and when r=1, it means complete overfitting.

[0085] Control the enhancement intensity P, initialize P to zero, and adjust the P value every 4 small batches according to the overfitting heuristic (9). When the r value is close to 1 or close to 0, it indicates that the fit is too large or too small. By incrementing or decrementing P by a fixed amount, the P value is adjusted so that P can rise from 0 to 1 quickly enough. This variant is called Adaptive Discriminator Enhancement (ADA). When the P value is too high, the generator cannot know which direction the generated image should face. Therefore, in this embodiment, the target value of r is 0.6, and the P value is kept below the safe value of 0.8 so that the image generated by the generator will not produce image penetration and orientation disorder.

[0086] Step 6.2: Use the adversarial loss function L GAN Cyclic consistency loss function L cyc (G,F) and multi-scale loss function L multi-scale The overall loss function of the network structure for optimizing the image translation domain transformation problem is expressed as:

[0087] Lall=λ1L GAN +λ2L cyc (G,F)+λ3L multi-scale (2)

[0088] In the formula, λ1, λ2, and λ3 are respectively L GAN L cyc (,F) and L multi-scale The hyperparameter values ​​are λ1 = 30, λ2 = 35, λ3 = 50 in this embodiment, G is the generator of domain X -> domain Y, and F is the generator of domain Y -> domain X.

[0089] Step 6.2.1, the adversarial loss function for the generator G is expressed as:

[0090]

[0091] In the formula, D Y It is the discriminator of the generator G from domain X to domain Y;

[0092] The adversarial loss function for generator F is expressed as:

[0093]

[0094] In the formula, D xIt is the discriminator of the generator D from domain Y to domain X;

[0095] Therefore, the total adversarial loss function L GAN Represented as:

[0096] L GAN =L GAN (G,D Y ,X,Y)+L GAN (F,D X (5)

[0097] Step 6.2.2: To constrain the consistency of content information between the generated image and the target image, a cycle consistency loss function L is introduced. cyc (G,F) ensures that the image in domain X, after passing through generator G, is input into generator F, and the generated image remains consistent with the original image x in domain X, i.e., x->G(x)->F(G(x))≈x. Similarly, the image y in domain Y also satisfies the inverse cycle consistency loss, i.e., y->F(y)->G(F(y))≈Y; the cycle consistency loss function L cyc (G,F) is represented as:

[0098]

[0099] In a Gaussian pyramid, the first layer and the set of feature maps of the same size produced after four Gaussian convolutions are called an octave. This embodiment uses three octaves, each containing five layers of images. The first layer image in the next octave is obtained by downsampling the last layer image in the previous octave and performing a Gaussian blur operation on it. The loss is calculated by extracting the first layer image of each octave, and the loss function is expressed as:

[0100]

[0101] In formulas (6) and (7), F i (y) and G represent the Gaussian filtering operation and the generator, respectively. i is the number of each octave, i = 1, 2, 3…l, where i represents different scale values, l is the final number of octaves, ||||smoot L1 represents the value of calculating the smoothL1 loss, and x, y, and z represent the input image, target image, and random noise in the target domain, respectively, x~p data (x) represents an image x, y ~ p from the real domain X. data (y) represents the image y from the real domain Y;

[0102] Since the L1 loss in the PCC-GAN network model directly calculates the difference between the generated facial image and the real facial image, it increases the constraints on the generated image. To weaken the constraints brought by the L1 loss, this embodiment replaces the L1 loss function with the SmoothL1 loss function. At the same time, it aligns the generated image and the real image on other scales after downsampling by Gaussian convolution kernels. The scale refers to the coarseness of the image content. The larger the scale, the more blurred the image. Different scales simulate the effect of people viewing images at different distances. Compared with the L1 loss function, the SmoothL1 loss is less sensitive to outliers and anomalies, and the gradient change is relatively smaller. Therefore, the SmoothL1 loss converges faster than the L1 loss function.

[0103] Step 6.2.3, Multi-scale loss function L multi-scale Represented as:

[0104]

[0105] In the formula, λ i Represents the weight of scale i;

[0106] Step 7: Input the skull and facial images into the craniofacial translation model PCC-GAN to generate two-dimensional skull and facial images, such as... Figure 3 As shown.

[0107] In step 6 of this embodiment, the network parameters of the Pyramid Cyclic Consistency Generative Adversarial Network (PCC-GAN) model are trained using Python 3.6 and PyTorch 1.2 and an NVIDIA A6000 graphics card. This invention uses the generator G structure of the ResNet-9 module as the baseline, while the discriminator D structure uses PatchGAN with a batch size of 4. The optimizer used is Adam, where β1 = 0.5 and β2 = 0.999. The total number of training time periods is set to 100, and the learning rate is adjusted at equal intervals, each time decreasing to one-tenth of its original value.

[0108] This embodiment's PCC-GAN network model, combined with Gaussian pyramids, trains on images at multiple scales to extract more comprehensive skull morphology information and generate facial images that more closely resemble the target. For small datasets, this embodiment introduces adaptive discriminator enhancement to prevent overfitting and training divergence. This mechanism significantly stabilizes training under limited data conditions. Figure 5 As shown in the experimental comparison, the image generated by this embodiment (the method of the present invention) is more similar to the target image, and intuitively and appropriately reflects the shape relationship between the skull and the face.

Claims

1. A craniofacial translation method based on a small dataset using GAN, characterized in that, Includes the following steps: Step 1: Acquire CT images of the skull and face; Step 2: Perform image preprocessing, 3D reconstruction, and smoothing on the CT image data of the skull and face to obtain a complete 3D model of the skull and face; Step 3: Place the 3D model of the skull and face in the Frankfurt coordinate system and perform normalization. Step 4: In the Frankfurt coordinate system, the three-dimensional models of the skull and face are vertically mapped to the XOZ plane to obtain the front view images of the skull and face. Step 5: Introduce a Gaussian pyramid into the GAN network to construct a pyramid cycle consistency generative adversarial network model PCC-GAN for skull and face translation; Step 6: Train the network parameters of the Pyramid Cyclic Consistency Generative Adversarial Network (PCC-GAN). The process is as follows: Step 6.1: Train the PCC-GAN network model on the skull and face paired datasets to learn the mapping between the image translation domain X and domain Y. The specific process is as follows: Step 6.1.1: Extract feature maps from skull and facial images and input them into the PCC-GAN network. After learning, the generator G generates skull and facial images. Step 6.1.2: Apply Gaussian convolution to scale the generated skull and facial images, then downsample the last layer of the obtained feature map, and then apply Gaussian convolution to scale four times to obtain the first octave. After downsampling the last layer of the first octave and applying Gaussian convolution to scale four times, the second octave is obtained. After downsampling the last layer of the second octave and applying Gaussian convolution to scale four times, the third octave is obtained. Step 6.1.3: Perform transformation enhancement on the image generated by generator G that is about to be input into discriminator D. Input the transformed and enhanced image and the target image into discriminator D. The intensity P value of the transformation enhancement is dynamically adjusted according to the degree of overfitting, which is expressed as: （9） In the formula, It is the output of discriminator D. It is the average of N consecutive mini-batches, N=4. When r=0, it means there is no overfitting, and when r=1, it means complete overfitting. By initializing the P value to zero and incrementing or decrementing it by a fixed amount, it can be rapidly increased from 0 to 1, thereby enhancing ADA by implementing an adaptive discriminator. Step 6.2: Use the adversarial loss function Cyclic consistency loss function and multi-scale loss function The overall loss function of the network structure for optimizing the image translation domain transformation problem is expressed as: （2） In the formula, , , They are , and The hyperparameter values ​​are: G is the generator for domain X -> domain Y, and F is the generator for domain Y -> domain X. Step 7: Input the skull and facial images into the craniofacial translation model PCC-GAN to generate two-dimensional skull and facial images.

2. The craniofacial translation method based on a small dataset using GAN according to claim 1, characterized in that, The specific process for acquiring skull and facial CT data in step 1 is as follows: using computed tomography (CT) technology, using... Fine scanning of thick sections of the skull using beam, gamma rays, or ultrasound yields several consecutive CT slice images of the skull and face.

3. The craniofacial translation method based on a small dataset using GAN according to claim 1, characterized in that, The specific process of scale transformation in step 6.1.2 is as follows: Step 6.1.2.1: Smooth the generated image and the target image using a low-pass filter. The low-pass filter uses a Gaussian kernel. , Standard deviation The two-dimensional Gaussian kernel is represented as: （1） In the formula, x is the x-coordinate of the pixel and y is the y-coordinate of the pixel; Step 6.1.2.2: Downsample the smoothed generated image and the target image to remove redundant pixels and reduce the image resolution; Step 6.1.2.3: Perform multiple Gaussian convolutions to achieve scale transformation and obtain feature maps of the same size.

4. The craniofacial translation method based on a small dataset using GAN according to claim 1, characterized in that, The transformation enhancement in step 6.1.3 includes pixel transformation, geometric transformation, color transformation, image spatial filtering, additive noise, and cropping; The pixel transformations include x-flip, 90° rotation, and integer translation.

5. The craniofacial translation method based on a small dataset using GAN according to claim 1, characterized in that, The target value for r is 0.6, and the P value is controlled below 0.

8.

6. The craniofacial translation method based on a small dataset using GAN according to claim 1, characterized in that, Step 6.2 specifically includes the following steps: Step 6.2.1, the adversarial loss function for the generator G is expressed as: （3） In the formula, It is the discriminator of the generator G from domain X to domain Y; The adversarial loss function for generator F is expressed as: （4） In the formula, It is the discriminator of the generator D from domain Y to domain X; Therefore, the total adversarial loss function Represented as: （5） Step 6.2.2: To constrain the consistency of content information between the generated image and the target image, a cycle consistency loss function is introduced. The image of domain X is processed by generator G and then input into generator F. The generated image is compared with the original image in domain X. Maintain consistency, that is ->G(x)->F(G(x))≈ Similarly, the image in the Y domain It also satisfies the reverse circular consistent loss, i.e. ->F(y)->G(F(y))≈ Cyclic consistency loss function Represented as: （6） In the Gaussian pyramid, the loss is calculated by extracting the first layer of the image in each octave. The loss function is expressed as: （7） In formulas (6) and (7), and These represent the Gaussian filtering operation and the generator, respectively. These are the numbers for each octave, and each octave represents a different scale. This indicates the value of the smoothL1 loss. , and Let them represent the input image, the target image, and random noise, respectively, in the target domain. Represents an image from the real domain X , Represents an image from the real domain Y ; Step 6.2.3, Multi-scale loss function Represented as: （8） In the formula, This represents the weight of scale i.

Images