Spacecraft image domain gap removal generation method and system based on supervised pairing GAN

Abstract

The invention provides a spacecraft image domain gap removing generation method and system based on supervised pairing GAN, and belongs to the field of spacecraft image processing. The problem that an existing generative adversarial network is difficult to effectively reduce a domain gap between simulation data and a real shooting data set is solved. According to the invention, the input image is preprocessed through the ConvLSTM module, so that the image generation quality and the detail recovery capability are improved; a GAN generative adversarial network comprising a generator and a discriminator is designed, and high-quality and high-consistency image generation is realized through adversarial training of the generator and the discriminator. According to the method, a domain gap between simulation data and a real shooting data set is effectively reduced, and a high-fidelity spacecraft image data set is generated, so that the precision and the reliability of a deep learning model in a spacecraft attitude estimation visual task are improved, the dependence on expensive and limited real image data is reduced, and the robustness of the spacecraft attitude estimation visual task is improved. And the adaptive capacity of the model in complex illumination and diversified scenes is enhanced.

Description

Technical Field

[0001] This invention relates to the field of spacecraft image processing technology, and more specifically, to a method and system for generating spacecraft image domain gaps based on supervised paired GANs. Background Technology

[0002] With the widespread application of deep learning technology in fields such as spacecraft autonomous navigation and attitude estimation, the demand for high-quality, large-scale, and diverse spacecraft image datasets for vision-based deep learning models is increasing. These models rely on rich training data to capture the characteristics of spacecraft under different attitudes, lighting, and environmental conditions, thereby achieving precise navigation and attitude control. However, real spacecraft image datasets are limited in number and do not comprehensively cover all attitude and lighting conditions due to high acquisition costs and complex acquisition processes, making it difficult to meet the needs of deep learning models for large-scale and diverse data. While ground-based spacecraft images are relatively rich in detail, they cannot fully reflect the complexities of the real space environment due to limitations imposed by ground lighting and shooting angles, which to some extent limits the effectiveness of models in actual space missions.

[0003] To compensate for the lack of real-world image datasets, researchers typically employ virtual synthetic datasets, generating spacecraft images using simulation software such as OpenGL, Blender, and Unreal Engine. These synthetic images can be generated quickly and in large quantities, significantly increasing the size and diversity of the dataset. However, despite their numerical advantage, synthetic images differ significantly from real-world images in terms of texture detail, lighting features, and background environment. This "domain gap" problem means that while models perform well when trained on simulated data, their generalization ability is limited in real-world applications due to the domain gap, making it difficult to achieve the expected performance in real-world space environments.

[0004] Current solutions to the domain gap problem mainly include domain randomization, domain adaptation, and generative adversarial networks (GANs). Domain randomization introduces more randomness into synthetic data, enabling the model to have stronger generalization ability and adapt to the diversity of real-world situations. Domain adaptation adjusts the model or data distribution, allowing the model to better transfer learning between the source domain (synthetic data) and the target domain (real data). However, these methods still have certain limitations in narrowing the gap between synthetic and real images, especially in preserving image details and edge sharpness. Generative adversarial networks (GANs), as powerful generative models, can improve the realism of synthetic images to some extent through adversarial training between the generator and the discriminator, but further optimization is still needed in detail preservation and edge processing. Therefore, there is an urgent need for a more effective method that can quickly generate large-scale, high-quality spacecraft images while minimizing the domain gap between synthetic and real images to meet the application requirements of deep learning models in practical space missions. Summary of the Invention

[0005] The technical problem to be solved by this invention is:

[0006] To address the problem that existing generative adversarial networks struggle to effectively bridge the domain gap between simulated data and real-world shooting datasets.

[0007] The technical solution adopted by the present invention to solve the above-mentioned technical problems is as follows:

[0008] This invention provides a method for generating spacecraft image domain gaps based on supervised paired GANs, comprising the following steps:

[0009] S100, Image data preprocessing, including preprocessing the input data, wherein the preprocessing involves pairing and normalizing the synthetic image and the real-shot image; wherein the synthetic image is the input source and the real image is the target output.

[0010] The S200 and ConvLSTM convolutional long short-term memory network module is designed to process the long-term and short-term dependencies of image sequences through LSTM units, preserving the spatial structural features of the images. Each ConvLSTM unit in the ConvLSTM convolutional long short-term memory network module processes the input image sequence through time steps and generates a feature representation that comprehensively considers temporal information by passing hidden state and memory state information. The final hidden state is then passed as input to the generator of the generative adversarial network.

[0011] S300 and GAN generative adversarial network design, including generator and discriminator.

[0012] The generator adopts a U-Net structure, which captures feature information at different levels and preserves the detailed features of the input image through the forward and deconvolution operations of the encoder and decoder.

[0013] The discriminator uses a PatchGAN structure to divide the input image into multiple small regions and minimizes the loss function to improve the discriminator's ability to judge real images, thereby guiding the generator to optimize the output.

[0014] S400, Model Training: The synthesized image processed in step S200 is input into the GAN generative adversarial network in step S300. The generator processes the synthesized image into an image close to the real image, and the discriminator distinguishes the real image from the generator image. Then, through adversarial training between the generator and the discriminator, a high-quality and highly consistent image generation effect is finally achieved.

[0015] Further, in step S100, the following is included:

[0016] S110, Input data, including real images and synthetic images;

[0017] S120, Image calibration and pairing: Alignment between each pair of real and synthetic images through calibration;

[0018] S130. Data standardization: Normalize the image to ensure that the pixel values ​​of the input data are within the range of [0,1]. At the same time, adjust the size to make the image size uniform. Then, stitch the images together. The synthesized image is input to the ConvLSTM module, and the real image is input to the discriminator in the GAN generative adversarial network.

[0019] Further, in step S200, the following are included:

[0020] The input image sequences obtained in steps S210 and S100 are processed by the ConvLSTM convolutional long short-term memory network module to extract temporal features; after processing, the output features are passed to the generator in the GAN generative adversarial network.

[0021] In the S220 ConvLSTM convolutional long short-term memory network module, the ConvLSTM units generate long-term and short-term memories by capturing the dependencies between each frame of the image; the hidden state ℎ𝑡 and the memory state 𝑐𝑡 of each ConvLSTM unit are updated over time steps and are calculated using the following formula:

[0022]

[0023] in, The input image at the current moment; The current hidden state; The current memory state;

[0024] The ConvLSTM unit operates by inputting the image at each time step. The hidden state of the previous moment and memory state Process them together to calculate the new hidden state. and memory state ;

[0025] S230. The dimensions of the input image sequence are set to (𝐵,5,3,𝐻,𝑊), where 𝐵 is the batch size, 5 is the number of time steps, that is, the input image sequence contains 5 frames of images, 3 is the number of RGB channels in each image, indicating that each image contains three color channels: red, green and blue, and 𝐻 and𝑊 are the height and width of each image, respectively, representing the spatial size of the image;

[0026] The output of the S240 and ConvLSTM convolutional long short-term memory network modules is the hidden state at the last time step. The hidden state This contains all temporal features of the input image sequence; the generator will receive this hidden state. As input, this temporal feature is used to generate the next frame image:

[0027]

[0028] in, This is a generator model that accepts the output of a ConvLSTM module as input. The generated image represents the next frame image generated based on temporal features.

[0029] Furthermore, in step S300, the generator's loss function for:

[0030]

[0031] in, To combat the losses; For L1 loss; For L2 loss; and These represent the weighting coefficients for the L1 and L2 loss terms, respectively, used to control the proportion of their respective losses in the total generator loss; Represents the L1 norm; Represents the L2 norm;

[0032] The generator maximizes the following adversarial loss :

[0033]

[0034] Where 𝐷(𝐺(𝐴)) represents the discriminator's judgment on the authenticity of the generated image, 𝐺(𝐴) is the generated image, 𝐷 is the discriminator; E represents the expectation.

[0035] Further, in step S300, the loss function of the discriminator... for:

[0036]

[0037] in, , .

[0038] A spacecraft image domain gap generation system based on supervised paired GAN, the system has program modules corresponding to the above steps, and executes the steps described above in the improved heuristic static path planning method for mobile robots when running.

[0039] A computer-readable storage medium storing a computer program configured to, when invoked by a processor, implement steps of a spacecraft image domain gap generation method based on supervised pairing GANs.

[0040] Compared with the prior art, the beneficial effects of the present invention are:

[0041] 1. Multi-dimensional optimization of image quality: In terms of image quality improvement, through paired learning between synthetic and real images, the generator utilizes the U-Net architecture and a designed loss function to effectively capture and restore the detailed features of the image, significantly reducing the domain gap between synthetic and real images. The discriminator adopts the PatchGAN architecture, which can accurately evaluate the realism of the image in local regions, further prompting the generator to optimize its output, making the generated spacecraft image highly similar to the real image in terms of texture, edges, and overall structure. At the same time, the introduced ConvLSTM module successfully integrates image sequence features, effectively solving the problems of texture inconsistency and edge blurring caused by the lack of modeling of the overall sequence information when generating a single frame, making the edges of the generated image sharper and clearer, and better preserving detailed information.

[0042] 2. The significant advantages of quantitative indicators are particularly evident in practical monocular spacecraft attitude estimation tasks. When using the generated image dataset for attitude estimation, its performance is extremely close to that of real-world image datasets, especially within a small error range, providing more accurate and reliable data support for spacecraft attitude estimation. In contrast, traditional simulation datasets have significant disadvantages in error range and estimation accuracy, further highlighting the effectiveness and superiority of this invention in solving the domain gap problem, improving image quality, and enhancing practical application performance, providing strong technical support for the development of fields such as spacecraft autonomous navigation.

[0043] 3. Exceptional performance in attitude estimation tasks: The effectiveness of this invention is particularly outstanding in practical monocular spacecraft attitude estimation tasks. When using the generated image dataset for attitude estimation, its performance is extremely close to that of real-world image datasets, especially within a small error range, providing more accurate and reliable data support for spacecraft attitude estimation. In contrast, traditional simulation datasets have significant disadvantages in error range and estimation accuracy, further highlighting the effectiveness and superiority of this invention in solving the domain gap problem, improving image quality, and enhancing practical application performance, providing strong technical support for the development of fields such as spacecraft autonomous navigation. Attached Figure Description

[0044] Figure 1 This is a flowchart of a spacecraft image domain gap generation method based on supervised pairing GAN in an embodiment of the present invention;

[0045] Figure 2 This is a schematic diagram of the SHIRT dataset in an embodiment of the present invention;

[0046] Figure 3 This is a structural diagram of the ConvLSTM module in an embodiment of the present invention;

[0047] Figure 4 This is a supervised pairing generation framework diagram in an embodiment of the present invention;

[0048] Figure 5 This is a diagram of the U-net structure in an embodiment of the present invention;

[0049] Figure 6 This is a diagram of the PatchGAN structure in an embodiment of the present invention;

[0050] Figure 7 This is a diagram illustrating the spacecraft image generation process in an embodiment of the present invention;

[0051] Figure 8 This is a comparison diagram of attitude evaluation in an embodiment of the present invention. Detailed Implementation Plan

[0052] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0053] Specific Implementation Plan 1: Combining Figures 1 to 8 As shown, this invention provides a method for generating spacecraft image domain gaps based on supervised paired GANs, comprising the following steps:

[0054] S100, Image data preprocessing, including preprocessing the input data, wherein the preprocessing involves pairing and normalizing synthetic images and real-shot images; wherein the synthetic image is used as the input source and the real image is used as the target output; the preprocessed images are divided into training set, validation set and test set to ensure that there is sufficient data to support the generalization ability of the model during training.

[0055] Specifically, including,

[0056] S110. Input data: Use the SHIRT dataset, such as... Figure 2 As shown, the dataset includes images taken by real spacecraft and synthetic spacecraft images; the images in the dataset contain spacecraft models in different attitudes, and the images have obvious differences in texture and lighting.

[0057] S120. Image calibration and pairing: To ensure effective pairing between the input image and the target image, each pair of synthetic images and real images are aligned through calibration to ensure their consistency in spatial position and pose.

[0058] S130, Data Standardization: The image is normalized to ensure that the pixel value of the input data is within the range of [0,1]. At the same time, the size is adjusted so that the image size is uniformly 512×512 pixels. Then, the images are stitched together to form 512×1024. The simulated image is input to the ConvLSTM module, and the real captured image is input to the discriminator in the generative adversarial network.

[0059] The S200 and ConvLSTM (Convolutional Long Short-Term Memory) module is designed to effectively process temporal information in image sequences. This module uses LSTM units to process long-term and short-term dependencies in the image sequence, preserving the spatial structural features (such as texture and geometry). Each ConvLSTM unit processes the input image sequence through time steps and generates a feature representation that comprehensively considers temporal information by passing hidden state and memory state information. The final hidden state is then passed as input to the generator, thereby improving the quality of image generation and detail recovery capabilities.

[0060] Specifically, including,

[0061] S210 and ConvLSTM (Convolutional Long Short-Term Memory) modules, such as Figure 3 As shown, it is used to learn the temporal information in image sequences, especially the spatial and temporal dependencies between multiple frames. Through this temporal information, the ConvLSTM module can effectively integrate local features in the image sequence, solving the problems of edge blurring, detail loss, and overfitting encountered by traditional GAN ​​models when generating images. In this invention, the ConvLSTM module is used as the input preprocessing layer of the generator. The input image sequence will be processed by the ConvLSTM module to extract its temporal features. After processing, the features output by the ConvLSTM module will be passed to the generator to generate more realistic spacecraft images.

[0062] S220, ConvLSTM Unit Design: ConvLSTM units generate long-term and short-term memories by capturing the dependencies between each frame of images; the hidden state ℎ𝑡 and memory state 𝑐𝑡 of each ConvLSTM unit are updated over time steps and calculated using the following formula:

[0063]

[0064] in, The input image at the current moment; This represents the hidden state (i.e., the output feature) at the current moment. It represents the current memory state, used to store key information from the past;

[0065] The ConvLSTM unit operates by inputting the image at each time step. The hidden state of the previous moment and memory state Process them together to calculate the new hidden state. and memory state This method can preserve temporal information, ensuring the continuity and consistency of image sequences, especially the edges and details when generating spacecraft images;

[0066] S230. In this embodiment, the dimension of the input image sequence is set to (𝐵, 5, 3, 𝐻, 𝑊), where 𝐵 is the batch size, 5 is the number of time steps (i.e., the input image sequence contains 5 frames of images), 3 is the number of RGB channels in each image (i.e., each image contains three color channels: red, green, and blue), and 𝐻 and 𝑊 are the height and width of each image, respectively, representing the spatial size of the image. Specifically, in this invention, the size of each frame of image is 512 × 1024 pixels, so the input dimension of the image sequence is 𝐵 × 5 × 3 × 512 × 1024, where 5 represents the 5 time steps in the sequence. The image at each time step will be processed by the ConvLSTM module to capture the spatiotemporal features in the image sequence and provide key temporal information for subsequent image generation tasks.

[0067] The output of the S240 and ConvLSTM modules is the hidden state of the last time step. It contains all the temporal features of the input image sequence; these features reflect the spatial and temporal dependencies between each frame in the image sequence, helping the generator to produce more coherent and detailed images; the generator will receive this hidden state. As input, this temporal feature is used to generate the next frame image (i.e., frame T+1):

[0068]

[0069] in, This is a generator model that accepts the output of a ConvLSTM module as input. The generated image represents the next frame image generated based on temporal features;

[0070] By leveraging the temporal features of the ConvLSTM module, the generator can better preserve the details, edges, and temporal consistency of the image, thereby generating spacecraft images that are highly similar to real images. In particular, the generator can maintain high coherence in details such as attitude and lighting changes in the image, ensuring a smooth and natural transition between the generated image and the previous frame.

[0071] S300, Generative Adversarial Network (GAN) design, including generator and discriminator, specifically includes,

[0072] S310, Generator: Employs a supervised pairwise generation framework, such as... Figure 4As shown, the generator adopts a U-Net structure, including an encoder and a decoder, which can efficiently extract features from the input image and reconstruct them. The encoder gradually downsamples the image through multiple convolutional layers to capture feature information at different levels. The decoder gradually restores the spatial resolution of the image through deconvolution operations while preserving the detailed features of the input image. To prevent information loss, the U-Net design uses skip connections to pass low-level features to the decoder stage, ensuring that the details and structure of the generated image are consistent.

[0073] The U-Net generator's loss function consists of adversarial loss, L1 loss, and L2 loss, designed to drive the generated image to approximate the real image; the loss function is... :

[0074]

[0075] in, To combat the loss, it is used to ensure that the generated image is realistic enough; L1 loss is used to maintain structural consistency between the generated image and the real image; L2 loss is used to improve the smoothness and detail consistency of the image; and These represent the weighting coefficients for the L1 and L2 loss terms, respectively, used to control the proportion of their respective losses in the total generator loss; Norm: (The sum of the absolute values ​​of all pixels and channels); Norm square: (Sum of the squares of all pixels and channels);

[0076] The generator maximizes the following adversarial loss :

[0077]

[0078] Where 𝐷(𝐺(𝐴)) represents the discriminator's judgment on the authenticity of the generated image, 𝐺(𝐴) is the generated image, 𝐷 is the discriminator; E represents the expectation;

[0079] S320, Discriminator: Combined with Figure 6 As shown, the discriminator uses the PatchGAN structure to divide the input image into multiple small regions (judging their authenticity block by block); this method can accurately capture local textures and details in the image, improve the discriminator's ability to judge real images, and thus guide the generator to optimize the output;

[0080] This implementation scheme divides the input image into 70×70 blocks and judges the authenticity of the image block by block; the discriminator's loss function for:

[0081]

[0082] in, , ;

[0083] The discriminator continuously optimizes its ability to distinguish between real and fake images by minimizing the loss function;

[0084] S400, model training, specifically including...

[0085] In this implementation scheme, the model training uses the SHIRT dataset, which contains real and synthetic spacecraft images, covering spacecraft models under different attitudes and lighting conditions. To ensure the effectiveness of training, this implementation scheme divides the dataset into 3200 training images, 800 validation images, and 600 test images. All input images are 512×1024 pixels in size, and the images are standardized during the data preprocessing stage to scale the pixel values ​​to the range of [0,1] to improve the learning efficiency of the model.

[0086] To ensure the stability and optimization effect of the training process, the initial learning rate of both the generator and the discriminator is set to 0.0001. During training, the learning rate will gradually decay according to the training progress. After 600 epochs, the learning rate will linearly decay to 0. This learning rate decay strategy helps to effectively avoid overfitting and improve the generalization ability of the model, thereby generating more realistic and consistent images.

[0087] During training, the generator optimizes the quality of generated images by minimizing the loss function, while the discriminator improves its ability to distinguish between generated and real images by minimizing its own loss function. Through adversarial training between the generator and the discriminator, high-quality and highly consistent image generation results are ultimately achieved.

[0088] S500, image generation and quality assessment, specifically including...

[0089] S510, Image Generation Method: Image generation is optimized through adversarial training, with the generator and discriminator updating alternately to produce high-quality spacecraft images. Specifically, the input image sequence is first processed by a ConvLSTM module, which effectively captures temporal dependencies in the image sequence and extracts dynamic features and spatial variation information. By encoding the temporal information of each frame, the hidden state generated by the ConvLSTM module contains rich temporal features, which provide crucial contextual information for subsequent image generation. After receiving the hidden state output by the ConvLSTM module, the generator generates the next frame image based on these temporal features. Figure 7 The image shown illustrates the process of generating an image using the method of the present invention.

[0090] S520, Image Quality Assessment Methods: A variety of image quality evaluation metrics are used, such as PSNR (Peak Signal-to-Noise Ratio), SSIM (Structural Similarity Index), and FID (Fréchet Inception Distance). These metrics quantitatively analyze the quality of the generated image from different aspects, including the image's brightness, contrast, structure, and texture details.

[0091] ①PSNR (Peak Signal-to-Noise Ratio) Evaluation

[0092] PSNR (Peak Signal-to-Noise Ratio) is a standard metric for measuring the quality difference between a generated image and a real image; the higher the value, the smaller the difference between the generated image and the real image, and the better the image quality. The formula for calculating PSNR is as follows:

[0093]

[0094] Among them, 𝑀𝐴𝑋 𝐼 The maximum possible pixel value in the image is 255 (for an 8-bit image), and 𝑀𝑆𝐸 is the mean square error, defined as:

[0095]

[0096] Among them, 𝐼 𝑖 and 𝐺 𝑖 These are the actual pixel values ​​in the image and the pixel values ​​in the generated image, respectively, where 𝑁 is the total number of pixels in the image;

[0097] A higher PSNR means better image quality and a smaller difference between the generated image and the real image.

[0098] ②SSIM (Structural Similarity Index) Assessment

[0099] SSIM (Structural Similarity Index) measures the structural similarity between a generated image and a real image. Its value ranges from -1 to 1, with values ​​closer to 1 indicating higher structural similarity. The formula for calculating SSIM is as follows:

[0100]

[0101] Among them, 𝜇 𝑥 and 𝜇 𝑦 , are the means of images and , respectively. and For its variance, Let their covariance be... and It is a constant, used to avoid the case where the division by zero;

[0102] This formula can be used to quantify the similarity of images in terms of brightness, contrast, and structure.

[0103] ③ FID score assessment

[0104] FID (Fréchet Inception Distance) measures the distribution distance between a generated image and a real image. A lower FID value indicates a more similar distribution between the generated and real images. The formula for calculating FID is:

[0105]

[0106] Among them, 𝜇 𝑟 and 𝜇 𝑔 These represent the feature mean values ​​of the real image and the generated image in the Inception network, respectively. and It is its covariance matrix; Tr is the Trace (trace of a matrix), which represents the sum of the elements on the main diagonal of a square matrix;

[0107] This metric reflects the similarity between generated and real images in a high-dimensional feature space by evaluating the distance between the mean and covariance of the feature space.

[0108] By comprehensively evaluating the above evaluation indicators, the quality, structural similarity, and distribution similarity of the generated images can be fully measured, thereby effectively verifying the performance of the generation model in generating high-quality images.

[0109] S530. Attitude Estimation Verification and Evaluation: To verify the effectiveness of the images generated in this invention for attitude estimation tasks, this invention employs a two-stage attitude estimation method based on a combination of YOLO and a feature pyramid network. This method extracts key features from the generated images through multi-scale feature fusion and combines geometric computation techniques to calculate the spacecraft's three-dimensional attitude. The specific steps are as follows:

[0110] S531. Feature Extraction: In this step, the YOLO target detection network is first used to detect spacecraft targets in the generated image. The YOLO network can accurately identify spacecraft targets in the image and provide necessary target information for subsequent attitude estimation. At the same time, multi-scale feature fusion is performed by combining the Feature Pyramid Network (FPN). The FPN can extract spatial and temporal information from feature maps of different scales, effectively enhancing the network's ability to recognize complex spacecraft structures, especially when processing detailed parts of spacecraft (such as solar panels, antennas, etc.), it can maintain good resolution and accuracy.

[0111] S532, Pose Estimation: The RANSAC (Random Sample Consensus) and PnP (Perspective-n-Point) algorithms, combined with extracted feature points, are used to calculate the 3D-2D correspondence in the image. Specifically, key feature points in the spacecraft image are first extracted using YOLO and FPN networks. Then, the RANSAC algorithm is used to filter these feature points, eliminating possible incorrect matches. Next, the PnP algorithm is used to calculate the spacecraft's 3D pose based on the selected feature points. The PnP algorithm calculates the spacecraft's 3D position and attitude by mapping 2D image feature points to known 3D models and utilizing camera intrinsic and extrinsic parameters and geometric constraints.

[0112] S533. Error Performance Evaluation: To quantify the accuracy of attitude estimation, Euler angles are used to represent the spacecraft's attitude, and the attitude error between the generated image and the real image is calculated. Specifically, the attitude error calculation is based on the transformation relationship between the rotation matrix and quaternions, as shown in the following formula.

[0113]

[0114] in, Indicates attitude error. The quaternion representation of the true pose. The quaternion representation obtained through estimation methods;

[0115] This formula calculates the difference in rotation angles represented by quaternions, which represents the rotation error between the actual attitude and the estimated attitude. Quaternions are an efficient method for attitude representation, which can avoid the gimbal lock problem caused by Euler angles, and have high accuracy and stability in rotation calculations in three-dimensional space.

[0116] S534. Evaluate attitude estimation, such as Figure 8 The diagram shows the pose estimation results (from left to right) obtained by comparing ground truth pose values, training with real images, training with generated images, and training with software-synthesized images, and performs error statistics and analysis. Based on the pose estimation errors, the performance of different datasets in the pose estimation task is evaluated. Specifically, the error range includes Euler angle error and quaternion error, and the accuracy is calculated at each error threshold (e.g., less than 5 degrees and less than 10 degrees). In this way, the advantages of generated images in terms of pose consistency and accuracy are further quantified.

[0117] Specific implementation scheme two: The present invention provides a spacecraft image domain gap generation system based on supervised pairing GAN. The system has program modules corresponding to the above steps and executes the above steps applied to the improved heuristic static path planning method for mobile robots when running.

[0118] The other combinations and connections in this implementation scheme are the same as in Specific Implementation Scheme 1.

[0119] Specific Implementation Scheme 3: The present invention provides a computer-readable storage medium storing a computer program configured to, when called by a processor, implement the steps of a spacecraft image domain gap generation method based on supervised pairing GAN.

[0120] The other combinations and connections in this implementation scheme are the same as in Specific Implementation Scheme 1.

[0121] While the present invention has been disclosed above, its scope of protection is not limited thereto. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention, and all such changes and modifications will fall within the scope of protection of the present invention.

Claims

1. A method for generating spacecraft image domain gaps based on supervised paired GAN, characterized in that, Includes the following steps: S100, Image data preprocessing, including preprocessing the input data, wherein the preprocessing involves pairing and normalizing the synthetic image and the real-shot image; wherein the synthetic image is the input source and the real image is the target output. The S200 and ConvLSTM convolutional long short-term memory network module is designed to process the long-term and short-term dependencies of image sequences through LSTM units, preserving the spatial structural features of the images. Each ConvLSTM unit in the ConvLSTM convolutional long short-term memory network module processes the input image sequence through time steps and generates a feature representation that comprehensively considers temporal information by passing hidden state and memory state information. The final hidden state is then passed as input to the generator of the generative adversarial network. S300 and GAN generative adversarial network design, including generator and discriminator. The generator adopts a U-Net structure, which captures feature information at different levels and preserves the detailed features of the input image through the forward and deconvolution operations of the encoder and decoder. The discriminator uses a PatchGAN structure to divide the input image into multiple small regions and minimizes the loss function to improve the discriminator's ability to judge real images, thereby guiding the generator to optimize the output. S400, Model Training: The synthesized image processed in step S200 is input into the GAN generative adversarial network in step S300. The generator processes the synthesized image into an image close to the real image, and the discriminator distinguishes the real image from the generator image. Then, through adversarial training between the generator and the discriminator, a high-quality and highly consistent image generation effect is finally achieved.

2. The method for generating spacecraft image domain gaps based on supervised paired GAN according to claim 1, characterized in that: In step S100, the following are included: S110, Input data, including real images and synthetic images; S120, Image calibration and pairing: Alignment between each pair of real and synthetic images through calibration; S130. Data standardization: Normalize the image to ensure that the pixel values ​​of the input data are within the range of [0,1]. At the same time, adjust the size to make the image size uniform. Then, stitch the images together. The synthesized image is input to the ConvLSTM module, and the real image is input to the discriminator in the GAN generative adversarial network.

3. The method for generating spacecraft image domain gaps based on supervised paired GAN according to claim 2, characterized in that: Step S200 includes, The input image sequences obtained in steps S210 and S100 are processed by the ConvLSTM convolutional long short-term memory network module to extract temporal features. After processing, the output features are passed to the generator in the GAN (Generative Adversarial Network). In the S220 ConvLSTM convolutional long short-term memory network module, the ConvLSTM units generate long-term and short-term memories by capturing the dependencies between each frame of the image; the hidden state ℎ𝑡 and the memory state 𝑐𝑡 of each ConvLSTM unit are updated over time steps and are calculated using the following formula: in, The input image at the current moment; The current hidden state; The current memory state; The ConvLSTM unit operates by inputting the image at each time step. The hidden state of the previous moment and memory state Process them together to calculate the new hidden state. and memory state ; S230. The dimensions of the input image sequence are set to (𝐵,5,3,𝐻,𝑊), where 𝐵 is the batch size, 5 is the number of time steps, that is, the input image sequence contains 5 frames of images, 3 is the number of RGB channels in each image, indicating that each image contains three color channels: red, green and blue, and 𝐻 and𝑊 are the height and width of each image, respectively, representing the spatial size of the image; The output of the S240 and ConvLSTM convolutional long short-term memory network modules is the hidden state at the last time step. The hidden state This contains all temporal features of the input image sequence; the generator will receive this hidden state. As input, this temporal feature is used to generate the next frame image: in, This is a generator model that accepts the output of a ConvLSTM module as input. The generated image represents the next frame image generated based on temporal features.

4. The method for generating spacecraft image domain gaps based on supervised paired GAN according to claim 3, characterized in that: In step S300, the generator's loss function for: in, To combat the losses; For L1 loss; For L2 loss; and These represent the weighting coefficients for the L1 and L2 loss terms, respectively, used to control the proportion of their respective losses in the total generator loss; Represents the L1 norm; Represents the L2 norm; The generator maximizes the following adversarial loss : Where 𝐷(𝐺(𝐴)) represents the discriminator's judgment on the authenticity of the generated image, 𝐺(𝐴) is the generated image, 𝐷 is the discriminator; E represents the expectation.

5. The method for generating spacecraft image domain gaps based on supervised paired GAN according to claim 4, characterized in that: In step S300, the loss function of the discriminator for: in, , .

6. A spacecraft image domain gap generation system based on supervised paired GAN, characterized in that: The system has a program module corresponding to the steps of any one of the claims 1-5 above, and executes the steps described above in the improved heuristic static path planning method for mobile robots when it is run.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program configured to, when invoked by a processor, implement the steps of any one of claims 1-5, a method for generating spacecraft image domain gaps based on supervised paired GANs.

Images