Image translation method and device based on decoupling guide bridging matching, electronic equipment and storage medium
By decoupling the combination of modal representation models and modal sharing to modality-specific networks, image translation is explicitly guided, solving the problem of incomplete feature decoupling in cross-modal image translation and achieving high-fidelity image generation.
Patent Information
- Application Number
- CN202511682544.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-17
- Publication Date
- 2026-02-10
Smart Images
Figure CN121505296A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The application belongs to the technical field of computer vision, and particularly relates to an image translation method and device based on decoupled guidance bridging matching, an electronic device and a storage medium. BACKGROUND
[0002] In the field of remote sensing image analysis, synthetic aperture radar (SAR) to optical (OPT) image translation technology aims to automatically convert SAR images, which are not affected by weather and illumination, into optical OPT images that are easier for humans to interpret and rich in color. This technology plays a key role in tasks such as all-weather ground monitoring, disaster assessment, data completion and fusion. Traditional processes usually rely on methods based on physical models or signal processing, such as removing the inherent speckle noise of SAR images through filtering algorithms, and then combining ground object scattering models and texture mapping rules to generate optical appearances. However, due to the significant differences in imaging mechanisms and information content between SAR and optical OPT images, traditional methods struggle to capture the complex nonlinear mapping relationship between the two, resulting in generated images that lack realism and are plagued by artifacts. Although there have been numerous studies proposing deep learning methods for cross-modal image translation, mainstream techniques such as generative adversarial networks (GANs) and their variants (such as Pix2pix, CycleGAN) have improved the realism of generated images to some extent. However, due to the inherent nature of adversarial training, these methods often face issues such as unstable training, mode collapse, and difficulty in controlling generated details. In particular, when dealing with SAR images, which are characterized by high noise, strong texture, and geometric distortion, the robustness of GANs may not be sufficient, leading to geometric distortion or loss of texture details in the generated images.
[0003] In recent years, methods based on diffusion models have received widespread attention due to their ability to generate high-quality, diverse images and more stable training processes, and have achieved significant performance improvements in image translation tasks. Existing methods typically use conditional diffusion models, taking the source SAR image as a condition input to guide the model to gradually denoise from noise and generate the target optical OPT image. Although these models have shown good performance, they essentially treat the source image as a whole, coupled condition, ignoring the inherent relationships and differences between different modal features. To address this issue, various techniques have been developed to enhance feature guidance during the conditional diffusion process, including multi-scale feature fusion and cross-attention mechanisms. These designs have improved the effectiveness of individual models, but they have failed to fundamentally address a core problem: how to effectively decouple and explicitly guide the robust geometric structure information in SAR images and the unique texture appearance information in optical OPT images. Therefore, the generated results still fall far short of satisfactory high fidelity. Summary of the Invention
[0004] This invention provides an image translation method, apparatus, electronic device, and storage medium based on decoupling-guided bridging matching, aiming to solve the problems of incomplete feature decoupling, reliance on target modality data in the inference stage, and low fidelity of generated images in existing cross-modal image translation.
[0005] This invention is implemented as follows: an image translation method based on decoupled guided bridging matching, comprising the following steps: S1: Constructing a set of models for image translation, including:
[0006] S11: Training DMRM (Decoupled Modal Representation Model): Obtain paired source modal images and target modal images, extract mixed features through a shared encoder, and decompose the mixed features into modal shared features representing geometric structure and modal specific features representing texture appearance using a feature decoupler;
[0007] S12: Training MS2N (Modality Sharing to Modality Specific Network): Using the DMRM trained in S11, extract the real modality sharing features and real modality specific features of the target modality image, and train the MS2N to learn the mapping relationship from the real modality sharing features to the real modality specific features;
[0008] S13: Training RCBM (Representation Conditional Bridging Matching Model): Extract shared features of the target modality image using the DMRM trained in S11, input them into the MS2N trained in S12 to generate conditional features; input the source modality image and the target modality image into the encoder respectively, and generate latent codes for the source modality image and the target modality image through the encoder; train the drift term network to predict the drift term evolving towards the latent code of the target modality image based on the latent code of the source modality image, the conditional features, and the time step;
[0009] S2: Input Image: Input the source modality image to be translated into the model set obtained in S1;
[0010] S3: Image Processing: Processing the models in the model set obtained in S1, including the following steps:
[0011] S31: The trained DMRM extracts modality-shared features and modality-specific features from the source modality image;
[0012] S32: Input the modality-shared features into the trained MS2N to predict the conditional features for generating the target modality;
[0013] S33: The dual features extracted in S31 are fused to generate the latent code of the source modality image; using a stochastic differential equation solver and the trained RCBM, under the guidance of the conditional features, the latent code of the source modality image is iteratively evolved into the latent code of the target modality image, and then decoded into the target modality image.
[0014] Preferably, in S11, the adversarial training mechanism is used to optimize the DMRM, including: training a modality discriminator to distinguish the source modality of the modality-specific features, and training a modality obfuscator to make the modality-shared features modality-agnostic.
[0015] Preferably, in step S12, training is performed by minimizing the L2 loss between the modality-specific features predicted by the network and the actual modality-specific features.
[0016] Preferably, in step S13, a hybrid loss function is used for training, which includes a potential drift term loss and a pixel-level loss.
[0017] Preferably, the shared encoder in S11 is a neural network for extracting deep features across modalities. After the hybrid features are decomposed by the feature decoupler, the information of the modal shared features and modal-specific features is complete and without redundancy.
[0018] Preferably, the encoder used to generate latent codes in S13 is the same network as the shared encoder used to extract mixed features in S11, and the dual feature fusion method ensures that the latent coding dimensions of the source modality and the target modality match.
[0019] Preferably, the stochastic differential equation solver in S33 uses a numerical solution method to realize the iterative evolution of the latent code, the decoding process is implemented through a neural network, and the resolution of the target modal image is consistent with that of the source modal image.
[0020] Preferably, the stochastic differential equation solver in S33 uses a numerical solution method to realize the iterative evolution of the latent code, the decoding process is implemented through a neural network, and the resolution of the target modal image is consistent with that of the source modal image.
[0021] An image translation device based on decoupled guided bridging matching includes:
[0022] DMRM training unit: used to acquire paired source modal images and target modal images; extract deep blending features from the paired images respectively through a shared encoder; and decompose the blending features into modal shared features representing geometric structure and modal specific features representing texture appearance using a feature decoupler;
[0023] MS2N Training Unit: Used to call the trained DMRM output by the DMRM training unit, extract the real modality shared features and real modality specific features of the target modality image; train MS2N to learn the mapping relationship from the real modality shared features to the real modality specific features;
[0024] RCBM training unit: used to call the DMRM output by the DMRM training unit to extract shared features of the target modality image; input the source modality image and the target modality image into the encoder respectively, and generate the latent code of the source modality image and the latent code of the target modality image through the encoder; train the drift term network so that it predicts the drift term evolving towards the latent code of the target modality image based on the latent code of the source modality image, conditional features and time steps;
[0025] Image input unit: used to receive the source modality image to be translated; input the source modality image to be translated into the model set composed of the DMRM training unit, MS2N training unit, and RCBM training unit;
[0026] Feature extraction unit: used to call the trained DMRM output by the DMRM training unit to extract modality-shared features and modality-specific features from the source modality image to be translated;
[0027] Conditional feature prediction unit: used to call the trained MS2N output by the MS2N training unit, input the modality shared features output by the feature extraction unit, and generate conditional features of the target modality;
[0028] Iterative generation unit: used to generate the latent code of the source modality image through the encoder; calling the trained RCBM output by the RCBM training unit, combined with the stochastic differential equation solver, and guided by the conditional features output by the conditional feature prediction unit, to iteratively evolve the latent code of the source modality image into the latent code of the target modality image; and converting the latent code of the target modality image into the target modality image through decoding and outputting it.
[0029] An electronic device includes a memory and a processor, wherein the memory and the processor are coupled.
[0030] The memory is used to store one or more computer instructions, which contain the execution logic of the above-described method; the processor is used to execute the one or more computer instructions to implement the above-described image translation method based on decoupled guided bridging matching.
[0031] A computer-readable storage medium storing one or more computer instructions, which, when executed by a processor, implement the above-described image translation method based on decoupled guided bridging matching.
[0032] Compared with existing technologies, the beneficial effects of this invention are as follows: The image translation method based on decoupled guided bridging matching, in the first stage, employs a decoupled modality representation model (DMRM) to decompose complex cross-modal image features into modality-shared features (geometric structure) and modality-specific features (texture appearance), thereby achieving preliminary purification of the latent image representation and laying a high-quality feature foundation for subsequent condition generation. In the second stage, a modality-to-modality-specific network (MS2N) is used to learn an efficient mapping from modality-shared features to their corresponding modality-specific features. This method provides a reliable approach for generating accurate modality-specific conditions in inference scenarios where only source modality images are available. In the third stage, the framework utilizes representational conditional bridging matching (RCBM), using the modality-specific features generated by MS2N as explicit guidance, and iteratively generates conditions by solving stochastic differential equations (SDEs). This method effectively distinguishes between the structural content and stylistic appearance of an image, without relying on target modality data in the inference stage, significantly improving the realism and detail clarity of the translated image, and is suitable for cross-modal image translation scenarios such as remote sensing and medical applications. Attached Figure Description
[0033] Figure 1 This is a flowchart of the method of the present invention;
[0034] Figure 2 This is a flowchart of the DMRM training process in this invention;
[0035] Figure 3 This is a flowchart of training RCBM in this invention;
[0036] Figure 4 This is a schematic diagram of the architecture of the electronic device in this invention. Detailed Implementation
[0037] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.
[0038] Please see Figure 1 This invention provides a technical solution: an image translation method based on decoupled guided bridging matching, comprising the following steps:
[0039] S1: Construct a set of models for image translation, including:
[0040] S11: Training the DMRM (Decoupled Modal Representation Model): Acquire paired source modal images and target modal images, wherein the source modal images are synthetic aperture radar (SAR) images and the target modal images are optical (OPT) images; extract deep hybrid features from the paired images using a shared encoder, and decompose the hybrid features into modality-shared features representing geometric structure and modality-specific features representing texture appearance using a feature decoupler; optimize the model using an adversarial training mechanism, including training a modality discriminator to distinguish the source modality of the modality-specific features, and training a modality obfuscator (connected via a gradient inversion layer) to make the modality-shared features modality-agnostic.
[0041] The goal of this step is to explicitly decouple the mode-shared features F from the paired SAR image x0 (source modality image) and optical OPT image x1 (target modality image). sh and modality-specific features F sp Specifically, it includes the following steps:
[0042] Input: Paired SAR image x0 and optical OPT image x1;
[0043] Feature extraction: using a shared feature encoder (E φ Extracting deep blended features (F) from images of two modalities mix );
[0044] Feature decoupling: using a feature decoupler (D Φ ), through two parallel projectors, F mix Decomposed into modal shared features F sh and modality-specific features F sp ;
[0045] Discriminative Loss (LDisc): Training a modality discriminator (D r This enables it to accurately identify modal-specific features F. sp The original mode (SAR or OPT). This forces the mode-specific feature F sp Preserve the unique distinguishing information of each modality.
[0046] Confusion Loss (LConf): Training a modality confused (D) u This is achieved through a gradient inversion layer (GRL), which connects the modalities. This forces the modalities to share features F. sh Becoming modally unknowable, i.e., confusing D u This makes it unable to recognize the modal sharing feature F. sh The original mode.
[0047] In step S11, to achieve pure decoupling, an adversarial training mechanism is introduced. The adversarial training mechanism includes: ① training a modality discriminator, which takes modality-specific features as input and outputs the probability of the feature source modality (SAR / OPT), and maximizes the discriminator's discrimination accuracy through cross-entropy loss; ② training a modality obfuscator, which is connected to modality-shared features through a gradient inversion layer (GRL), takes modality-shared features as input, outputs the probability of the feature source modality, and minimizes the obfuscator's discrimination accuracy through cross-entropy loss, making the modality-shared features modality-agnostic.
[0048] S12: Training MS2N (Modality Sharing to Modality Specific Network): Utilize the DMRM trained in S11 to extract the true modality shared features and true modality specific features of the target modality image, and train the MS2N to learn the mapping relationship from the true modality shared features to the true modality specific features; freeze the parameters of the trained decoupled modality representation model, input the target modality image into the frozen model, and obtain the true modality shared features and true modality specific features; train the modality sharing to modality specific network to learn the mapping relationship from the true modality shared features to the true modality specific features. The training process is achieved by minimizing the L2 loss between the modality specific features predicted by the network and the true modality specific features.
[0049] Since only SAR images are available during inference, the corresponding optical modality-specific features cannot be directly obtained. The goal of this step is to train a lightweight network (MS2N) that can accurately predict the corresponding modality-specific features based solely on modality-shared features. The specific steps are as follows:
[0050] Input: Use a trained, frozen DMRM model;
[0051] Data preparation: Input a batch of optical OPT images x1 into the frozen DMRM to obtain their corresponding true modality-shared features (F). sh,True ) and true modality-specific features (F sp,True );
[0052] Mapping Training: Training the MS2N network to receive F sh,True As input, and predict
[0053] Loss function: The training objective is to minimize the predicted value. Compared with the true value F sp,True The L2 distance (L2Loss) between them enables them to imagine the appearance of corresponding optical textures based on geometric structures.
[0054] S13: Training the RCBM (Representation Conditional Bridging Matching Model): Freeze the trained decoupled modality representation model and modality sharing to modality-specific network parameters, input the target modality image into the frozen model combination, obtain the target modality shared features and input them into MS2N to generate conditional features (i.e., target modality-specific features) for guiding modality translation; input the source modality image and the target modality image into the pre-trained VAE encoder, and generate the encoding of the source modality and target modality latent space through the VAE encoder; train the drift term network so that it accurately predicts the real drift term evolving towards the latent encoding of the target modality image with the intermediate latent state, time step variable and the conditional features as input (the real drift term calculation formula is: (Latent encoding of target modality image - intermediate latent state) / (1 - time step variable)); the training process adopts a hybrid loss function, which includes latent drift term loss (calculating the difference between the predicted drift term and the real drift term) and pixel-level loss (decoding the latent encoding corresponding to the predicted drift term into an image, and calculating the LPIPS loss of the decoded image and the real target modality image).
[0055] The goal of this step is to train a powerful generative model (RCBM) that uses features generated by MS2N as explicit conditional guidance to progressively evolve source SAR images into high-fidelity optical OPT images.
[0056] Input: Paired SAR image x0 and optical OPT image x1, and the frozen DMRM and MS2N models trained in steps one and two.
[0057] Conditional generation: Input the optical OPT image x1 into the frozen DMRM and MS2N to generate the corresponding modality-specific features, and use them as condition C.
[0058] Bridge matching training:
[0059] The source modality SAR image x0 and the target modality OPT image x1 are input into a pre-trained RS Encoder (i.e., VAE encoder). This encoder maps the high-dimensional image data to a low-dimensional latent space, obtaining the corresponding latent codes z0 and z1.
[0060] Based on the random interpolation formula, combined with the latent codes z0 and z1 and a random sampling time step t, a noisy intermediate state z is generated. t This state can be viewed as a random point on the evolution path from the source mode to the target mode.
[0061] Combine the conditional feature C generated in step one with the noisy intermediate state z generated in step two. t Along with the time step t, it is input into the core drift term network v. θ (zt In the network (t, C), the core task is to predict the value of the current state z. t The drift term (i.e., evolution direction and velocity) required to reach the final target z1.
[0062] Loss function: A hybrid loss function L is used. RCBM =L LBM +λL pixel ;
[0063] In addition to the potential drift term loss mentioned above, a pixel-level loss (L) is also introduced. pixel ). Pixel-level loss is achieved by ν θ Predicted The input is fed into the RSDecoder (i.e., the VAE decoder) to generate the predicted target image. And calculate Differences between the generated image and the real x1 (such as LPIPS loss) are used to enhance the visual details of the generated image.
[0064] S2: Input Image: Input the source modality image to be translated into the model set obtained in S1;
[0065] S3: Image Processing: Processing the models in the model set obtained in S1, including the following steps:
[0066] S31: The trained DMRM extracts modality-shared features and modality-specific features from the source modality image;
[0067] S32: Input the modality-shared features into the trained MS2N to predict the conditional features for generating the target modality;
[0068] S33: The latent encoding of the source modality image is generated by the VAE encoder; using the stochastic differential equation solver and the trained RCBM, under the guidance of the conditional features, the latent encoding of the source modality image is iteratively evolved into the latent encoding of the target modality image, and then decoded into the target modality image.
[0069] In S12, the training process of MS2N is achieved by minimizing the loss function, which is L2 loss, that is, the squared Euclidean distance between the modality-specific features predicted by the network and the true modality-specific features.
[0070] In S13, a hybrid loss function is used for training. This hybrid loss function combines the potential drift term loss L through a weighted summation. LBM and pixel-level loss L pixel This process involves two parallel computational branches. Among them, L... LBM Part of the loss is calculated directly in the latent space. First, based on the true z1 and z...t Calculate the true drift term v true =(z1-z t Then, calculate the drift term v predicted by the network. θ With the real drift term v true The difference between them (e.g., mean squared error). L pixel Part of the loss is calculated in the image space to ensure the visual quality of the generated image. First, the drift term predicted by the network is used, through the formula... Obtain the predicted target latent code Next, The input is fed into the RSDecoder (i.e., the VAE decoder) to generate the predicted target image. Finally, the generated predicted image is calculated. The difference between the actual OPT image x1 and the real OPT image x1 (e.g., LPIPS loss). Finally, the two losses are weighted and summed to obtain the hybrid loss function L. RCBM =L LBM +λL pixel λ is a preset weight hyperparameter used to balance the contributions of the two losses. The gradient of this loss function is used to update the drift term network v. θ The parameters.
[0071] The shared encoder in S11 is a backbone network based on the Transformer architecture and pre-trained on a large-scale remote sensing dataset. This encoder segments the input image into a sequence of image patches and then efficiently captures long-range spatial context information through its core multi-head self-attention mechanism, overcoming the limitations of traditional convolutional networks in terms of receptive field. Finally, it outputs a deep hybrid feature map F rich in global information. mix By employing a pre-trained Transformer architecture, the shared encoder in this invention can more effectively handle complex scenes and multi-scale targets in remote sensing images, providing higher-quality feature representations with more comprehensive contextual information for subsequent feature decoupling steps.
[0072] The encoder used to generate the latent code in S13 is the encoder part of a separate, pre-trained variational autoencoder (VAE) with its parameters fixed during the RCBM training phase. This VAE is pre-trained on a large-scale, general-purpose image dataset.
[0073] The stochastic differential equation (SDE) solver in S33 iteratively evolves the latent code of the source modality image into the latent code of the target modality through a discretized numerical solution method during the inference phase. This process specifically includes:
[0074] 1. Time step discretization: Discretizing a continuous time interval into N preset, fixed time step sequences {t0, t1, ..., t2}. N}, where t0 = 0 and t N =1. To achieve efficient inference, this time step sequence is consistent with the time step sampling strategy used in the S13 training phase. For example, iteration can be performed using four equally spaced time steps.
[0075] 2. Iterative Update: Starting from the initial state z(t0) = z0, iterative updates are performed using the standard SDE numerical solver. In the i-th step, starting from t... i to t i+1 The update rules are as follows: z(t) i+1 )=z(t i )+v θ (z(t i ),t i ,C)·(t i+1 -t i Where, z(t) i ) is the potential encoding of the current step, v θ (z(t i ),t i C) is the drift term predicted by the trained drift term network based on the current state, the current time step, and the conditional feature C.
[0076] 3. Final Output: After N iterations, the final latent code z(t) is obtained. N This encoding is the latent encoding of the evolved target modality image. The decoding process is the process of converting the final latent code into a single, fully encoded value. Convert to target modal image The process is implemented through a VAE decoder paired with the encoder in S13, ensuring that the latent code can be reconstructed into a target modal image with a resolution consistent with the source modal image, and with high quality.
[0077] An image translation device based on decoupled guided bridging matching includes:
[0078] DMRM training unit: used to acquire paired source modal images and target modal images; extract deep blending features from the paired images respectively through a shared encoder; and decompose the blending features into modal shared features representing geometric structure and modal specific features representing texture appearance using a feature decoupler;
[0079] MS2N Training Unit: Used to call the trained DMRM output by the DMRM training unit, extract the real modality shared features and real modality specific features of the target modality image; train MS2N to learn the mapping relationship from the real modality shared features to the real modality specific features;
[0080] RCBM training unit: used to call the DMRM output by the DMRM training unit to extract shared features of the target modality image; call the MS2N output by the MS2N training unit to input the shared features to generate conditional features; input the source modality image and the target modality image into the encoder respectively, and generate the latent codes of the source modality image and the target modality image through the encoder; train the drift term network so that it predicts the drift term evolving towards the latent code of the target modality image based on the latent code of the source modality image, the conditional features and the time step;
[0081] Image input unit: used to receive the source modality image to be translated; input the source modality image to be translated into the model set composed of the DMRM training unit, MS2N training unit, and RCBM training unit;
[0082] Feature extraction unit: used to call the trained DMRM output by the DMRM training unit to extract modality-shared features and modality-specific features from the source modality image to be translated;
[0083] Conditional feature prediction unit: used to call the trained MS2N output by the MS2N training unit, input the modality shared features output by the feature extraction unit, and generate conditional features of the target modality;
[0084] Iterative generation unit: used to generate the latent code of the source modality image through the encoder; calling the trained RCBM output by the RCBM training unit, combined with the stochastic differential equation solver, and guided by the conditional features output by the conditional feature prediction unit, to iteratively evolve the latent code of the source modality image into the latent code of the target modality image; and converting the latent code of the target modality image into the target modality image through decoding and outputting it.
[0085] An electronic device includes a memory and a processor, wherein the memory and the processor are coupled.
[0086] The memory is used to store one or more computer instructions, which contain the execution logic of the above-described method; the processor is used to execute the one or more computer instructions to implement the above-described image translation method based on decoupled guided bridging matching.
[0087] The electronic device may also include at least one network interface. The various components of the electronic device are coupled together via a bus system, which is understood to be used to enable communication between these components. In addition to a data bus, the bus system also includes a power bus, a control bus, and a status signal bus. The network interface is used for wired or wireless communication between the electronic device and other devices.
[0088] A computer-readable storage medium having stored thereon one or more computer instructions, which, when executed by a processor, implement the above-described image translation method based on decoupled guided bridging matching.
[0089] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. An image translation method based on decoupled guided bridging matching, characterized in that: Including the following step: S1: Construct a set of models for image translation, including: S11: Training DMRM (Decoupled Modal Representation Model): Obtain paired source modal images and target modal images, extract mixed features through a shared encoder, and decompose the mixed features into modal shared features representing geometric structure and modal specific features representing texture appearance using a feature decoupler; S12: Training MS2N (Modality Sharing to Modality Specific Network): Using the DMRM trained in S11, extract the real modality sharing features and real modality specific features of the target modality image, and train the MS2N to learn the mapping relationship from the real modality sharing features to the real modality specific features; S13: Training RCBM (Representation Conditional Bridging Matching Model): Extract shared features of the target modality image using the DMRM trained in S11, input them into the MS2N trained in S12 to generate conditional features; input the source modality image and the target modality image into the encoder respectively, and generate latent codes for the source modality image and the target modality image through the encoder; train the drift term network to predict the drift term evolving towards the latent code of the target modality image based on the latent code of the source modality image, the conditional features, and the time step; S2: Input Image: Input the source modality image to be translated into the model set obtained in S1; S3: Image Processing: Processing the models in the model set obtained in S1, including the following steps: S31: Feature extraction: The trained DMRM extracts modality-shared features and modality-specific features from the source modality image; S32: Conditional feature prediction: Input the modality shared features into the trained MS2N to predict the conditional features for generating the target modality; S33: Iterative generation: The latent code of the source modality image is generated by the encoder; under the guidance of the conditional features, the latent code of the source modality image is iteratively evolved into the latent code of the target modality image by using the stochastic differential equation solver and the trained RCBM, and then decoded into the target modality image.
2. The image translation method based on decoupled guided bridging matching as described in claim 1, characterized in that: In S11, an adversarial training mechanism is used to optimize the DMRM, including: training a modality discriminator to distinguish the source modality of the modality-specific features, and training a modality obfuscator to make the modality-shared features modality-agnostic.
3. The image translation method based on decoupled guided bridging matching as described in claim 1, characterized in that: In step S12, training is completed by minimizing the L2 loss between the modality-specific features predicted by the network and the actual modality-specific features.
4. The image translation method based on decoupled guided bridging matching as described in claim 1, characterized in that: In step S13, a hybrid loss function is used for training, which includes potential drift term loss and pixel-level loss.
5. The image translation method based on decoupled guided bridging matching as described in claim 1, characterized in that: The shared encoder in S11 is a neural network used to extract deep features across modalities. After the hybrid features are decomposed by the feature decoupler, the information of the modal shared features and modal-specific features is complete and without redundancy.
6. The image translation method based on decoupled guided bridging matching as described in claim 1, characterized in that: The encoder used to generate latent codes in S13 is the same network as the shared encoder used to extract mixed features in S11. The dual feature fusion method ensures that the latent coding dimensions of the source modality and the target modality match.
7. The image translation method based on decoupled guided bridging matching as described in claim 1, characterized in that: The stochastic differential equation solver in S33 uses a numerical solution method to realize the iterative evolution of the latent code. The decoding process is implemented through a neural network, and the resolution of the target modal image is consistent with that of the source modal image.
8. An image translation device based on decoupled guided bridging matching, characterized in that: include: DMRM training unit: used to acquire paired source modality images and target modality images; Deep blending features are extracted from the paired images using a shared encoder. The hybrid features are decomposed into modality-shared features representing geometric structure and modality-specific features representing texture appearance using a feature decoupler; MS2N training unit: used to call the trained DMRM output by the DMRM training unit to extract the real modality shared features and real modality specific features of the target modality image; The MS2N is trained to learn the mapping relationship from shared features of the real modality to specific features of the real modality; RCBM training unit: used to call the DMRM output by the DMRM training unit to extract shared features of the target modality image; Call the MS2N output of the MS2N training unit and input the shared features to generate conditional features; The source modal image and the target modal image are input into the encoder respectively, and the encoder generates the latent code of the source modal image and the latent code of the target modal image. Train the drift term network to predict the drift term evolution towards the latent code of the target modality image based on the latent code of the source modality image, conditional features, and time steps; Image input unit: used to receive the source modality image to be translated; input the source modality image to be translated into the model set composed of the DMRM training unit, MS2N training unit, and RCBM training unit; Feature extraction unit: used to call the trained DMRM output by the DMRM training unit to extract modality-shared features and modality-specific features from the source modality image to be translated; Conditional feature prediction unit: used to call the trained MS2N output by the MS2N training unit, input the modality shared features output by the feature extraction unit, and generate conditional features of the target modality; Iterative generation unit: used to generate the latent code of the source modality image through the encoder; The trained RCBM output by the RCBM training unit is invoked, and combined with the stochastic differential equation solver, under the guidance of the conditional features output by the conditional feature prediction unit, the latent code of the source modality image is iteratively evolved into the latent code of the target modality image; the latent code of the target modality image is converted into the target modality image through decoding and output.
9. An electronic device, characterized in that: Includes a memory and a processor, wherein the memory and the processor are coupled; The memory is used to store one or more computer instructions, the computer instructions containing the execution logic of the method according to any one of claims 1-7; the processor is used to execute the one or more computer instructions to implement the image translation method based on decoupled guided bridging matching according to any one of claims 1-7.
10. A computer-readable storage medium storing one or more computer instructions thereon, characterized in that: When the instruction is executed by the processor, it implements the image translation method based on decoupled guided bridging matching as described in any one of claims 1-7.