Method for generating few-shot target remote sensing image based on structure perception and detail enhancement
By introducing structure awareness and detail enhancement methods into the diffusion model, the problems of training instability and structure degradation in few-shot remote sensing image generation are solved, achieving stable generation and enhanced realism under few-shot conditions.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- 10TH RES INST OF CETC
- Filing Date
- 2026-05-06
- Publication Date
- 2026-06-02
AI Technical Summary
Under limited sample conditions, diffusion models are unstable during training in remote sensing image generation, structural information degrades, and it is difficult to balance perceptual quality and geometric consistency. Existing methods perform poorly or introduce new problems in limited sample scenarios.
We employ a structure-aware and detail-enhancing approach, selectively unfreezing model parameters and introducing a structure-aware edge supervision module and a structure-isolated image detail enhancement module. We also perform quality self-evaluation control during the inference phase to ensure that the model can stably generate remote sensing images under conditions of few samples.
It effectively alleviates the problems of boundary blurring and structural degradation, improves the geometric consistency and visual realism of the generated results, and enhances the quality and reliability of the generated images.
Smart Images
Figure CN122133737A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of computer vision and deep learning technology, specifically to a method for generating remote sensing images of few-shot targets based on structure awareness and detail enhancement. Background Technology
[0002] Remote sensing images, as a crucial data source for Earth observation, are widely used in urban planning, land use monitoring, disaster assessment, and infrastructure management. Compared to natural scene images, remote sensing images typically feature a fixed viewpoint, large scale variations, and strong regularity in target structures. Targets such as buildings, roads, and airport runways often exhibit distinct geometric shapes and boundary features. Therefore, in remote sensing image generation and reconstruction tasks, structural consistency and geometric accuracy are usually more critical than texture diversity.
[0003] In recent years, with the development of deep learning technology, generative models such as Generative Adversarial Networks (GANs), Variational Autoencoders (VAEs), and Diffusion Models have been widely applied in the field of image generation. Among them, the Diffusion Model, by progressively adding noise to the data and learning a reverse denoising process, has shown significant advantages in terms of generation quality and training stability, and has become the mainstream direction of current image generation research. In the field of natural images, methods based on the Diffusion Model have achieved good results in tasks such as high-resolution generation and multimodal conditional generation.
[0004] However, directly applying existing diffusion generation methods to remote sensing image scenarios, especially under conditions of limited samples, still faces several technical challenges. On one hand, diffusion models typically rely on large-scale, diverse data for training to ensure sufficient modeling capability for the data distribution. In practical remote sensing applications, due to high data acquisition costs, annotation difficulties, and the scarcity of samples in specific regions or for specific targets, it is often difficult to obtain a sufficient number of high-quality training samples, leading to overfitting or catastrophic forgetting during model training. On the other hand, structural information in remote sensing images has strong deterministic characteristics, while diffusion models, with noise regression as their core objective function, naturally favor low-frequency statistical properties, making them more prone to problems such as blurred boundaries and structural collapse when the number of samples is insufficient.
[0005] Current research suggests that while some studies have attempted to improve diffusion model performance by introducing conditional information, overlaying image details to enhance networks, or employing perceptual loss, these methods largely rely on assumptions about large-scale training data. In low-sample remote sensing scenarios, these methods often prove ineffective and may even introduce new structural degradation and training instability issues.
[0006] For remote sensing scenarios with few samples, the obvious shortcomings of the relevant methods are as follows: First, conditional guidance methods often require additional structural annotations or high-quality conditional inputs, which are difficult to obtain with limited samples. Introducing perceptual loss directly into the diffusion backbone network can easily dominate the model's optimization direction when the sample size is limited, leading to over-smoothing of building boundaries and degradation of structural information. If the image detail enhancement module is too tightly coupled with the diffusion backbone network, its gradient may interfere with the diffusion model's learning of structural features, making it more difficult for the model to converge stably with limited samples. If multi-branch training strategies do not uniformly constrain the diffusion time step and noise, inconsistent gradient directions are more likely to occur when the sample size is limited, thus reducing training stability. Summary of the Invention
[0007] The purpose of this invention is to address the problems of unstable training of diffusion models, degradation of structural information, and difficulty in simultaneously achieving perceptual quality and geometric consistency in the generation of remote sensing images of few-shot targets. Therefore, a method for generating remote sensing images of few-shot targets based on structure perception and detail enhancement is proposed. This invention achieves the advantages of stable and controllable generation of remote sensing images under few-shot conditions while enhancing realism.
[0008] The present invention employs the following technical solutions to achieve its objective: A method for generating few-shot target remote sensing images based on structure awareness and detail enhancement includes the following steps: S1. Under the condition of training with few samples, the parameters of the diffusion generative model are selectively unfrozen, allowing only modules related to conditional modeling and structural representation to participate in parameter updates, while freezing the remaining parameters of the latent space encoder, conditional encoder and diffusion backbone network. This reduces the scale of model parameter updates under the condition of limited training samples, reduces the risk of catastrophic forgetting and overfitting, and improves the stability of diffusion model training. S2. By using the Structural-Aware Edge Supervision Module (SAESM), the predicted results output by the diffusion generation model are subject to structural edge constraints. This allows the diffusion generation model to learn the stable geometric structure of specific targets such as buildings and roads in remote sensing images without the need for additional manual contour annotation, thus alleviating the problems of boundary blurring and structural degradation under conditions of few samples. S3. The Structure-Isolated Image Refinement Module (SIIRM) enhances the details of the pixel images output by the diffusion generation model. The gradient isolation mechanism prevents the perceptual constraints of the image detail enhancement stage from being backpropagated to the diffusion backbone network, thereby improving the perceptual realism of the generated images under conditions of few samples and avoiding adverse effects on the learning of geometric structures such as building boundaries. S4. Through the Inference-time Quality Self-Evaluation and Control Module (IQSECM), the generated results are evaluated and output is controlled during the model inference stage. When the generated image does not meet the preset quality conditions, regeneration is triggered, thereby adaptively improving the overall quality and reliability of the final remote sensing image generation results without human intervention.
[0009] Preferably, in step S1, the parameters of the diffusion generation model are selectively unfrozen, including: updating the parameters only for the cross-attention module, latent space encoder, and intermediate semantic fusion module in the diffusion backbone network, while freezing the remaining parameters in the text encoder and diffusion backbone network; the parameters that are updated are recorded as trainable parameters, and the remaining frozen parameters are recorded as frozen parameters; during the training and optimization process of the diffusion generation model, only the trainable parameters are updated, as shown in the following formula:
[0010] In the formula, The learning rate; Total loss; Represents a subset of trainable parameters. Represents a subset of frozen parameters; This represents the gradient operator.
[0011] Specifically, the set of all parameters of the diffusion generation model is denoted as and the set Divided into a subset of trainable parameters and frozen parameter subset And satisfy the following formula:
[0012] In the formula, Represents the empty set; a subset of trainable parameters. This only includes parameters for the cross-attention module, latent space encoder, and intermediate semantic fusion module; all other parameters are grouped into the frozen parameter subset. The parameters include those of the text encoder and the remaining convolutional / residual modules in the diffusion backbone network.
[0013] Specifically, in step S2, for the pixel image predicted by the diffusion generation model and the real remote sensing image, the structure-aware edge supervision module extracts the corresponding structural edge features respectively, and constructs a consistency constraint function based on the difference between the two structural edge features. As shown in the following formula:
[0014] In the formula, Represents an edge feature extraction operator; For the predicted pixel image, These are real remote sensing images.
[0015] Specifically, when applying structural edge constraints to the prediction results output by the diffusion generation model, the latent space prediction results obtained by the diffusion generation model at any step are decoded back into the pixel space; the decoder used in the decoding operation is denoted as... The predicted pixel image is , These are the latent variables corresponding to the prediction operation; For the predicted pixel images respectively and real remote sensing images Calculate edge features and use the gradient edge feature extraction operator. Let the horizontal gradient operator be... The vertical gradient operator is Then the gradient magnitude edge map of the image is represented by the following formula:
[0016] In the formula, the symbol Represents convolution operation; For the corresponding image features, i.e. or Therefore, a consistency constraint function is defined and constructed. , As a supervisory signal, it is fed back to the trainable parameter subset of the diffusion backbone network. middle.
[0017] Specifically, in step S3, the pixel image predicted by the diffusion generation model is enhanced through the structural isolation image detail enhancement module. This includes: after the diffusion generation model completes structural modeling, the pixel image output by the diffusion generation model is used as input, and a gradient isolation operation is introduced to block the back propagation of the gradient generated in the image detail enhancement stage to the diffusion backbone network; subsequently, without destroying the stable geometric structure of the specific target, the texture details and perceived realism of the generated image are enhanced. The final output of the structure-isolated image detail enhancement module is denoted as... As shown in the following formula:
[0018] In the formula, and Represents the convolutional mapping layer corresponding to the beginning and end; Represents a residual block sequence layer; Represents the channel attention module layer; This is the predicted pixel image.
[0019] Preferably, during the training phase of the diffusion generation model, the structure-isolated image detail enhancement module optimizes its own image detail enhancement network through pixel consistency constraints and perceptual consistency constraints. Pixel consistency constraints are used to limit the deviation between the thinning result and the real remote sensing image in pixel space, while perceptual consistency constraints are used to limit the difference between the thinning result and the real remote sensing image in high-level feature space.
[0020] Preferably, in step S4, the inference quality self-evaluation control module performs quality evaluation and output control on the generated results during the model inference stage. Specifically, this includes: obtaining the target region from the candidate generated images based on conditional information, and using a target category classifier to classify the target region to obtain a category consistency score. Simultaneously, a cross-modal visual-text alignment model is used to evaluate the quality of candidate generated images and obtain image quality scores. ; Class consistency score and image quality score Each image is compared with a corresponding preset threshold. When the preset discrimination condition is met, the corresponding generated image is output as the final prediction result.
[0021] Specifically, the preset discrimination conditions are as follows: and
[0022] In the formula, Scoring for consistency in the corresponding category The preset threshold, Image quality score The preset threshold is set; when the preset discrimination condition is not met, the corresponding generated image is not output, and the diffusion generation model is triggered to re-execute the generation process.
[0023] Specifically, in the inference phase of the diffusion generation model, conditional information is first received, including text description, target category information, and spatial constraint information of the target region. The diffusion generation model executes the generation process based on the conditional information and outputs candidate generated images. The candidate generated images are then entered into the target region acquisition unit and the image quality evaluation unit, respectively. In the target region acquisition unit, the corresponding target region is extracted from the candidate generated image. After being input into the target category classifier, the category probability distribution is obtained, thereby classifying the target region of the candidate generated image and obtaining a category consistency score. ; In the image quality assessment unit, cross-modal visual-text alignment is performed using CLIP-based image encoders and text encoders to obtain an image quality score. .
[0024] In summary, due to the adoption of this technical solution, the beneficial effects of this invention are as follows: This invention introduces a structure-aware edge supervision module, which explicitly constrains the structural edges of the generated results without requiring additional manual structural annotation. This guides the diffusion backbone network to focus on learning the geometric boundary features of remote sensing targets such as buildings and roads, effectively alleviating problems such as blurred boundaries and structural degradation in generated images under conditions with few samples, and improving the geometric consistency of the generated results.
[0025] This invention introduces a structural isolation image detail enhancement module to enhance the details of the diffusion model generation results in pixel space. By isolating the gradient transfer between the perceptual loss and the diffusion backbone network, it decouples the perceptual realism enhancement from the structural modeling process. Thus, under conditions of few samples, it improves the visual realism of the generated images and avoids the adverse effects of perceptual loss on the learning of geometric structures such as building boundaries.
[0026] This invention introduces a self-evaluation control module for inference quality, which comprehensively evaluates the category consistency and perception quality of the generated images during the model inference stage, and automatically filters and regenerates the generated images based on the evaluation results. This effectively eliminates low-quality generated results without human intervention, and adaptively improves the overall quality and application reliability of the final remote sensing image generation results. Attached Figure Description
[0027] The present invention is described in detail with reference to the following figures, which include five figures as follows: Figure 1 This is a schematic diagram illustrating the overall process of the method for generating remote sensing images of targets with few samples according to the present invention. Figure 2 This is a block diagram of the training structure of the diffusion generation model in the method of the present invention; Figure 3 This is a structural block diagram of the structure isolation image detail enhancement module in this invention; Figure 4 This is a structural block diagram of the reasoning quality self-assessment control module in this invention; Figure 5This is a schematic diagram comparing the generation results under the superposition of different module configurations in this invention. Detailed Implementation
[0028] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.
[0029] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.
[0030] Example 1 A method for generating few-shot target remote sensing images based on structure awareness and detail enhancement. Figure 1 The overall process of this method is briefly described below and can be viewed concurrently; the key steps of this method can be summarized as follows: S1. Under the condition of few-sample training, the parameters of the diffusion generative model are selectively unfrozen, allowing only modules related to conditional modeling and structural representation to participate in parameter updates, while freezing the remaining parameters of the latent space encoder, conditional encoder and diffusion backbone network. S2. By using the Structure-Aware Edge Supervision Module (SAESM), structural edge constraints are applied to the prediction results output by the diffusion generation model, guiding the diffusion generation model to learn the stable geometric structure of a specific target in the remote sensing image. S3. The pixel image output by the diffusion generation model is enhanced with detail through the Structural Isolation Image Detail Enhancement Module (SIIRM), and the perceptual constraints of the image detail enhancement stage are not propagated back to the diffusion backbone network through the gradient isolation mechanism. S4. The Inference Quality Self-Assessment Control Module (IQSECM) is used to assess the quality of the generated results and control the output during the model inference stage. When the generated image does not meet the preset quality conditions, regeneration is triggered.
[0031] This embodiment will provide a preferred description of the specific details involved in each step of the above method.
[0032] In step S1 of this embodiment, a conditional diffusion generation model based on latent space is used as the basic framework, and its training structure diagram is as follows. Figure 2 As shown. Let the training samples be... ,in To determine the number of training samples, in scenarios with few samples ; For real remote sensing images, The sequence number is used. This embodiment first uses a latent space encoder. Real remote sensing images in pixel form Mapped to the corresponding latent variables As shown in the following formula:
[0033] Subsequently, Gaussian noise is gradually introduced into the latent space during the forward diffusion process; given the diffusion step... Sampling noise It represents the true noise sampled from the standard normal distribution, and constructs a latent variable... As shown in the following formula:
[0034] In the formula, This is the cumulative coefficient in the noise dissemination scheduling, determined by the preset noise schedule.
[0035] In the diffusion generation model, the diffusion backbone network UNet uses conditional information. Conditions, including text / category / box encoding, are used to predict noise. In this embodiment, the training target uses the noise regression loss commonly used in conventional diffusion models. As shown in the following formula:
[0036] In the formula, Represents the calculation of expected value, symbol This represents the square of the L2 norm.
[0037] To mitigate the risks of catastrophic forgetting and overfitting associated with full parameter updates under limited sample conditions, this embodiment selectively unfreezes the model parameters. The set of all parameters of the diffusion-generated model is denoted as... , will set Divided into a subset of trainable parameters and frozen parameter subset And satisfy the following formula:
[0038] In the formula, Represents the empty set; a subset of trainable parameters. This only includes parameters for the cross-attention module, latent space encoder, and intermediate semantic fusion module; all other parameters are grouped into the frozen parameter subset. The parameters include those of the text encoder and the remaining convolutional / residual modules in the diffusion backbone network.
[0039] During the training and optimization of the diffusion generation model, only the trainable parameters are updated, as shown in the following equation:
[0040] In the formula, The learning rate; Total loss; Represents a subset of trainable parameters. Represents a subset of frozen parameters; This represents the gradient operator.
[0041] In step S2 of this embodiment, under conditions of few samples, the diffusion model generation results are prone to blurring of building / road boundaries. Therefore, this embodiment introduces a Structure-Aware Edge Supervision (SAESM) module to apply structural edge consistency constraints to the generation results, decoding the latent space prediction results obtained by the diffusion generation model at any step back to the pixel space; the decoder used for the decoding operation is denoted as... The predicted pixel image is , These are the latent variables corresponding to the prediction operation.
[0042] For the predicted pixel images respectively and real remote sensing images Calculate edge features and use the gradient edge feature extraction operator. Let the horizontal gradient operator be... The vertical gradient operator is Then the gradient magnitude edge map of the image is represented by the following formula:
[0043] In the formula, the symbol Represents convolution operation; For the corresponding image features, i.e. or Therefore, a consistency constraint function is defined and constructed. As shown in the following formula:
[0044] This will serve as a supervisory signal, which requires no additional manual contour annotation and is directly calculated from the real image. This supervisory signal will then be fed back to the trainable parameter subset of the diffusion backbone network via a differentiable decoding path. middle.
[0045] In step S3 of this embodiment, after completing the structural learning of the diffusion backbone network, in order to further improve the performance of the generated remote sensing image in terms of texture details, local contrast and overall visual naturalness, this embodiment introduces the Structure Isolation Image Detail Enhancement Module (SIIRM) during the training phase to refine the pixel images generated by the diffusion model through post-processing.
[0046] It should be noted that, under conditions of few samples, directly introducing perceptual loss into the training process of the diffusion backbone network can easily cause the model to prioritize texture similarity and ignore geometric consistency during optimization, resulting in blurred building boundaries or structural drift. Therefore, to avoid the above problems, this embodiment adopts a "structural isolation" approach to design the image detail enhancement process.
[0047] In this embodiment, the pixel image is predicted by the diffusion model. Before being fed into the image detail enhancement network, the input is first processed through a gradient isolation operation to obtain isolated input. As shown in the following formula:
[0048] This represents the gradient isolation operation, which ensures that gradients generated by pixel loss and perceptual loss will not be backpropagated to the diffusion backbone network during subsequent network refinement training. This ensures that the diffusion model is only subject to structure-related supervision, such as the constraints of diffusion loss and structural edge loss.
[0049] Subsequently, the imported cases will be isolated. The image is input into an image detail enhancement network for detail enhancement processing, resulting in a refined output image. .like Figure 3 As shown, the image detail enhancement network in the SIIRM module includes a feature mapping unit, a multi-scale residual feature extraction unit, a channel attention enhancement unit, and an output mapping unit. In the output stage, the thinning result is fused with the input image through a global residual connection.
[0050] The multi-scale residual feature extraction unit consists of multiple residual blocks, each with a different receptive field configuration, used to extract contextual texture information at different scales without changing the image spatial resolution; let the first... The void ratio of each residual block is Its output is denoted as , is represented as:
[0051] In the formula, The residual branch consists of two layers of convolution and nonlinearity; the index is omitted. Its form is described below:
[0052] In the formula, This represents a convolution operation, specifically a two-dimensional convolution. For convolution kernel; Represents ReLU activation; the convolution kernel uses a dilatation rate. To expand the receptive field, the corresponding equation for triple residual stacking is as follows:
[0053]
[0054]
[0055] Representing the residual stacking, the channel attention enhancement unit then enhances the feature responses that contribute significantly to image perceptual quality by adaptively weighting the feature channels, as shown in the following equation:
[0056] In the formula, Represents the channel attention module layer; symbol This represents element-wise multiplication broadcast via a channel; This is the channel weight vector, which is a feature. (here) For the number of channels, For high, (For width) global average pooling, then through two layers Convolution and The result is as follows:
[0057] In the formula, This represents the feature vector after global average pooling. Global residual connections are used to maintain the overall geometric stability of the original image, preventing the thinning process from damaging structural information such as building outlines and road directions. In this embodiment, the final output of the Structure Isolated Image Detail Enhancement Module (SIIRM) is denoted as... As shown in the following formula:
[0058] In the formula, and Represents the convolutional mapping layer corresponding to the beginning and end; Represents a residual block sequence layer; Represents the channel attention module layer; This is the predicted pixel image.
[0059] As a preferred embodiment, during the training phase of the diffusion generation model, the Structure Isolated Image Detail Enhancement Module (SIIRM) optimizes its own image detail enhancement network through pixel consistency constraints and perceptual consistency constraints. Pixel consistency constraints are used to limit the deviation between the thinning result and the real remote sensing image in pixel space, while perceptual consistency constraints are used to limit the difference between the thinning result and the real remote sensing image in high-level feature space.
[0060] Due to the existence of the gradient isolation mechanism, the above constraints are only used to update the parameters of the image detail enhancement module SIIRM and do not participate in the update of the diffusion backbone network parameters, thereby achieving effective decoupling between the stability of the generated structure and the enhancement of visual realism under the condition of few samples.
[0061] In step S4 of this embodiment, as follows Figure 4 As shown, during the inference stage, an inference quality self-assessment control module (IQSECM) is introduced to assess the quality of candidate images output by the diffusion generation model and control the output, thereby improving the reliability and stability of the final generated remote sensing images.
[0062] In the reasoning process, conditional information is received first. This includes text descriptions, target category information, and spatial constraint information of the target region. The diffusion generation module bases its decisions on this conditional information. Execute the generation process and output candidate generated images. Candidate generated images They will respectively enter the target area acquisition unit and the image quality assessment unit.
[0063] In the target area acquisition unit, based on condition information The spatial constraints provided in the image are used to generate images from the candidates. Extract the corresponding target region As shown in the following formula:
[0064] In the formula, This represents the operation of extracting the target region of interest; target region After extraction, the data is input into the target category classifier. In, and from this, output the class probability distribution. As shown in the following formula:
[0065] In the formula, This represents the output of the softmax layer. Based on the output class probability distribution. For candidate generated images The target region is then classified to obtain a class consistency score. As shown in the following formula:
[0066] In the formula, Represents a category.
[0067] In the image quality assessment unit, candidate generated images Image quality scores were obtained through cross-modal visual-text alignment. As shown in the following formula:
[0068] In the formula, and These are CLIP's image encoder and text encoder, respectively.
[0069] In this embodiment, after obtaining the category consistency score... and image quality score Then, the two types of scores are input into the threshold determination unit for joint determination. The class consistency score of the generated image is then used for this determination. Not lower than the preset category threshold And image quality score Not lower than the preset quality threshold When the generated image satisfies the output condition, the preset discrimination condition is as follows: and
[0070] The output control unit outputs the generated image that meets the preset discrimination conditions as the final generation result. When the generated image fails to meet the preset threshold conditions in any of the above-mentioned scoring dimensions, the output control unit does not output the current generation result, but instead triggers the diffusion generation module to re-execute the generation process, thus forming a control flow of generation, evaluation, judgment, and regeneration. Through this control mechanism, low-quality generation results can be automatically filtered out without manual intervention, and generation can continue until a remote sensing image that meets the quality requirements is obtained.
[0071] Example 2 Based on Example 1, this example illustrates a comparison of the application of methods under the same test scenario. Figure 5 This is a comparative diagram showing the remote sensing image generation results after the method of the present invention gradually introduces different technical modules; Figure 5 The images, from left to right, show the generated results after full-parameter training, fine-tuning training, introducing a structure-aware edge supervision module (+SAESM), further introducing a structure-isolated image detail enhancement module (+SIIRM), and finally introducing an inference quality self-evaluation control module (+IQSECM).
[0072] Among them, the "full parameter training" method is prone to problems such as structural disorder, texture noise and semantic instability in the generated results due to large-scale updates of model parameters under the condition of few samples; "fine-tuning training" alleviates the model divergence phenomenon to a certain extent, but still has shortcomings such as blurred target structure and unclear building boundaries.
[0073] Based on this, after introducing the Structure-Aware Edge Supervision (SAESM) module, the contours and geometric structures of targets such as roads and buildings in the generated images are significantly more stable, indicating that edge consistency constraints can effectively guide the diffusion model to learn key structural information in remote sensing scenes.
[0074] By further introducing the Structure Isolation Image Detail Enhancement Module (SIIRM), the generated results show significant improvement in image detail and visual naturalness while maintaining the stability of the original geometric structure. This indicates that by isolating the perceptual refinement process from the learning process of the diffusion backbone network structure, image detail can be enhanced without destroying the structure.
[0075] Based on this, by combining the inference quality self-assessment control module IQSECM, the generated results can be effectively screened out for low-quality generated samples and output results that meet the quality requirements through quality assessment and closed-loop control, thereby further improving the stability and reliability of the final generated remote sensing images.
[0076] The above comparison shows that the various modules proposed in this invention work together under conditions of few samples to gradually improve the structural stability, detail representation, and output consistency in the remote sensing image generation process, thus verifying the effectiveness and rationality of the technical solution of this invention.
Claims
1. A method for generating few-shot target remote sensing images based on structure awareness and detail enhancement, characterized in that, Includes the following steps: S1. Under the condition of few-sample training, the parameters of the diffusion generative model are selectively unfrozen, allowing only modules related to conditional modeling and structural representation to participate in parameter updates, while freezing the remaining parameters of the latent space encoder, conditional encoder and diffusion backbone network. S2. Through the structure-aware edge supervision module, the predicted results output by the diffusion generation model are subject to structural edge constraints, which guides the diffusion generation model to learn the stable geometric structure of a specific target in the remote sensing image. S3. The image detail enhancement module with structural isolation enhances the pixel image output by the diffusion generation model and uses a gradient isolation mechanism to prevent the perceptual constraints of the image detail enhancement stage from being backpropagated to the diffusion backbone network. S4. Through the inference quality self-evaluation control module, the quality of the generated results is evaluated and the output is controlled during the model inference stage. When the generated image does not meet the preset quality conditions, regeneration is triggered.
2. The method for generating remote sensing images of targets with few samples according to claim 1, characterized in that, In step S1, the parameters of the diffusion generation model are selectively unfrozen, including: updating the parameters only for the cross-attention module, latent space encoder, and intermediate semantic fusion module in the diffusion backbone network, while freezing the remaining parameters in the text encoder and diffusion backbone network; the parameters that are updated are recorded as trainable parameters, and the remaining frozen parameters are recorded as frozen parameters; during the training and optimization process of the diffusion generation model, only the trainable parameters are updated, as shown in the following formula: In the formula, The learning rate; Total loss; Represents a subset of trainable parameters. Represents a subset of frozen parameters; This represents the gradient operator.
3. The method for generating remote sensing images of targets with few samples according to claim 2, characterized in that: Let the set of all parameters of the diffusion generation model be denoted as and the set Divided into a subset of trainable parameters and frozen parameter subset And satisfy the following formula: In the formula, Represents the empty set; a subset of trainable parameters. This only includes parameters for the cross-attention module, latent space encoder, and intermediate semantic fusion module; all other parameters are grouped into the frozen parameter subset. The parameters include those of the text encoder and the remaining convolutional / residual modules in the diffusion backbone network.
4. The method for generating remote sensing images of targets with few samples according to claim 1, characterized in that: In step S2, for the pixel image predicted by the diffusion generation model and the real remote sensing image, the structure-aware edge supervision module extracts the corresponding structural edge features respectively, and constructs a consistency constraint function based on the difference between the two structural edge features. As shown in the following formula: In the formula, Represents an edge feature extraction operator; For the predicted pixel image, These are real remote sensing images.
5. The method for generating remote sensing images of targets with few samples according to claim 4, characterized in that: When applying structural edge constraints to the prediction results output by the diffusion generation model, the latent space prediction results obtained by the diffusion generation model at any step are decoded back to the pixel space; the decoder used in the decoding operation is denoted as . The predicted pixel image is , These are the latent variables corresponding to the prediction operation; For the predicted pixel images respectively and real remote sensing images Calculate edge features and use the gradient edge feature extraction operator. Let the horizontal gradient operator be... The vertical gradient operator is Then the gradient magnitude edge map of the image is represented by the following formula: In the formula, the symbol Represents convolution operation; For the corresponding image features, i.e. or Therefore, a consistency constraint function is defined and constructed. , As a supervisory signal, it is fed back to the trainable parameter subset of the diffusion backbone network. middle.
6. The method for generating remote sensing images of targets with few samples according to claim 1, characterized in that, In step S3, the pixel image predicted by the diffusion generation model is enhanced through the structural isolation image detail enhancement module. Specifically, after the diffusion generation model completes structural modeling, the pixel image output by the diffusion generation model is used as input, and a gradient isolation operation is introduced to block the back propagation of the gradient generated in the image detail enhancement stage to the diffusion backbone network. Subsequently, the texture details and perceived realism of the generated image are enhanced without destroying the stable geometric structure of the specific target. The final output of the structure-isolated image detail enhancement module is denoted as... As shown in the following formula: In the formula, and Represents the convolutional mapping layer corresponding to the beginning and end; Represents a residual block sequence layer; Represents the channel attention module layer; This is the predicted pixel image.
7. The method for generating remote sensing images of targets with few samples according to claim 6, characterized in that: During the training phase of the diffusion generation model, the structural isolation image detail enhancement module optimizes its own image detail enhancement network through pixel consistency constraints and perceptual consistency constraints. Pixel consistency constraints are used to limit the deviation between the thinning result and the real remote sensing image in pixel space, while perceptual consistency constraints are used to limit the difference between the thinning result and the real remote sensing image in high-level feature space.
8. The method for generating remote sensing images of targets with few samples according to claim 1, characterized in that, In step S4, the inference quality self-evaluation control module performs quality evaluation and output control on the generated results during the model inference stage. Specifically, this includes: obtaining the target region from the candidate generated images based on conditional information, and using a target category classifier to classify the target region to obtain a category consistency score. Simultaneously, a cross-modal visual-text alignment model is used to evaluate the quality of candidate generated images and obtain image quality scores. ; Class consistency score and image quality score Each image is compared with a corresponding preset threshold. When the preset discrimination condition is met, the corresponding generated image is output as the final prediction result.
9. The method for generating remote sensing images of targets with few samples according to claim 8, characterized in that, The preset discrimination condition is as follows: and In the formula, Scoring for consistency in the corresponding category The preset threshold, Image quality score The preset threshold is set; when the preset discrimination condition is not met, the corresponding generated image is not output, and the diffusion generation model is triggered to re-execute the generation process.
10. The method for generating remote sensing images of targets with few samples according to claim 8, characterized in that: In the inference phase of the diffusion generation model, conditional information is first received, including text description, target category information, and spatial constraint information of the target region. The diffusion generation model performs the generation process based on the conditional information and outputs candidate generated images. The candidate generated images are then entered into the target region acquisition unit and the image quality evaluation unit, respectively. In the target region acquisition unit, the corresponding target region is extracted from the candidate generated image. After being input into the target category classifier, the category probability distribution is obtained, thereby classifying the target region of the candidate generated image and obtaining a category consistency score. ; In the image quality assessment unit, cross-modal visual-text alignment is performed using CLIP-based image encoders and text encoders to obtain an image quality score. .