Progressive multi-scale diffusion with hybrid upsampling with regional noise injection for high-resolution image generation

Abstract

A computer-implemented method for generating high-resolution images using a single diffusion process with progressive latent-space resolution scaling, hybrid upsampling, and targeted regional refineme

Description

The invention relates to computer-implemented and Al-based (artificial intelligence) image generation systems, particularly those using diffusion-based generative models combined with hybrid non-diffusion upsampling, targeted localised detail enhancement, and temporal stability techniques for producing high-resolution images. It is applicable to still images, video sequences, and other visual domains requiring high perceptual fidelity under constrained computational and memory resources. BACKGROUND OF THE INVENTION Diffusion-based generative models have demonstrated state-of-the-art results in synthesising images from textual or visual prompts. GB2634366 discloses a method using a low-resolution diffusion pass, upsampling without noise injection, and a second diffusion model to refine the high-resolution output. While effective, this approach requires two separate models, has higher computational costs, and lacks mechanisms for selective regional detail enhancement. There is a need for a high-resolution image generation process that: Operates within tighter compute / memory budgets. Improves fidelity in low-detail regions without degrading already-detailed regions. Extends easily to video and other domains. SUMMARY OF THE INVENTION The invention provides a method for generating high-resolution images that combines progressive latent-space resolution scaling within a single diffusion process, a hybrid upsampling stage, and a localised detail enhancement process guided by a deficiency map. Unlike existing two-model cascades such as GB2634366, which require separate low- and high-resolution diffusion models and omit targeted regional enhancement, the invention performs all resolution scaling within a single diffusion trajectory and uses a sequential hybrid upsampler comprising a structure-preserving transformer-based super-resolution network followed by a texture-focused generative adversarial network. A deficiency map is computed from the upsampled image using at least one of frequency-domain, entropy-based, or gradient magnitude analysis to identify low-detail regions. A localised detail enhancement process is then applied exclusively to the identified regions; in some embodiments this comprises injecting noise into those regions, while in others it comprises region-specific refinement operations without noise. A masked diffusion refinement is then performed only on the enhanced regions, leaving the rest of the image unchanged. This coordinated sequence of steps produces a synergistic technical effect, achieving up to 28% fewer diffusion steps for equal or improved output quality at a fixed GPU / memory limit compared to conventional methods. The approach improves objective quality metrics such as FID, CLIP-Score, and LPIPS at 4x and 8x scaling, reduces peak memory requirements, and extends naturally to still images, video sequences, and other high-fidelity visual domains. The integration of progressive resolution scaling, hybrid upsampling, and deficiency-map-driven localised enhancement was not suggested by any known combination of prior art techniques at the priority date and would not be predictable from the individual elements alone. In contrast to existing systems, the invention is not a mere juxtaposition of known components but a coordinated sequence of processing stages that interact to produce a synergistic technical effect. By combining progressive latent-space resolution scaling within a single diffusion process, a hybrid upsampling stage comprising a structure-preserving transformer-based super-resolution network followed by a texture-focused generative adversarial network, and a deficiency-map-driven masked micro-diffusion refinement, the invention achieves high-resolution outputs with fewer computational steps and lower peak memory requirements than multi-model cascades such as those described in GB2634366. Progressive upscaling methods such as those disclosed in academic work address resolution growth but do not incorporate a sequential hybrid upsampler tuned for complementary global structure and fine texture restoration. Likewise, known inpainting or local refinement techniques do not teach computing a low-detail mask from frequency, entropy, or gradient analysis and confining re-noise injection and diffusion passes to only those regions. The integration of these elements yields an efficiency-quality profile, demonstrated by benchmark data, that would not be predictable from the individual techniques alone and was not suggested by any combination of prior art known at the priority date. INDUSTRIAL APPLICABILITY The invention is applicable in digital content creation, advertising, film visual effects, gaming asset production, e-commerce, and any domain requiring high-fidelity image synthesis under constrained compute budgets. MARKET ADVANTAGE The invention delivers measurable performance and cost benefits over prior art methods such as those disclosed in GB2634366. By reducing the number of required diffusion steps by approximately 28% (fig 2) and lowering peak VRAM usage by approximately 25%, (fig 2) the invention enables high-fidelity image generation on a broader range of hardware, including mid-tier consumer and professional GPUs that cannot execute prior art methods within available memory limits. This improved efficiency reduces per-image generation time, lowering operating costs for cloudbased services and increasing throughput for production workflows. The ability to achieve equal or better objective quality metrics (including FID, CLIP-Score, and LPIPS) while operating with reduced computational and memory demands provides a competitive advantage in markets where turnaround time, scalability, and hardware accessibility are critical. These technical efficiencies expand the addressable market to include mobile and edge devices, creative professionals using commodity hardware, and service providers operating under strict cost-per-operation constraints. As a result, the invention has broad commercial applicability across digital media production, advertising, entertainment, e-commerce, and other sectors where high-resolution image synthesis is required. DETAILED DESCRIPTION OF THE INVENTION Referring to FIG. 1, the invention comprises a computer-implemented method for generating a high-resolution image through a sequence of coordinated stages. The stages are described below with corresponding reference numerals from FIG. 1. 1. Input Handling (100) The system receives prompts in text, image, or multimodal form (100), optionally with metadata defining style, resolution, and domain. In some embodiments, the metadata includes target scale factors (e.g., 4x or 8x) and compute constraints such as peak VRAM usage. 2. Progressive Diffusion Stage (110) The diffusion process (110) begins at a low resolution (for example, a 128x128 latent grid). At one or more predefined timesteps during the denoising trajectory, the latent representation is upscaled to a higher resolution without restarting the noise schedule, allowing previously generated detail to be preserved and refined. In some embodiments, the resolution is increased at least twice, for example from the base resolution to an intermediate resolution and from the intermediate resolution to the final target resolution. Model weights are shared across these stages, and attention layers dynamically adapt to the new resolution. 3. Hybrid Upsampling (120) Following the progressive diffusion stage (110), a hybrid upsampling process (120) is applied. In some embodiments, the hybrid process comprises a transformer-based structural super-resolution network that enhances global geometry, followed in sequence by a generative adversarial network (GAN) trained for texture enhancement. The GAN training set may be distinct from the transformer training set and may include fine texture patches such as fabric weaves, foliage details, skin pores, or stone surfaces to ensure realistic microstructure. In alternative embodiments, the structural enhancement stage of the hybrid upsampling process (120) may be implemented using non-transformer-based architectures such as convolutional neural network (CNN) super-resolution models, recurrent neural network (RNN) super-resolution models, or graph neural network (GNN) upsamplers, each trained to preserve global image structure while increasing resolution. Likewise, the texture enhancement stage may be implemented using other non-diffusion detail synthesis architectures such as variational autoencoders (VAE) configured for fine detail generation, patch-based texture synthesis networks, or style transfer networks trained to add high-frequency texture content. These alternative architectures may be used individually or in combination to achieve a complementary effect between structural fidelity and perceptual texture quality in the generated high-resolution output. 4. Deficiency Map Computation (130) The upsampled image is analysed to identify regions lacking fine detail (130). This is achieved by generating a deficiency map using one or more algorithms selected from frequency-domain analysis to detect low-frequency dominant areas, entropy-based analysis to detect uniform or low-information regions, gradient magnitude analysis to identify areas with low edge density, waveletdomain decomposition to isolate areas lacking high-frequency components, Laplacian variance measures to quantify sharpness, saliency mapping to identify perceptually important but underdetailed regions, or machine learning-based region quality prediction models trained on annotated image quality datasets. In some embodiments, such machine learning models may be neural networks that take as input pixel values, frequency-domain coefficients, gradient measures, or temporal consistency metrics, and output a per-pixel or per-region quality score. In certain configurations, two or more of these analyses are combined to form a binary or weighted low-detail mask for subsequent localised detail enhancement. 5. Localised Detail Enhancement Process (140) A localised detail enhancement process is applied only to regions indicated by the deficiency map (140). This process is configured to increase perceptual quality and fine detail in the identified regions without unnecessarily altering already-detailed areas. In some embodiments, the localised detail enhancement process comprises injecting noise into the low-detail regions; the noise may be Gaussian noise, Perlin noise, or a learned noise pattern generated by a trained noise-prediction network. In other embodiments, the localised detail enhancement process comprises a regionspecific refinement operation without noise injection, such as applying a trained texture synthesis network, patch-based detail reconstruction, or style transfer targeted to the masked regions. The amplitude, pattern, and spatial distribution of any noise or generated detail can be controlled to avoid oversaturation or distortion of image features. The process may be applied once or iteratively, with each iteration recalculating the deficiency map to target residual low-detail areas. 6. Masked Micro-Diffusion Refinement (150) A short diffusion refinement process is applied only to the masked regions into which noise has been injected (150), leaving unmasked regions unchanged. In some embodiments, this refinement is limited to between three and five denoising steps to conserve computational resources while improving local detail. The mask is used to confine the model's attention to the deficient areas. 7. Output Post-Processing The refined image is optionally passed through a post-processing stage including colour grading and tone mapping to optimise visual appeal. In some embodiments, histogram equalisation or local contrast enhancement is also applied. 8. Video Output and Temporal Consistency In some embodiments, the method outputs a sequence of frames for a video. Temporal consistency correction is applied between frames to maintain coherence of structures and textures over time. This may be achieved using one or more of the following techniques: optical flow correlation, in which inter-frame motion vectors are computed and used to warp or align features between consecutive frames; feature map tracking, in which intermediate neural network feature maps are tracked across frames to preserve consistent activation patterns; depth and parallax correlation, in which per-pixel depth maps or disparity maps are used to ensure correct spatial alignment of objects during camera or subject motion; and learned temporal embeddings, in which a neural network encodes temporal relationships into the latent space so that generated content maintains stylistic and structural consistency over multiple frames. These methods may be used individually or in combination, and in certain configurations the deficiency map computation is extended to incorporate temporal stability metrics so that low-detail or inconsistent regions are selectively enhanced while preserving frame-to-frame continuity. 9. Iterative Refinement In certain configurations, the hybrid upsampling (120) and masked micro-diffusion refinement (150) are applied iteratively for multi-stage enhancement. Each iteration may compute a new deficiency map (130) based on the updated image. 10. Compute Efficiency The method may be configured to operate within a predefined peak VRAM budget by controlling the number of latent resolution increments and reusing intermediate feature maps between stages. This allows deployment on hardware with limited memory capacity. Compared to a two-model cascade approach, the method reduces redundant denoising passes and limits refinement to deficient regions, thereby reducing the number of diffusion steps required to achieve a target Frechet Inception Distance (FID) score. 11. Extended Domains In addition to still images and sequential video frames, the method is applicable to panoramic images, spherical projections, volumetric datasets, and rendered 3D scene outputs. For panoramic and spherical projections, the progressive diffusion and hybrid upsampling stages are adapted to operate in a coordinate system aligned with the projection type, preserving geometric continuity across seams. For volumetric or 3D scene outputs, the method processes depth-aligned slices or view-dependent renderings, with temporal or spatial consistency correction applied to ensure coherence across perspectives. These adaptations enable the invention to generate high-resolution content for virtual reality, augmented reality, simulation environments, and other immersive media applications, further extending its commercial applicability beyond the capabilities of existing prior art such as GB2634366. 12. In some embodiments, the deficiency map may be computed by a learned model, such as a convolutional or transformer-based neural network trained on labelled image patches representing high- and low-detail regions. Other approaches include wavelet transform analysis, Laplacian variance computation, edge density histograms, saliency prediction maps, or combinations thereof. BENCHMARK EVALUATION Setup: Dataset: 100 COCO-2017 prompts + 20 art prompts. Hardware: NVIDIA A100 (40 GB), peak VRAM <22 GB. Target: 4x and 8x upscale from base 512x512. Method Models Used Total Steps Peak VRAM FID xp CLIP-Score 1s LPIPS xp Baseline (GB2634366-type) 2x diffusion 100 28 GB 14.2 0.273 0.081 Ours lx diffusion + hybrid 72 21GB 13.7 0.279 0.078 | Config. | Steps । ------------------ । ..... | Full | 72 | No prog, resize | 94 | No hybrid SR | 80 | No masked re-noise | 75 | FID | CLIP 1s | LPIPS 4 / | | ..... | ...... | ....... | | 13.7 | 0.279 | 0.078 | | 14.6 | 0.274 | 0.081 | | 14.1 | 0.276 | 0.080 | | 14.0 | 0.275 | 0.080 | TECHNICAL ADVANTAGES OVER GB2634366 Feature / Metric GB2634366(Adobe) Present Invention Improvement Diffusion Models Used Two separate diffusion models (low-res + high-res) Single diffusion process with progressive latent resolution scaling Eliminates need for second model Diffusion Steps 100 72 28% fewer steps Peak VRAM Usage 28 GB 21GB 25% lower VRAM requirement Upsampling Method Diffusion-to-diffusion upsampling without noise Hybrid non-diffusion upsampling (Transformer + GAN) with optional alternative architectures Broader architectural flexibility Targeted No selective regional Deficiency-map-driven localised Region-specific improvement without over-processing Refinement refinement detail enhancement process Feature / Metric GB2634366(Adobe) Present Invention Improvement Output Quality FID: 14.2, CLIP: 0.273, LPIPS: 0.081 FID: 13.7, CLIP: 0.279, LPIPS: 0.078 Equal or better across all metrics Hardware Compatibility High-end GPUs only Mid-tier and high-end GPUs Expands addressable market Domain Applicability Still images only Still images, video frames, panoramas, 3D projections Wider commercial applicability

Claims

Independent Claimsl.A computer-implemented method for generating a high-resolution image, comprising receiving an input prompt, generating by a diffusion process an initial latent-space image, progressively increasing a resolution of the latent-space image during the diffusion process without resetting the diffusion noise schedule, applying a hybrid upsampling process comprising a transformer-based network configured for structural enhancement that outputs intermediate feature maps and a generative adversarial network configured for texture enhancement receiving the intermediate feature maps concatenated with a residual from the progressive diffusion output, computing a deficiency map identifying low-detail regions in the upsampled image by performing a multi-resolution frequency analysis, applying adaptive entropy thresholding, and refining with a neural network trained on perceptual detail loss maps, applying a localised detail enhancement process to the low-detail regions identified in the deficiency map, the localised detail enhancement process comprising at least one of injecting noise into the low-detail regions and performing a region-specific refinement operation, performing a masked diffusion refinement within the same diffusion noise schedule only on the low-detail regions that have undergone the localised detail enhancement process, and outputting the refined high-resolution image wherein for sequential frame generation the deficiency map is propagated to subsequent frames via motion-compensated warping with temporal embedding refinement to maintain cross-frame coherence.2.A system comprising one or more processors and memory storing instructions that, when executed by the one or more processors, cause the system to perform the method of claim 1.3.A non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause the method of claim 1 to be performed.Dependent Claims4.The method of claim 1, wherein the progressive increase in resolution occurs at least twice during the diffusion process with an intermediate resolution stage followed by a final target resolution stage.5.The method of claim 1, wherein the transformer-based network is a vision transformer trained for structural super-resolution using paired low- and high-resolution training data.6.The method of claim 1, wherein the generative adversarial network is a patch-based GAN trained on texture patch datasets distinct from those used for the transformer-based network and having multiple receptive field sizes for multi-scale texture synthesis.7.The method of claim 1, wherein the deficiency map computation further comprises combining results of gradient magnitude analysis, wavelet-domain decomposition, Laplacian variance measures, and saliency map generation to refine low-detail region detection.8.The method of claim 1, wherein the localised detail enhancement process comprises injecting noise into the low-detail regions, the noise being Gaussian noise, Perlin noise, or a learned noise pattern, and dynamically adjusting noise amplitude or refinement strength based on local image structure and target perceptual quality metrics.9.The method of claim 1, wherein the masked diffusion refinement is limited to between three and five denoising steps and reuses intermediate feature maps from the progressive diffusion stage to reduce computational cost.lO.The method of claim 1, further comprising applying colour grading, tone mapping, and local contrast enhancement to the final image output.ll.The method of claim 1, wherein the temporal consistency correction comprises combining optical flow correlation, feature map tracking, depth and parallax correlation, and learned temporal embeddings.12.The method of claim 1, wherein the progressive resolution scaling and masked diffusion refinement are applied iteratively for multi-stage enhancement with each stage recalculating the deficiency map.13.The method of claim 1, wherein the method operates within a predefined peak VRAM budget by controlling latent resolution increments and reusing cached intermediate features.14.The method of claim 1, wherein said integration of hybrid upsampling and masked micro-diffusion refinement reduces peak VRAM usage by at least 20 percent and total diffusion steps by at least 25 percent compared to applying said processes in separate passes.15.The method of claim 1, wherein the structural enhancement network comprises at least one of a convolutional neural network-based super-resolution model, a recurrent neural network-based super-resolution model, or a graph neural network-based upsampler.16.The method of claim 1, wherein the texture enhancement network comprises at least one of a variational autoencoder configured for detail synthesis, a style transfer network trained to add high-frequency texture content, or a diffusion-based super-resolution module configured for non-global refinement.17.The method of claim 1, wherein for video or 3D panoramic content the deficiency map propagation is further conditioned on depth-aware warping to maintain parallax consistency in refined regions.18.The method of claim 1, wherein the localised detail enhancement process incorporates learned temporal embeddings to adjust refinement strength across frames thereby reducing temporal flicker.

Images