Image Processing Method and System Based on Global Correction Posterior Sampling

Abstract

This invention provides an image processing method and system based on global correction posterior sampling, comprising: constructing input data and task conditions; wherein, the input data includes: an input image and a task type; initializing latent variables and adding noise in a single step on the input image to obtain noisy latent variables; using the noisy latent variables as input to a pre-trained diffusion model, and estimating the current clean sample in conjunction with the task type; using the current clean sample to perform a correction update oriented towards global error, and applying specific constraints to the correction update process for different task types and their task conditions to obtain the corrected estimation result; using the corrected estimation result to generate the sample of the previous time step and iterating to obtain the image processing result corresponding to the task type. This invention achieves multi-task compatibility, balances global correction and task-specific constraints, and possesses zero-sample training-free deployment capability within a unified framework.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and computer vision technology, specifically to an image processing method and system based on global correction posterior sampling, and also to a corresponding computer terminal and computer-readable storage medium. Background Technology

[0002] With the widespread application of diffusion models in image generation and restoration, utilizing image priors from pre-trained diffusion models to perform tasks such as image editing, image reconstruction, super-resolution, and deblurring has become an important technical approach in the field of image processing. For low-level vision tasks, it is necessary to recover detailed textures under degraded observation conditions; for high-level vision tasks, it is necessary to perform semantic modifications based on target text prompts or conditional constraints while maintaining the main structure, background, and layout of the source image.

[0003] Existing technologies can be broadly categorized into two types: trained methods and training-free methods. Trained methods typically require collecting large-scale paired samples or cue-image samples for a specific task and retraining an additional control network or task-specific model. While these approaches may achieve good results on a single task, they have high data and computational costs, limited generalization capabilities, and are difficult to flexibly transfer between different degradation models, different editing conditions, and different application scenarios.

[0004] Existing training-free methods typically incorporate observation constraints into the diffusion sampling process, gradually correcting the current state through posterior sampling, thereby utilizing a pre-trained diffusion model as an image prior. However, research reveals that most existing training-free methods still rely on an update approach of "local correction at adjacent time steps": on the one hand, in the generation process based on classifier-free guidance, unconditional branches accumulate biases over multiple inversions or sampling steps, causing the results to gradually deviate from the source image or target constraints; on the other hand, relying solely on local updates between the current and previous steps makes it difficult to promptly correct the global errors accumulated in previous steps, easily leading to local optima and resulting in problems such as background drift, semantic shift, loss of detail, and texture artifacts.

[0005] Furthermore, the degree of ill-conditioning varies significantly across different image processing tasks. Image editing and reconstruction typically have strong constraints from source image features, while in lower-level tasks such as super-resolution and deblurring, observation information is weaker and the inverse problem is more ill-posed. Therefore, relying solely on general sampling updates is insufficient to simultaneously achieve observation consistency, source image preservation, and target detail restoration. Summary of the Invention

[0006] To address the aforementioned shortcomings in the existing technology, one embodiment of the present invention provides an image processing method and system based on global correction posterior sampling, and also provides a corresponding computer terminal and computer-readable storage medium.

[0007] According to a first aspect of the present invention, an image processing method based on global correction posterior sampling is provided, comprising: Construct input data and task conditions; wherein, the input data includes: input image and task type; The input image is initialized with latent variables and noise is added in one step to obtain initial noisy latent variables; Based on the initial noisy latent variables, the following iterative steps are performed: s1, using the initial noisy latent variables as input to the pre-trained diffusion model, and combining the task type, estimate the current clean sample; s2, using the current clean sample, perform a correction update oriented towards global error, and apply specific constraints to the correction update process for different task types and their task conditions to obtain the corrected estimation result; s3, using the corrected estimation result, generate the previous time sample as a new noisy latent variable and repeat s1~s3 until the preset stopping condition is met; After the iterative steps are completed, the image processing result corresponding to the task type is obtained.

[0008] According to a second aspect of the present invention, an image processing system based on global correction posterior sampling is provided, comprising: A data construction module is used to construct input data and task conditions; wherein, the input data includes: input image and task type; The noise-adding module is used to initialize latent variables and add noise in one step to the input image to obtain initial noisy latent variables, which are then sent to the diffusion model estimation module as initial input latent variables. The diffusion model estimation module continuously uses the received noisy latent variables as input to the pre-trained diffusion model and, in conjunction with the task type, estimates the current clean sample until a preset stopping condition is reached to obtain the target output latent variable. The global correction and update module is used to continuously use the current clean samples to perform correction and update for global errors, and to apply specific constraints to the correction and update process for different task types and task conditions to obtain the corrected estimation results until the preset stopping condition is reached. An iterative generation module is used to continuously generate samples from the previous time step using the corrected estimation results, and input them as new noise latent variables into the diffusion model estimation module until a preset stopping condition is reached to obtain the target output latent variable. The image output module is used to obtain image processing results corresponding to the task type based on the obtained target output latent variables.

[0009] According to a third aspect of the present invention, a computer terminal is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it can be used to perform the methods described above in the present invention, or to run the system described above in the present invention.

[0010] According to a fourth aspect of the present invention, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, can be used to perform the methods described above in the present invention, or to run the system described above in the present invention.

[0011] By adopting the above technical solution, the present invention has at least one of the following beneficial effects compared with the prior art: This invention employs a unified posterior sampling framework and a pre-trained diffusion model prior, eliminating the need to retrain a dedicated network for each type of image processing task. It can support multiple tasks such as image reconstruction, image editing, super-resolution, and deblurring under zero-shot conditions, achieving greater versatility and lower deployment costs. It solves the problem of how to support multiple image processing tasks such as image reconstruction, image editing, super-resolution, and deblurring simultaneously without retraining the base model.

[0012] This invention employs a global correction strategy based on input feature injection, observation consistency, and stochastic dynamics. It can directly correct historical accumulated errors during the sampling process, reduce background drift and semantic shift caused by unconditional branching, and achieve a better balance between preserving key features of the source image and satisfying target constraints. It solves the problem of how to suppress error accumulation caused by unconditional branching and stepwise local correction during diffusion sampling and improve the ability of the result to preserve key features of the source image.

[0013] This invention designs task-specific constraints for highly ill-posed inverse problems, including a mask observation mechanism for image editing, a data consistency and null space constraint mechanism for super-resolution, and an image-blur kernel joint optimization mechanism for deblurring. Therefore, it can improve the restoration quality, structural stability, and robustness under complex degradation conditions. It solves the problem of how to introduce task-appropriate constraint mechanisms in low-level inverse problem scenarios so that the results can simultaneously satisfy observation consistency, detail restoration, and structural stability.

[0014] This invention replaces multi-step inversion with single-step noise addition and combines a few-step solver with global correction updates. While maintaining high quality, this invention has good inference efficiency and is suitable for deployment in scenarios such as scientific research, industrial image enhancement, content production, and human-computer interactive image editing. It solves the problem of how to improve the efficiency, robustness, and engineering deployability of training-free image processing methods with a unified inference paradigm.

[0015] This invention addresses the problems of high training costs, severe local error accumulation, and insufficient adaptability to complex inverse problems in existing technologies by combining a unified training-free framework, global correction, and task-specific constraints. Attached Figure Description

[0016] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is a flowchart illustrating the image processing method based on global correction posterior sampling in a preferred embodiment of the present invention.

[0017] Figure 2 This is a schematic diagram of the components of an image processing system based on global correction posterior sampling in a preferred embodiment of the present invention.

[0018] Figure 3 This is a schematic diagram of the posterior sampling process based on a global optimization strategy for different tasks in a specific application example of the present invention.

[0019] Figure 4 This is a schematic diagram of the overall operation of image processing in a specific application example of the present invention. Detailed Implementation

[0020] The present application will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present application, but do not limit the present application in any way. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present application, and these all fall within the protection scope of the present application. Parts not described in detail in the following embodiments can be implemented using existing technology.

[0021] In existing technologies, techniques that utilize image priors from pre-trained diffusion models to perform tasks such as image editing, image reconstruction, super-resolution, and deblurring typically suffer from drawbacks such as high training costs, strong task specificity, significant accumulation of local sampling errors, difficulty in correcting global biases, and unstable recovery quality when faced with highly ill-posed inverse problems. Existing technologies lack an image processing solution that can accommodate multiple tasks within a unified framework, balance global correction with task-specific constraints, and possess zero-sample training-free deployment capabilities.

[0022] To address the aforementioned issues, one embodiment of the present invention provides an image processing method based on global correction posterior sampling. This method constructs a unified image processing technology framework based on diffusion model priors, posterior sampling, global correction, and task constraints. It uses a pre-trained diffusion model as the image prior and, within the unified posterior sampling framework, combines global correction and task-specific constraints to perform image reconstruction, editing, super-resolution restoration, or deblurring. This achieves multi-task compatibility within a unified framework, balances global correction and task-specific constraints, and possesses zero-sample, training-free deployment capabilities. It is applicable to various image processing scenarios such as image reconstruction, image editing, super-resolution reconstruction, and deblurring.

[0023] Specifically, such as Figure 1 As shown, the image processing method based on global correction posterior sampling provided in this embodiment may include: S1, Construct the input data and task conditions; where the input data includes: input image and task type; S2, perform latent variable initialization and single-step noise addition on the input image to obtain noisy latent variables; S3 uses noisy latent variables as input to a pre-trained diffusion model and, in conjunction with the task type, estimates the current clean sample. S4. Using the current clean samples, perform a correction update oriented towards the global error, and apply specific constraints to the correction update process for different task types and their task conditions to obtain the corrected estimation results. S5. Using the corrected estimation results, generate the previous time step sample and input it as a new noisy latent variable into the pre-trained diffusion model for iteration. Repeat S3~S5 until the preset iteration stopping condition is reached to obtain the target output latent variable. S6. Based on the obtained target output latent variables, obtain the image processing results corresponding to the task type.

[0024] In some preferred embodiments, the above-mentioned S1, constructing the input data and task conditions, may further include: Obtain the input image to be processed and the task type, and construct conditional information corresponding to the task type; where: The task type must include at least one of the following: image reconstruction, image editing, super-resolution, and deblurring; For image editing tasks, the corresponding condition information includes: source cue words, target cue words, and / or masked observation images; For image reconstruction tasks, the corresponding condition information includes: masked observation image; For super-resolution tasks, the corresponding conditional information includes: low-resolution observation images and their degradation operators; For the deblurring task, the conditional information includes: the blurred observation image and the blur kernel parameters to be estimated.

[0025] In some preferred embodiments, S2 above, which initializes latent variables and adds noise in a single step to the input image to obtain noisy latent variables, may further include: The input image is encoded into the latent space to obtain the initial latent variables. ; According to the preset noise scheduling, the initial latent variables By directly sampling the target noise time, the noisy latent variable is obtained. .

[0026] Unlike the traditional stepwise inversion method, this step uses direct random noise addition instead of multi-step DDIM inversion to reduce inference overhead and avoid the accumulation of inversion errors.

[0027] In some preferred embodiments, S3 above, which uses noisy latent variables as input to the pre-trained diffusion model and estimates the current clean samples in conjunction with the task type, may further include: S31, Based on the task type, obtain the pre-trained diffusion model or the distillation solver system corresponding to the pre-trained diffusion model; S32, input the noisy latent variables into the pre-trained diffusion model or distillation solver system to obtain the corresponding clean sample estimates.

[0028] In a specific application example, for image editing and image reconstruction tasks, solvers such as LCM (Latent Consistency Model) or DDIM (Denoising Diffusion Implicit Model) can be used to obtain an initial estimate that is more consistent with the target text conditions in fewer steps. For super-resolution and deblurring tasks, solvers such as flow Matching and meanflow can be used.

[0029] In some preferred embodiments, the above-described S4, which utilizes the current clean sample to perform a correction update oriented towards global error, may further include: S41, using the input latent variable weighted injection method to retain the key layout and background features of the source image, combines the current clean sample with the input image features, observation consistency term and stochastic dynamics term to perform a global search for target estimation and construct a global correction update; S42, based on global correction update, obtain the corrected latent variables; S43, based on the corrected latent variables, reconstructs the sampling distribution of the previous time step and continues iterating until the final corrected estimation result is output.

[0030] In some preferred embodiments, the above-mentioned S41, the input latent variable weighted injection method, may further include: The input latent variable weighted injection method is represented as follows: In the formula, This represents the latent variable representation of the image obtained after weighted injection; This represents the input feature injection weights, used to strike a balance between the ability to change the semantics of the target and the ability to preserve the structure of the source image; This represents a clean sample obtained through diffusion model prediction. Represents the initial latent variable; The current clean sample is combined with the input image features, the observation consistency term, and the stochastic dynamics term, and expressed as: In the formula, The latent variable representation of the image for the next optimization step; The latent variable representation of the image at the current optimization step, i.e., the current clean sample; This represents the latent variable representation used to describe the input real image; Indicates the intensity coefficient as it changes over time; superscript Indicates a cyclical indicator; This represents a constant used to control the strength of the optimization term; This represents the observation operator corresponding to the task; This represents the observations used to describe the observation consistency term; Indicates the step size; Indicates random noise; This represents a stochastic dynamics term. By introducing stochastic dynamics, the probability of getting trapped in local optima can be reduced, and the ability to explore the global feasible solution space can be enhanced.

[0031] In some preferred embodiments, the above-mentioned S4, which imposes specific constraints on the correction and update process for different task types and their task conditions, may further include: For different tasks, specific constraints are constructed to optimize the global correction and update process; among which: For image reconstruction and image editing tasks, we construct observation terms based on mask matrices so that the update process can simultaneously preserve the source image and transfer the target semantics. For super-resolution tasks, data consistency and null space constraints are constructed, and the observation-satisfied parts and recoverable details are processed separately to ensure data consistency and limit the solution space. For the deblurring task, independent priors are established for the clear image and the blur kernel, and joint optimization is performed on the two to improve the recovery stability in complex inverse problems.

[0032] In some preferred embodiments, for image reconstruction and image editing tasks, constructing observations based on the mask matrix may further include: In the formula, N(·) represents a Gaussian distribution. This represents a mask matrix composed of unit vectors. For observation Dimensions The dimension representing the latent variables of the image. For the size of the variance, It is an identity matrix.

[0033] In some preferred embodiments, for super-resolution tasks, constructing data consistency and null space constraints may further include: Data consistency and null space constraints are expressed as: In the formula, This represents an image predicted by diffusion priors; Represents the degenerate operator; Indicates the pseudo-inverse of the degenerate operator; This represents the high-resolution prior estimate given by the diffusion model; The identity matrix is ​​used. By first fixing the observation consensus portion and then performing global correction in the null space, the observation satisfaction portion and recoverable detail portion are processed separately, which can improve the stability of high-magnification super-resolution restoration.

[0034] In some preferred embodiments, for the deblurring task, independent priors are established for the sharp image and the blur kernel, and joint optimization is performed on the two. This may further include: In the deblurring task, the blurred observation image in the conditional information is represented as: In the formula, Represents a blurred observation image; This indicates that a clear image needs to be restored. Indicates the fuzzy kernel to be estimated. Indicates noise.

[0035] By separately adjusting the image variables and fuzzy kernel variables By establishing diffusion priors and performing joint stochastic dynamics corrections, blind defuzzification can be achieved in the case of unknown fuzzy operators.

[0036] In some preferred embodiments, S5 above, which generates the sample from the previous time moment using the corrected estimation result, may further include: Based on the corrected clean sample estimate, the sampling distribution of the previous time step is reconstructed and the state of the previous time step is generated, thus obtaining the new noisy latent variable, which serves as the input for the next sampling step of the diffusion model. Repeat steps S3 through S5 until the termination time is reached to obtain a clean image estimate without noise, which is used as the target output latent variable.

[0037] In some preferred embodiments, the above-mentioned S6, which outputs a result image corresponding to the task type, may further include: The target output latent variables are mapped back to the image space through a decoder, outputting a result image corresponding to the task type. This result image can be a reconstructed image, an edited image, a super-resolution image, or a deblurred image.

[0038] Based on the same inventive concept, another embodiment of the present invention provides an image processing system based on global correction posterior sampling.

[0039] Specifically, such as Figure 2 As shown, the image processing system based on global correction posterior sampling provided in this embodiment may include: The data construction module is used to construct the input data and task conditions; the input data includes: the input image and the task type. The noise-adding module is used to initialize latent variables and add noise to the input image in one step to obtain initial noisy latent variables, which are then sent to the diffusion model estimation module as initial input latent variables. The diffusion model estimation module continuously takes the received noisy latent variables as input to the pre-trained diffusion model and estimates the current clean samples in combination with the task type until a preset stopping condition is reached to obtain the target output latent variables. The global correction and update module is used to continuously utilize the current clean samples to perform correction and update oriented towards global errors. It also applies specific constraints to the correction and update process for different task types and their conditions to obtain the corrected estimation results until the preset stopping condition is reached. The iterative generation module continuously uses the corrected estimation results to generate samples from the previous time step and inputs them as new noise latent variables into the diffusion model estimation module until a preset stopping condition is reached to obtain the target output latent variable. The image output module is used to obtain image processing results corresponding to the task type based on the obtained target output latent variables.

[0040] In some preferred embodiments, the data construction module described above, which acquires input and constructs task conditions, may further include: The data acquisition unit is used to acquire the input image to be processed and the task type; wherein the task type includes at least one of image reconstruction, image editing, super-resolution and deblurring.

[0041] Conditional information construction unit; this unit is used to construct corresponding conditional information based on task type; wherein: For image editing tasks, the conditional information includes source cue words and target cue words; For image reconstruction or editing tasks, mask observation can also be constructed; For super-resolution tasks, the conditional information includes low-resolution observation images and their degradation operators; For the deblurring task, the conditional information includes the blurred observation image and the blur kernel parameters to be estimated.

[0042] In some preferred embodiments, the above-mentioned noise-adding module, which performs latent variable initialization and single-step noise addition, may further include: The initial latent variable construction unit is used to encode the input image into the latent space to obtain the initial latent variables. ; The noisy latent variable construction unit is used to directly sample latent variables at the target noise time according to a preset noise schedule, thereby obtaining noisy latent variables. .

[0043] In some preferred embodiments, the above-mentioned diffusion model estimation module, which estimates the current clean sample using a pre-trained diffusion model, may further include: The diffusion model acquisition unit is used to acquire a pre-trained diffusion model or a distillation solver system corresponding to the pre-trained diffusion model, based on the task type. The latent variable estimation unit is used to input noisy latent variables into a pre-trained diffusion model or its distillation solver system to obtain corresponding clean sample estimates.

[0044] In some preferred embodiments, the global correction and update module described above may further include: A global error correction and update execution unit is introduced. This unit combines the current estimation result with input image features, observation consistency terms, and stochastic dynamics terms to construct a global correction update. A weighted injection method of latent variables is used to preserve key layout and background features of the source image, and Langevin stochastic dynamics is combined to perform a global search on the target estimate, thereby correcting historical accumulated errors that are difficult to correct by relying solely on updates in adjacent steps. Apply specific constraint units according to different tasks. For image reconstruction and image editing tasks, this unit is used to construct observation terms based on mask matrix so that the update process can simultaneously take into account the preservation of source image and the transfer of target semantics. For super-resolution tasks, data consistency and null space constraints are constructed to separate the observation satisfaction part from the recoverable details part. For deblurring tasks, independent priors are established for the clear image and the blur kernel respectively, and joint optimization is performed on the two to improve the recovery stability in complex inverse problems.

[0045] In the global correction update module, the weighted injection of latent variables can be represented as: in, Weights are injected into the input features to strike a balance between the ability to modify the target semantics and preserve the structure of the source image. The global correction update employs a joint update approach including latent variable preservation terms, observation consistency terms, and random perturbation terms, for example: in, This represents the observation operator corresponding to the task. Representing the observed quantity, Indicates step size, This represents random noise. By introducing stochastic dynamics, the probability of getting trapped in local optima can be reduced, and the ability to explore the global feasible solution space can be enhanced.

[0046] In super-resolution tasks, to ensure data consistency and limit the solution space, the following null space constraint can be used: in, This represents the pseudoinverse of the degenerate operator. This represents the high-resolution prior estimate given by the diffusion model. By first fixing the observational consensus portion and then performing global correction in the null space, the stability of high-magnification super-resolution restoration can be improved.

[0047] In the deblurring task, fuzzy observations are represented as: in, This indicates that a clear image needs to be restored. Indicates the fuzzy kernel to be estimated. Representing noise. This is achieved by separately processing image variables. and fuzzy kernel variables By establishing diffusion priors and performing joint stochastic dynamics corrections, blind defuzzification can be achieved even with unknown fuzzy operators.

[0048] In some preferred embodiments, the above-mentioned iterative generation module may further include: The iterative unit generates samples from the previous time step using the corrected estimate and iterates. Specifically, based on the corrected clean sample estimate, the sampling distribution of the previous time step is reconstructed and the state of the previous time step is generated, rather than simply relying on the current state to locally repair the previous state. Then, the steps of estimating the current clean sample using the pre-trained diffusion model, performing correction updates for global errors, applying specific constraints according to different tasks, and generating samples from the previous time step using the corrected estimate and iterating are repeated until the termination time is reached to obtain the target output latent variable. The processing result output unit maps the final latent variables back to the image space through the decoder and outputs the result image corresponding to the task type; the result image can be a reconstructed image, an edited image, a super-resolution image, or a deblurred image.

[0049] It should be noted that the steps in the method provided by the present invention can be implemented using the corresponding components in the system. Those skilled in the art can refer to the technical solution of the system to implement the steps of the method, and can also refer to the technical solution of the method to implement the composition of the system. That is, the embodiments in the system and the embodiments in the method can be understood as preferred examples of each other, which will not be elaborated here.

[0050] The following detailed description, using specific application examples, further illustrates the application effects of the technical solutions provided in the above embodiments of the present invention in specific image processing tasks. These specific application examples are intended to illustrate the application of the technical solutions of the present invention in specific tasks, and are not intended to limit the scope of protection of the present invention.

[0051] like Figure 3 The figure shows a schematic diagram of the posterior sampling process based on a global optimization strategy for different tasks. Figure 3 In this process, noise is first added to the input image. Then, optimization strategies such as Langevin dynamics are used to iteratively optimize the clean image predicted by the diffusion prior. Since recovering a high-quality image using diffusion in the latent space is an ill-conditioned problem, further solution space shrinkage is needed to improve the generation quality, ultimately outputting a high-quality recovered image. Figure 3 As can be seen, this invention enables simultaneous support for text-driven editing and low-level recovery within a unified posterior sampling framework.

[0052] like Figure 4 The diagram shown is a schematic representation of the overall image processing workflow. Figure 4 In this process, noise is first added to the damaged image, and then a clean, high-quality image is predicted using diffusion prior. After observing the deviation between the observed image and the input image, a posterior sampling algorithm is used to estimate the image for correction, ultimately resulting in a high-quality, unbiased image. .

[0053] based on Figure 3 and Figure 4 The flowchart shown further explains the implementation process for a specific task.

[0054] Task 1: Image Editing Task In this image processing task, the user provides a source image and target editing text prompts, such as changing object attributes, replacing local semantics, or adding or deleting specified text descriptions. First, the source image is encoded into a latent space to obtain latent variables. Noisy latent variables are obtained by direct sampling according to the preset noise scheduling. .

[0055] Subsequently, a pre-trained diffusion model solver was used to estimate clean latent variables under the target text conditions. And construct a mask observation term so that the unedited regions in the input image still participate in the observation consistency update; at the same time, through weights A portion of the latent variables from the source image is re-injected into the target estimation to preserve the background, layout, and subject outline.

[0056] During the global correction phase, Langevin stochastic dynamics updates are performed based on the latent variable preservation term and the mask observation consistency term to obtain the corrected result. Based on The sampling distribution from the previous moment is reconstructed and iterated until the final edited result is output. This implementation method is suitable for scenarios such as attribute editing, long text editing, and content replacement.

[0057] Task 2: Super-resolution Task In this task, the input is a low-resolution image. It is formed by high-definition images through degradation operators And noise perturbation generation. First, a high-resolution prior estimate is obtained through a pre-trained diffusion model. Then, a data consistency term is constructed using pseudo-retrospective projection to separate the observation satisfaction part from the null space detail recovery part.

[0058] For the part of the observation that satisfies the condition, through To ensure consistency with low-resolution input; for the null space part, detail recovery is performed by combining stochastic dynamics correction under the guidance of diffusion prior, thereby avoiding the problem that traditional stepwise local correction schemes are unable to escape pseudo-solutions.

[0059] For this task, the technical solution provided by this invention can be used for image super-resolution reconstruction at 4x and other magnifications, and is particularly suitable for detail texture restoration, complex background restoration and structure preservation in multi-object scenes.

[0060] Task 3: Blind Deblurring Task In this task, the input is a blurred image. The fuzzy process can be represented as Among them, clear images and fuzzy kernel Both are unknown. Independent diffusion priors are established for the image variables and the blur kernel variables, and their corresponding noisy states are initialized.

[0061] Next, prior estimation and stochastic dynamics correction are performed in two variable spaces: on the one hand, the sharp image is corrected based on the current blur kernel estimate; on the other hand, the blur kernel is corrected inversely based on the current sharp image estimate. Through alternating or joint optimization, the image and blur kernel simultaneously tend to satisfy observation consistency and their respective prior constraints.

[0062] Compared to existing solutions that only perform local repairs in adjacent time steps, the technical solution provided by this invention can more effectively correct global errors and improve the recovery quality in complex motion blur or Gaussian blur scenarios when the blur operator is unknown.

[0063] Based on the same technical concept, in other embodiments of the present invention, a computer terminal is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it can be used to execute the method of any of the above embodiments of the present invention, or to run the system of any of the above embodiments of the present invention.

[0064] Furthermore, the computer terminal can be a GPU server, an edge computing device, or a terminal device with AI inference capabilities; when deployed on the terminal side, one or more processors can call program instructions in the storage medium to complete the above steps.

[0065] Optionally, the memory is used to store programs; the memory may include volatile memory, such as random-access memory (RAM), such as static random-access memory (SRAM), double data rate synchronous dynamic random-access memory (DDR SDRAM), etc.; the memory may also include non-volatile memory, such as flash memory. The memory is used to store computer programs (such as application programs and functional modules that implement the above methods), computer instructions, etc., and the aforementioned computer programs and computer instructions can be partitioned and stored in one or more memories. Furthermore, the aforementioned computer programs, computer instructions, data, etc., can be accessed by the processor.

[0066] A processor is used to execute computer programs stored in memory to implement the various steps of the methods or various modules of the systems involved in the above embodiments. For details, please refer to the relevant descriptions in the preceding method and system embodiments.

[0067] The processor and memory can be separate structures or integrated structures. When the processor and memory are separate structures, they can be coupled together via a bus.

[0068] Based on the same technical concept, in other embodiments of the present invention, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, can be used to perform the method of any of the above embodiments of the present invention, or to run the system of any of the above embodiments of the present invention.

[0069] Computer-readable media include computer storage media and communication media, wherein communication media include any medium that facilitates the transfer of computer programs from one place to another. Storage media can be any available medium accessible to a general-purpose or special-purpose computer. An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and storage medium can reside in an ASIC. Alternatively, the ASIC can reside in a user device. Of course, the processor and storage medium can also exist as discrete components in a communication device.

[0070] This invention employs a unified posterior sampling framework and a pre-trained diffusion model prior, eliminating the need to retrain a dedicated network for each type of image processing task. It can support multiple tasks such as image reconstruction, image editing, super-resolution, and deblurring under zero-shot conditions, achieving greater versatility and lower deployment costs. It solves the problem of how to support multiple image processing tasks such as image reconstruction, image editing, super-resolution, and deblurring simultaneously without retraining the base model.

[0071] The image processing method and system based on global correction posterior sampling provided in the above embodiments of the present invention adopt a global correction strategy based on input feature injection, observation consistency, and stochastic dynamics. This strategy can directly correct historical accumulated errors during the sampling process, reduce background drift and semantic shift caused by unconditional branching, and achieve a better balance between preserving key features of the source image and satisfying target constraints. Task-specific constraints are designed for highly ill-posed inverse problems, including a mask observation mechanism for image editing, a data consistency and null space constraint mechanism for super-resolution, and an image-blur kernel joint optimization mechanism for deblurring. Therefore, it can improve the restoration quality, structural stability, and robustness under complex degradation conditions. By using single-step noise addition instead of multi-step inversion, and combining a few-step solver with global correction updates, the present invention maintains high quality while having good inference efficiency, making it suitable for deployment in scenarios such as scientific research, industrial image enhancement, content production, and human-computer interactive image editing. Through the combination of a unified training-free framework, global correction, and task-specific constraints, the present invention can solve the problems of high training costs, severe local error accumulation, and insufficient adaptability to complex inverse problems in existing technologies.

[0072] Any matters not covered in the above embodiments of the present invention are well-known in the art.

[0073] The foregoing has described some specific embodiments of this application. It should be understood that this application is not limited to the specific embodiments described above, and those skilled in the art can make various modifications or variations within the scope of the claims, which do not affect the substantive content of this application. The above-described preferred features can be used in any combination without conflict.

Claims

1. An image processing method based on global correction posterior sampling, characterized in that, include: Construct input data and task conditions; wherein, the input data includes: input image and task type; The input image is initialized with latent variables and noise is added in one step to obtain initial noisy latent variables; Based on the initial noisy latent variables, the following iterative steps are performed: s1, using the initial noisy latent variables as input to the pre-trained diffusion model, and combining the task type, estimate the current clean sample; s2, using the current clean sample, perform a correction update oriented towards global error, and apply specific constraints to the correction update process for different task types and their task conditions to obtain the corrected estimation result; s3, using the corrected estimation result, generate the previous time sample as a new noisy latent variable and repeat s1~s3 until the preset stopping condition is met; After the iterative steps are completed, the image processing result corresponding to the task type is obtained.

2. The image processing method based on global correction posterior sampling according to claim 1, characterized in that, The input data and task conditions for construction include: Obtain the input image to be processed and the task type, and construct conditional information corresponding to the task type; where: The task type includes at least one of image reconstruction, image editing, super-resolution, and deblurring; For image editing tasks, the corresponding condition information includes: source cue words, target cue words, and / or masked observation images; For image reconstruction tasks, the corresponding condition information includes: masked observation image; For super-resolution tasks, the corresponding conditional information includes: low-resolution observation images and their degradation operators; For the deblurring task, the conditional information includes: the blurred observation image and the blur kernel parameters to be estimated.

3. The image processing method based on global correction posterior sampling according to claim 1, characterized in that, The step of initializing latent variables and adding noise in a single step to the input image to obtain initial noisy latent variables includes: The input image is encoded into the latent space to obtain the initial latent variables. ; According to the preset noise scheduling, the initial latent variables are... The initial noisy latent variable is obtained by directly sampling the target noise time. .

4. The image processing method based on global correction posterior sampling according to claim 1, characterized in that, The step of using the initial noisy latent variables as input to the pre-trained diffusion model and, in conjunction with the task type, estimating the current clean samples includes: Based on the task type, obtain the pre-trained diffusion model or the distillation solver system corresponding to the pre-trained diffusion model; The initial noisy latent variables are input into the pre-trained diffusion model or the distillation solver system to obtain the corresponding clean sample estimates.

5. The image processing method based on global correction posterior sampling according to claim 1, characterized in that, The step of performing a global error correction update using the current clean sample includes: The key layout and background features of the source image are preserved by using a weighted injection method of latent variables. The current clean sample is combined with the features of the input image, the observation consistency term and the stochastic dynamics term to perform a global search for the target estimation and construct a global correction update. Based on the global correction update, the corrected latent variables are obtained; Based on the corrected latent variables, the sampling distribution of the previous time step is reconstructed and the iteration continues until the final corrected estimation result is output. The specific constraints imposed on the correction and update process for different task types and their conditions include: For different tasks, specific constraints are constructed to optimize the global correction and update process; wherein: For image reconstruction and image editing tasks, we construct observation terms based on mask matrices so that the update process can simultaneously preserve the source image and transfer the target semantics. For super-resolution tasks, data consistency and null space constraints are constructed, and the observation-satisfied parts and recoverable details are processed separately to ensure data consistency and limit the solution space. For the deblurring task, independent priors are established for the clear image and the blur kernel, and joint optimization is performed on the two to improve the recovery stability.

6. The image processing method based on global correction posterior sampling according to claim 5, characterized in that, The input latent variable weighted injection method includes: The weighted injection method for latent variables is expressed as follows: ， In the formula, The latent variable representation of the image obtained after weighted injection; This represents the input feature injection weights, used to strike a balance between the ability to change the target semantics and the ability to preserve the structure of the source image; This represents a clean sample obtained through diffusion model prediction. Represents the initial latent variable; The combination of the current clean sample with the input image features, the observation consistency term, and the stochastic dynamics term is expressed as: ， In the formula, The latent variable representation of the image for the next optimization step; The latent variable representation of the image at the current optimization step, i.e., the current clean sample; This represents the latent variable representation used to describe the input real image; Indicates the intensity coefficient as it changes over time; superscript Indicates a cyclical indicator; This represents a constant used to control the strength of the optimization term; This represents the observation operator corresponding to the task; This represents the observations used to describe the observation consistency term; Indicates the step size; Indicates random noise; It is a stochastic dynamic term; For image reconstruction and image editing tasks, observations based on the mask matrix are constructed, including: ， In the formula, N(·) represents a Gaussian distribution. This represents a mask matrix composed of unit vectors. For observation Dimensions The dimension representing the latent variables of the image. For the magnitude of variance, It is the identity matrix; For super-resolution tasks, data consistency and null space constraints are constructed as follows: ， In the formula, This represents the current clean sample predicted by the diffusion model; Represents a degenerate operator; Indicates the pseudo-inverse of the degenerate operator; This represents the high-resolution prior estimate given by the diffusion model; The identity matrix is ​​used; by first fixing the observation-consistent part and then performing global correction in the null space, the observation-satisfied part and the recoverable detail part are separated and processed. For the deblurring task, independent priors are established for the sharp image and the blur kernel, and joint optimization is performed on the two, including: In the deblurring task, the blurred observation image in the conditional information is represented as: ， In the formula, Represents a blurred observation image; This indicates that a clear image needs to be restored. Indicates the fuzzy kernel to be estimated. Indicates noise; By separately adjusting the image variables and fuzzy kernel variables A diffusion prior is established and a joint stochastic dynamics correction is performed to blindly defuzzify the fuzzy operator.

7. The image processing method based on global correction posterior sampling according to claim 1, characterized in that, The step of generating the previous time-time sample using the corrected estimation result includes: Based on the corrected clean sample estimate, the sampling distribution of the previous time step is reconstructed and the state of the previous time step is generated, thus obtaining a new noisy latent variable, which serves as the input for the next sampling step of the diffusion model. Repeat the iterative steps until the preset termination time is reached to obtain the final clean image estimate without noise, which is used as the target output latent variable; The target output latent variables are mapped back to the image space through the decoder, and the resulting image corresponding to the task type is output.

8. An image processing system based on global correction posterior sampling, characterized in that, include: A data construction module is used to construct input data and task conditions; wherein, the input data includes: input image and task type; The noise-adding module is used to initialize latent variables and add noise in one step to the input image to obtain initial noisy latent variables, which are then sent to the diffusion model estimation module as initial input latent variables. The diffusion model estimation module continuously uses the received noisy latent variables as input to the pre-trained diffusion model and, in conjunction with the task type, estimates the current clean sample until a preset stopping condition is reached to obtain the target output latent variable. The global correction and update module is used to continuously use the current clean samples to perform correction and update for global errors, and to apply specific constraints to the correction and update process for different task types and task conditions to obtain the corrected estimation results until the preset stopping condition is reached. An iterative generation module is used to continuously generate samples from the previous time step using the corrected estimation results, and input them as new noise latent variables into the diffusion model estimation module until a preset stopping condition is reached to obtain the target output latent variable. The image output module is used to obtain image processing results corresponding to the task type based on the obtained target output latent variables.

9. A computer terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it can be used to perform the method of any one of claims 1-7, or to run the system of claim 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program can be used to perform the method of any one of claims 1-7, or to run the system of claim 8.

Images