A text-guided image editing method, system, device and medium for scene layout occlusion perception

Abstract

A scene layout occlusion-aware text-guided image editing method, system, device and medium, the method comprising: preprocessing the occlusion layout dataset and the image editing dataset; constructing an image consistency editing model based on occlusion layout awareness for image editing in an occluded scene; including an occlusion layout generation module and an instruction layout awareness diffusion module; training the image consistency editing model based on occlusion layout awareness; loading the trained image consistency editing model based on occlusion layout awareness to perform image editing; the invention generates occlusion layout features through the occlusion layout generation module, decouples the occlusion layout features through the instruction layout awareness diffusion module and injects them into the diffusion network, and through the guidance of the occlusion layout, the model can perceive the geometric structure of the occluded area, effectively eliminate the flicker or block artifacts caused by discrete representation, and avoid structure distortion and texture breakage artifacts in the revealed area.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of computer vision and computer graphics, and specifically relates to a text-guided image editing method, system, device and medium for scene layout occlusion perception. Background Technology

[0002] Text-based image editing technology has broad application prospects in content creation, visual design, and virtual reality. This technology aims to achieve controllable editing of specific areas without compromising overall visual consistency, involving deep collaboration between scene semantic perception, spatial structure understanding, and object interaction relationships. In content creation, this technology enables rapid replacement and attribute modification of specific objects, improving creation efficiency; in visual design, it allows for precise adjustment of scene layout according to design requirements, shortening iteration cycles; and in virtual interaction, it can generate dynamic scene changes that conform to physical logic, enhancing user experience. However, its implementation faces numerous challenges, such as balancing the semantic accuracy of the editing area with the visual stability of the non-editing area during editing operations, adapting to complex object occlusion and spatial hierarchy in real-world scenes, and ensuring the unity and coordination of lighting, texture, and other relationships.

[0003] Existing image editing methods have made some progress in simple scenes with weak occlusion or clear targets, but significant technical bottlenecks remain for editing tasks under complex occlusion conditions. Most current mainstream methods lack explicit perception of the overall scene structure and occlusion relationships, making it difficult to meet the high-quality requirements of physical state reconfiguration (such as object removal or movement) in practical applications. This is especially true when dealing with the re-revealing of occluded areas caused by editing operations, where they exhibit a clear lack of reasoning. This blind reliance on potential priors easily leads to structural distortions, texture breaks, and boundary artifacts in newly revealed areas, severely compromising the physical plausibility and visual realism of the scene, and failing to meet the refined editing requirements that conform to human cognition.

[0004] Patent application CN120931769 discloses an image editing method and related apparatus, belonging to the field of artificial intelligence technology. This method utilizes a finely tuned multimodal large language model to infer the visual feature information of the target image. This feature information is then used in a diffusion model to guide image reconstruction and editing, significantly improving image editing performance under complex text commands. However, this method primarily focuses on enhancing the semantic understanding and visual feature conversion capabilities of the edited text. When handling editing tasks involving occlusion, the model struggles to logically complete the missing content in the unoccluded area, easily generating artifacts such as structural distortion and texture breaks in the re-revealed occluded area. This makes it difficult to meet the requirement of maintaining spatial logic and physical realism in occluded scenes.

[0005] Patent application CN121074160A discloses an image editing method, apparatus, device, medium, and product, belonging to the field of computer and artificial intelligence technology. This method transforms input images and text instructions into a sequence of discrete units that the model can process at least once. It then utilizes the routing mechanism of a multimodal large language model to generate a predicted sequence and combines a diffusion model to decode latent variables to reconstruct the image. This effectively integrates the semantic reasoning capabilities of the multimodal large language model with the high-quality generation capabilities of the diffusion model, achieving end-to-end automated image editing. However, this method relies on discretized visual primitives to represent continuous images. Its vector quantization process inevitably leads to the loss of high-frequency texture details and edge blurring, making it difficult to guarantee pixel-level precise alignment of de-occluded areas and easily producing flickering or blocky artifacts. Summary of the Invention

[0006] To overcome the shortcomings of the prior art, the present invention aims to provide a text-guided image editing method, system, device, and medium for scene layout occlusion perception. The method generates occlusion layout features through an occlusion layout generation module, decouples the occlusion layout features through an instruction layout perception diffusion module, and injects them into its own diffusion network. Guided by the occlusion layout, the model can perceive the geometric structure of the occluded area, effectively eliminating flickering or block artifacts caused by discretization representation, and avoiding structural distortion and texture breakage artifacts in the occluded re-revealed area.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A text-guided image editing method with scene layout occlusion awareness includes the following steps: Step 1, preprocessing the occlusion layout dataset and the image editing dataset; Step 2: Construct an image consistency editing model based on occlusion layout awareness for image editing in occlusion scenarios; including an occlusion layout generation module and an instruction layout awareness diffusion module; Step 3: Use the occlusion layout dataset and image editing dataset preprocessed in Step 1 to train the image consistency editing model based on occlusion layout awareness constructed in Step 2; Step 4: Load the occlusion layout-aware image consistency editing model trained in Step 3 and perform image editing.

[0008] The specific method of step 1 includes: Step 1.1: Construct the occlusion layout dataset and image editing dataset and determine the annotation information; The occlusion layout dataset includes the original RGB image and the corresponding occlusion scene annotations. The annotation information of the occlusion scene includes the visible mask of the object, the full-view mask, and the relative depth relationship between the objects. The image editing dataset includes the original RGB image, natural language text instructions, and the corresponding edited real image; Step 1.2: Perform standardization preprocessing on the occlusion layout dataset and image editing dataset constructed in Step 1.1 using an image processing library; Using an image processing library, the original RGB images from the occlusion layout dataset constructed in step 1.1 and the original RGB images and edited real images from the image editing dataset are read. The image resolution is uniformly adjusted using a bilinear interpolation algorithm and converted into floating-point Tensor format. At the same time, the three color channels of the image are normalized using specified mean and standard deviation as distribution parameters to map the range of image pixel values ​​to fixed values, thus obtaining the preprocessed occlusion layout dataset and image editing dataset. Step 1.3: Use the annotation parsing tool to perform feature processing on the annotation information; For mask-type data, the visible mask and full-view mask of the objects in the annotation information of the occlusion scene in the occlusion layout dataset constructed in step 1.1 are adjusted to a resolution that matches the model feature map and binarized to be converted into floating-point Tensors to obtain the feature-processed visible mask and full-view mask. For depth data, the relative depth relationship between objects is transformed into an antisymmetric matrix, and the numerical values ​​are standardized to represent the front and back occlusion order between objects, thus obtaining the depth adjacency matrix. For the natural language text instructions in the image editing dataset constructed in step 1.1, a pre-trained word embedding device is used to transform the natural language text instructions into high-dimensional word embedding vectors, thus obtaining the text instruction word embedding vectors.

[0009] The specific method for step 2 includes: Step 2.1, construct the occlusion layout generation module; Based on the Transformer architecture, an occlusion layout generation module is constructed. The occlusion layout generation module is used to extract the visual features and spatial relationships of potential objects from the occlusion layout dataset constructed in step 1.1, predict the overall geometric structure and depth order of objects in the scene, and output an occlusion layout containing complete occlusion information. The occlusion layout generation module is composed of an occlusion layout decoding network and a layout prediction output network connected in sequence. Step 2.1.1: Initialize image features and query vectors. The occlusion layout generation module receives image features extracted by the pre-trained Swin Transformer network and a set of randomly initialized learnable query vectors to represent potential object instances in the scene. Step 2.1.2: Construct the occlusion layout decoding network. The occlusion layout decoding network consists of three stacked decoder layers. Each decoder layer consists of an embedding layer, a cross-attention layer, a layer normalization layer, a cross-attention layer, a layer normalization layer, a feedforward network layer, a depth-sensing attention layer, a layer normalization layer, a self-attention layer, a layer normalization layer, and a feedforward network layer. The input is the image features and learnable query vector initialized in step 2.1.1. First, the image features and learnable query vectors initialized in step 2.1.1 are input into the embedding layer for feature mapping, resulting in vectors mapped from the image features and vectors mapped from the learnable query vectors. Then, a cross-attention layer is used to interactively compute the vector mapped from the learnable query vectors as the query term and the vector mapped from the image features as the key term, aggregating the image's appearance information. The output features are then processed by layer normalization. Next, the features output after layer normalization are again fed into the cross-attention layer and layer normalization, and then fused through a feedforward network layer. Subsequently, the fused features are input into a depth-aware attention layer, where depth constraints are embedded during feature interaction to correct spatial geometric representations. After layer normalization, a self-attention layer and layer normalization are used to establish global dependencies between features. Finally, after nonlinear mapping by the feedforward network layer, a feature sequence containing the geometric features of the occluded region is output. Step 2.1.3: Construct the layout prediction output network. The layout prediction output network is connected to the end of the occlusion layout decoding network. It consists of a linear layer and a multilayer perceptron in parallel. The feature sequence output from Step 2.1.2 is input into the two branches respectively. The linear layer is used to predict the depth level of the object, and the multilayer perceptron is used to generate the full-view mask of the object. The two together constitute the occlusion layout features, which are used as the output of the occlusion layout generation module. Step 2.2, construct the instruction layout awareness diffusion module; The instruction layout-aware diffusion module is used to semantically align and decouple the occlusion layout features output by the occlusion layout generation module in step 2.1 with the natural language text instructions in the image editing dataset constructed in step 1.1 to obtain conditional features, and then inject the processed conditional features into the generation process; The instruction layout-aware diffusion module is composed of an instruction layout alignment network, a conditional feature decoupling network and an image editing network connected in sequence. Step 2.2.1: Construct the instruction layout alignment network. The instruction layout alignment network, as a feature interaction and splitting model, consists of an embedding layer, a cross-attention layer, a multilayer perceptron, and a feature splitting layer. First, the natural language text instructions in the image editing dataset constructed in Step 1.1 are processed through the embedding layer to obtain text embeddings. Then, the cross-attention layer is used to calculate the correlation between the occlusion layout features output by the occlusion layout generation module in Step 2.1 and the text embeddings. After being mapped to alignment scores by the multilayer perceptron, the feature splitting layer divides the occlusion layout features into an edit set and a consistency set according to the alignment scores, and outputs the decoupled edit set Tokens and consistency set Tokens. Step 2.2.2: Construct a conditional feature decoupling network. As a feature fusion model, the conditional feature decoupling network consists of a concatenation layer, a self-attention layer, a layer normalization layer, and a feedforward network layer. The conditional feature decoupling network concatenates the text embeddings and edit set tokens obtained in Step 2.2.1 in a dimensional manner. Through deep processing by the self-attention layer, the layer normalization layer, and the feedforward network layer, it generates strong conditional features that fuse editing intent and structural priors. Step 2.2.3: Construct an image editing network. The image editing network is based on the Stable Diffusion v1.5 architecture and includes a diffusion network. The diffusion network receives the original RGB images from the image editing dataset constructed in Step 1.1 as input, the text embeddings obtained in Step 2.2.1, the consistency set tokens generated in Step 2.2.1, and the strong conditional features generated in Step 2.2.2 as mixed conditional features. During the denoising process of the diffusion network, the mixed conditional features are used to guide the generation direction. While preserving the structure of the non-edited region, the target region of the input image is edited, and the edited image is finally output.

[0010] The specific method for step 3 includes: Step 3.1: Configure the training parameters and inputs for the occlusion layout generation module in the occlusion layout-aware image consistency editing model, and perform training: The original RGB images from the occlusion layout dataset preprocessed in step 1.2 and the visible mask after feature processing in step 1.3 are used as inputs to the occlusion layout generation module constructed in step 2.1. The corresponding full-view mask and the depth adjacency matrix after feature processing in step 1.3 are used as the real labels. The occlusion layout generation module is trained iteratively with parameters through backpropagation, and finally the weight file of the trained occlusion layout generation module is obtained. Step 3.2: Configure the training parameters and inputs for the occlusion layout-aware image consistency editing model, and perform training: The original RGB images from the image editing dataset after preprocessing in step 1.2 and the embedded vectors of the text instruction words after feature processing in step 1.3 are used as inputs to the image consistency editing model based on occlusion layout awareness constructed in step 2. The edited real images from the image editing dataset after preprocessing in step 1.2 are used as training targets. The weight file of the occlusion layout generation module trained in step 3.1 is loaded and frozen as the parameters of the occlusion layout generation module in the image consistency editing model based on occlusion layout awareness. The weights of the pre-trained Stable Diffusion v1.5 model are loaded and frozen as the parameters of the diffusion network of the instruction layout awareness diffusion module in the image consistency editing model based on occlusion layout awareness. The parameters are updated only for the instruction layout alignment network constructed in step 2.2.1 and the conditional feature decoupling network constructed in step 2.2.2. Through backpropagation and parameter iteration, the weight file of the trained image consistency editing model based on occlusion layout awareness is finally obtained.

[0011] In step 2.2.3, the specific method for using hybrid conditional features to guide the generation direction during the denoising process of the diffusion network, while preserving the structure of the non-edited region, to edit the target region of the input image and finally output a high-quality scene image includes: Random Gaussian noise is added to the original RGB images in the image editing dataset constructed in step 1.1. The original RGB images in the image editing dataset constructed in step 1.1 with added random Gaussian noise, the mixture conditional features, and the current time step are used as input to the diffusion network. The noise added between the current time step and the previous time step is predicted. This predicted noise is then subtracted from the original RGB image with added random Gaussian noise to obtain the predicted image for the previous time step. The diffusion network is then used again as input to this predicted image, the mixed conditional features, and the next time step to predict the noise added between the current time step and the previous time step, thus obtaining the predicted image for the previous time step again. This process is repeated until the initial time step is reached, resulting in the final edited image.

[0012] During the training process in step 3.1, the model parameters of the occlusion layout generation module in step 3.1 are optimized using a hybrid supervised loss function; the hybrid supervised loss function includes two parts: full-view mask prediction loss and depth ranking loss. For the overall mask prediction loss, a combination of binary cross-entropy loss and Dice loss is used, and the calculation formula is as follows: in, The pixel values ​​of the full-view mask obtained in step 1.3 are the feature-processed values. These are the mask pixel values ​​predicted by the model. Total number of pixels; To address the depth ranking loss, an ordinal depth ranking loss is introduced to supervise the model's correct inference of the object occlusion order. The calculation formula is as follows: in, Representing objects and The predicted depth value; As an indicator variable, if the object In objects The back (which is obscured) is... ,otherwise Depth ranking loss penalizes depth predictions that violate true occlusion relationships through logistic regression. Combining the overall mask prediction loss and the depth ranking loss, the total loss function of the occlusion layout generation module is obtained as follows: in, and These are the preset loss weighting coefficients; During the training process in step 3.2, a noise prediction loss based on latent space is adopted, and the optimization objective is defined as minimizing the mean square error between the predicted noise and the actual added noise: in, For noise sampled from a standard Gaussian distribution, For the noise predicted by the diffusion network, For diffusion time step, Encoding of the image in the latent space, and These are the consensus set Tokens after decoupling in step 2.2.1 and the strongly conditional features of the fusion editing intent and structural priors output in step 2.2.2.

[0013] The specific method for step 4 includes: Step 4.1: Read the weight file of the image consistency editing model based on occlusion layout awareness trained in Step 3, load it into the network structure of the image consistency editing model based on occlusion layout awareness, set the state of the image consistency editing model based on occlusion layout awareness to inference mode, fix all parameters of each network layer, and turn off gradient calculation to meet inference requirements. Step 4.2 involves inputting the original image to be edited and the natural language text instructions into the occlusion layout-aware image consistency editing model to execute the editing process; specifically including: The occlusion layout generation module encodes the features of the original image to be edited. A pre-trained SwingTransformer network extracts the image features of the original image and infers the overall mask and relative depth relationship of objects within the scene, outputting the occlusion layout features. The instruction layout alignment network in the instruction layout perception diffusion module receives the occlusion layout features and the input natural language text instructions. Attention scores are calculated to decouple the occlusion layout features into edit sets (Tokens) and consistency sets (Tokens). The decoupled edit sets (Tokens) and natural language text instructions are deeply fused through a conditional feature decoupling network to generate strong conditional features that fuse the editing intent and structural priors. The diffusion network uses the latent variables of the original image to be edited with added random Gaussian noise as the initial diffusion state. The strong conditional features that fuse the editing intent and structural priors, the consistency sets (Tokens), and the natural language text instructions are injected into the denoising process as guiding signals. Through multi-step noise prediction and gradual restoration of image pixels, an edited image that conforms to the text instructions and has a reasonable spatial structure is finally generated at the initial time step.

[0014] The present invention also provides a text-guided image editing system with scene layout occlusion awareness, comprising: The dataset preprocessing module is used to preprocess the occlusion layout dataset and the image editing dataset; The occlusion layout-aware image consistency editing model construction module is used to build an occlusion layout-aware image consistency editing model for image editing in occlusion scenarios; it includes an occlusion layout generation module and an instruction layout-aware diffusion module. The occlusion layout-aware image consistency editing model training module is used to train the occlusion layout-aware image consistency editing model using a preprocessed occlusion layout dataset and an image editing dataset. The image editing module is used to load the trained image consistency editing model based on occlusion layout awareness and perform image editing.

[0015] The present invention also provides a text-guided image editing device with scene layout occlusion awareness, comprising: Memory: A computer program storing the above-described text-guided image editing method for scene layout occlusion perception, which is a computer-readable device; Processor: Used to implement the scene layout occlusion perception text-guided image editing method when executing the computer program.

[0016] The present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, enables the implementation of the aforementioned text-guided image editing method based on scene layout occlusion perception.

[0017] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This invention constructs an occlusion layout generation module, effectively aggregates the global appearance information of an image using a cross-attention mechanism (i.e., a cross-attention layer and a self-attention layer), and introduces depth constraints in feature interaction through a depth-aware attention mechanism (i.e., a depth-aware attention layer), thereby accurately inferring the geometry and spatial hierarchy of the occluded area. By injecting this information into the diffusion network, it effectively avoids the problem of artifacts such as structural distortion and texture breakage that are easily generated in the occluded re-revealed area during subsequent editing.

[0018] 2. This invention constructs an instruction layout alignment network including a feature splitting layer, calculates the correlation between occlusion layout features and text embedding, and explicitly decouples the features into an edit set and a consistency set based on the alignment score through the feature splitting layer. This decoupling mechanism can automatically identify the target area to be edited and the background area to be retained, avoiding accidental damage to non-target areas during the editing process and achieving precise positioning of the editing intent.

[0019] 3. This invention introduces hybrid conditional features, consisting of consistency set tokens and strong conditional features, into the diffusion network denoising process. This locks the visual features of the non-editable region during generation while simultaneously guiding changes in the target editing region. This dual-guidance mechanism ensures that, when performing complex occlusion relationship changes, high-quality edited content is generated while perfectly maintaining the integrity and consistency of the background structure and unedited objects.

[0020] In summary, this invention, through the occlusion layout generation module and the instruction layout perception diffusion module, combines the structural guidance of occlusion layout features and the semantic guidance of natural language text instructions, and has the advantages of being able to achieve feature decoupling between the edited area and the non-edited area, and generating high-quality edited images with complete structure and consistent background in complex occlusion scenarios. Attached Figure Description

[0021] Figure 1 This is a structural diagram of the occlusion layout generation module of the present invention.

[0022] Figure 2 This is a structural diagram of the instruction layout perception diffusion module of the present invention.

[0023] Figure 3 This is a flowchart of the training process for the occlusion layout generation module of the present invention.

[0024] Figure 4This is a flowchart of the training process for the image consistency editing model based on occlusion layout awareness of the present invention.

[0025] Figure 5 This is a flowchart of the image editing process of the image consistency editing model based on occlusion layout awareness of the present invention.

[0026] Figure 6 This is a comparison diagram of the generation effects of the present invention and existing editing methods. Detailed Implementation

[0027] To make the implementation process and features of the method clearer and easier to understand, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0028] The core of this invention includes: an occlusion layout generation module that uses cross-attention and depth-aware attention mechanisms to progressively infer the complete geometric shape of objects in a scene from shallow to deep layers, generating an occlusion layout that includes a full-view mask and the depth level of the objects, thus solving the problem of traditional methods struggling to infer information black holes; and an instruction layout-aware diffusion module that divides scene objects into an edit set and a consistency set through natural language parsing, injecting editing intent and structural priors into the diffusion network respectively. Under this dual guidance, the diffusion network collaboratively generates new regions, ensuring a high degree of integration between the newly generated regions and the original scene in terms of lighting, texture, and perspective, achieving precise modification of specific targets while strictly maintaining the visual stability of non-edited areas. These technologies collectively solve the problems of structural collapse and texture inconsistency that easily occur in existing editing models when handling occlusion relationships, significantly improving the effectiveness and practicality of editing complex scenes.

[0029] A text-guided image editing method with scene layout occlusion awareness includes the following steps: Step 1, preprocessing the occlusion layout dataset and the image editing dataset; The specific method of step 1 includes: Step 1.1: Construct the occlusion layout dataset and image editing dataset and determine the annotation information; The occlusion layout dataset includes the original RGB image and the corresponding occlusion scene annotations. The annotation information of the occlusion scene includes the visible mask of the object, the full-view mask, and the relative depth relationship between the objects. The image editing dataset includes the original RGB image, natural language text instructions, and the corresponding edited real image; Step 1.2: Perform standardization preprocessing on the occlusion layout dataset and image editing dataset constructed in Step 1.1 using an image processing library; Using an image processing library, the original RGB images from the occlusion layout dataset constructed in step 1.1 and the original RGB images and edited real images from the image editing dataset are read. The image resolution is uniformly adjusted using the bilinear interpolation algorithm and converted into floating-point Tensor format. At the same time, the three color channels of the image are normalized using specified mean and standard deviation as distribution parameters to map the range of image pixel values ​​to fixed values, so as to eliminate illumination differences and stabilize gradient convergence during model training. The preprocessed occlusion layout dataset and image editing dataset are obtained. Step 1.3: Use the annotation parsing tool to perform feature processing on the annotation information; For mask-type data, the visible mask and full-view mask of the objects in the annotation information of the occlusion scene in the occlusion layout dataset constructed in step 1.1 are adjusted to a resolution that matches the model feature map and binarized to be converted into floating-point Tensors to obtain the feature-processed visible mask and full-view mask. For depth data, the relative depth relationship between objects is transformed into an antisymmetric matrix, and the numerical values ​​are standardized to represent the front and back occlusion order between objects, thus obtaining the depth adjacency matrix. For the natural language text instructions in the image editing dataset constructed in step 1.1, a pre-trained word embedding device is used to transform the natural language text instructions into high-dimensional word embedding vectors, thus obtaining the text instruction word embedding vectors.

[0030] Step 2: Construct an image consistency editing model based on occlusion layout awareness for image editing in occlusion scenarios; including an occlusion layout generation module and an instruction layout awareness diffusion module; like Figure 1 As shown, step 2.1 involves constructing an occlusion layout generation module; Based on the Transformer architecture, an occlusion layout generation module is constructed. The occlusion layout generation module is used to extract the visual features and spatial relationships of potential objects from the occlusion layout dataset constructed in step 1.1, predict the overall geometric structure and depth order of objects in the scene, and output an occlusion layout containing complete occlusion information. The occlusion layout generation module is composed of an occlusion layout decoding network and a layout prediction output network connected in sequence. Step 2.1.1: Initialize image features and query vectors. The occlusion layout generation module receives image features extracted by the pre-trained Swin Transformer network and a set of randomly initialized learnable query vectors to represent potential object instances in the scene. Step 2.1.2: Construct the occlusion layout decoding network. The occlusion layout decoding network consists of three stacked decoder layers. Each decoder layer consists of an embedding layer, a cross-attention layer, a layer normalization layer, a cross-attention layer, a layer normalization layer, a feedforward network layer, a depth-aware attention layer, a layer normalization layer, a self-attention layer, a layer normalization layer, and a feedforward network layer. The input is the image features and learnable query vector initialized in step 2.1.1. The occlusion layout decoding network can capture appearance information using the cross-attention mechanism and introduce depth constraints in feature interaction through the depth-aware self-attention mechanism, thereby inferring the geometry of the occluded region.

[0031] First, the image features and learnable query vectors initialized in step 2.1.1 are input into the embedding layer for feature mapping, resulting in vectors mapped from the image features and vectors mapped from the learnable query vectors. Then, a cross-attention layer is used to interact with the vectors mapped from the learnable query vectors as query terms and the vectors mapped from the image features as key terms, aggregating the appearance information of the image. The output features are then processed by layer normalization. Next, to enhance the aggregation capability, the features output after layer normalization are again fed into the cross-attention layer and layer normalization, and then fused through a feedforward network layer. Subsequently, the fused features are input into a depth-aware attention layer, embedding depth constraints during feature interaction to correct spatial geometric representations. After layer normalization, a self-attention layer and layer normalization are used to establish global dependencies between features. Finally, after nonlinear mapping by the feedforward network layer, a feature sequence containing the geometric features of the occluded region is output. Step 2.1.3: Construct the layout prediction output network. The layout prediction output network is connected to the end of the occlusion layout decoding network. It consists of a linear layer and a multilayer perceptron in parallel. The feature sequence output from Step 2.1.2 is input into the two branches respectively. The linear layer is used to predict the depth level of the object, and the multilayer perceptron is used to generate the full-view mask of the object. The two together constitute the occlusion layout features, which are used as the output of the occlusion layout generation module. like Figure 2 As shown in step 2.2, construct the instruction layout awareness diffusion module; The instruction layout-aware diffusion module is used to semantically align and decouple the occlusion layout features output by the occlusion layout generation module in step 2.1 with the natural language text instructions in the image editing dataset constructed in step 1.1 to obtain conditional features, and then inject the processed conditional features into the generation process; The instruction layout-aware diffusion module is composed of an instruction layout alignment network, a conditional feature decoupling network and an image editing network connected in sequence. Step 2.2.1: Construct the instruction layout alignment network. The instruction layout alignment network, as a feature interaction and splitting model, consists of an embedding layer, a cross-attention layer, a multilayer perceptron, and a feature splitting layer. First, the natural language text instructions in the image editing dataset constructed in Step 1.1 are processed through the embedding layer to obtain text embeddings. Then, the cross-attention layer is used to calculate the correlation between the occlusion layout features output by the occlusion layout generation module in Step 2.1 and the text embeddings. After being mapped to alignment scores by the multilayer perceptron, the feature splitting layer divides the occlusion layout features into an edit set and a consistency set according to the alignment scores, and outputs the decoupled edit set Tokens and consistency set Tokens. Step 2.2.2: Construct a conditional feature decoupling network. As a feature fusion model, the conditional feature decoupling network consists of a concatenation layer, a self-attention layer, a layer normalization layer, and a feedforward network layer. The conditional feature decoupling network concatenates the text embeddings and edit set tokens obtained in Step 2.2.1 in a dimensional manner. Through deep processing by the self-attention layer, the layer normalization layer, and the feedforward network layer, it generates strong conditional features that fuse editing intent and structural priors. Step 2.2.3: Construct an image editing network. The image editing network is based on the Stable Diffusion v1.5 architecture, and its core includes a diffusion network. The diffusion network receives the original RGB images in the image editing dataset constructed in Step 1.1 as input, the text embeddings obtained in Step 2.2.1, the consistency set tokens generated in Step 2.2.1, and the strong conditional features generated in Step 2.2.2 as mixed conditional features. During the denoising process of the diffusion network, the mixed conditional features are used to guide the generation direction. While preserving the structure of the non-edited region, the target region of the input image is accurately edited, and finally, a high-quality edited image is output.

[0032] In step 2.2.3, the specific method for using hybrid conditional features to guide the generation direction during the denoising process of the diffusion network, while preserving the structure of the non-edited region, to accurately edit the target region of the input image and finally output a high-quality scene image includes: Random Gaussian noise is added to the original RGB images in the image editing dataset constructed in step 1.1. The original RGB images in the image editing dataset constructed in step 1.1 with added random Gaussian noise, the mixture conditional features, and the current time step are used as input to the diffusion network. The noise added between the current time step and the previous time step is predicted. This predicted noise is then subtracted from the original RGB image with added random Gaussian noise to obtain the predicted image for the previous time step. The diffusion network is then used again as input to this predicted image, the mixed conditional features, and the next time step to predict the noise added between the current time step and the previous time step, thus obtaining the predicted image for the previous time step again. This process is repeated until the initial time step is reached, resulting in the final edited image.

[0033] Step 3: Use the occlusion layout dataset and image editing dataset preprocessed in Step 1 to train the image consistency editing model based on occlusion layout awareness constructed in Step 2; The specific method for step 3 includes: like Figure 3 As shown in step 3.1, configure the training parameters and inputs for the occlusion layout generation module in the occlusion layout-aware image consistency editing model, and then train it: The original RGB images from the occlusion layout dataset preprocessed in step 1.2 and the visible mask after feature processing in step 1.3 are used as inputs to the occlusion layout generation module constructed in step 2.1. The corresponding full-view mask and the depth adjacency matrix after feature processing in step 1.3 are used as the ground truth labels. AdamW is used as the parameter optimizer, and the occlusion layout generation module is trained iteratively with the learning rate decaying through backpropagation using a cosine annealing strategy. Finally, the weight file of the trained occlusion layout generation module is obtained, which is used for subsequent image consistency editing tasks.

[0034] like Figure 4 As shown in step 3.2, configure the training parameters and inputs for the occlusion layout-aware image consistency editing model, and then train it: The original RGB images from the image editing dataset after preprocessing in step 1.2 and the embedded vectors of the text instruction words after feature processing in step 1.3 are used as inputs to the image consistency editing model based on occlusion layout awareness constructed in step 2. The edited real images from the image editing dataset after preprocessing in step 1.2 are used as training targets. The weight file of the occlusion layout generation module trained in step 3.1 is loaded and frozen as the parameters of the occlusion layout generation module in the image consistency editing model based on occlusion layout awareness. The weights of the pre-trained Stable Diffusion v1.5 model are loaded and frozen as the parameters of the diffusion network of the instruction layout awareness diffusion module in the image consistency editing model based on occlusion layout awareness. The parameters are updated only for the instruction layout alignment network constructed in step 2.2.1 and the conditional feature decoupling network constructed in step 2.2.2. Through backpropagation and parameter iteration, the weight file of the trained occlusion layout-aware image consistency editing model is finally obtained, which can be used for subsequent image consistency editing tasks.

[0035] During the training process in step 3.1, the model parameters of the occlusion layout generation module in step 3.1 are optimized using a hybrid supervised loss function; the hybrid supervised loss function includes two parts: full-view mask prediction loss and depth ranking loss. For the overall mask prediction loss, a combination of binary cross-entropy loss and Dice loss is used to ensure that the generated mask is consistent with the true label in terms of pixel-level classification and overall shape overlap. The calculation formula is as follows: in, The pixel values ​​of the full-view mask obtained in step 1.3 are the feature-processed values. These are the mask pixel values ​​predicted by the model. Total number of pixels; To address the depth ranking loss, an ordinal depth ranking loss is introduced to supervise the model's correct inference of the object occlusion order. The calculation formula is as follows: in, Representing objects and The predicted depth value; As an indicator variable, if the object In objects The back (which is obscured) is... ,otherwise Depth ranking loss penalizes depth predictions that violate true occlusion relationships through logistic regression. Combining the overall mask prediction loss and the depth ranking loss, the total loss function of the occlusion layout generation module is obtained as follows: in, and The preset loss weight coefficient.

[0036] During the training process in step 3.2, a noise prediction loss based on latent space is used. To ensure that the image generated by the model conforms to the editing intent and maintains structural consistency, the optimization objective is defined as minimizing the mean square error between the predicted noise and the actual added noise. in, For noise sampled from a standard Gaussian distribution, For the noise predicted by the diffusion network, For diffusion time step, Encoding of the image in the latent space, and These are the consensus set Tokens after decoupling in step 2.2.1 and the strongly conditional features of the fusion editing intent and structural priors output in step 2.2.2.

[0037] Step 4: Load the occlusion layout-aware image consistency editing model trained in Step 3 and perform image editing.

[0038] like Figure 5 As shown, in step 4.1, the weight file of the image consistency editing model based on occlusion layout awareness trained in step 3 is read and loaded into the network structure of the image consistency editing model based on occlusion layout awareness. The state of the image consistency editing model based on occlusion layout awareness is set to inference mode, all parameters of each network layer are fixed, and gradient calculation is turned off to meet the inference requirements. Step 4.2 involves inputting the original image to be edited and the natural language text instructions into the occlusion layout-aware image consistency editing model to execute the editing process; specifically including: The occlusion layout generation module encodes the features of the original image to be edited. A pre-trained SwingTransformer network extracts the image features of the original image and infers the overall mask and relative depth relationship of objects in the scene, outputting the occlusion layout features. The instruction layout alignment network in the instruction layout perception diffusion module receives the occlusion layout features and the input natural language text instructions. Attention scores are calculated to decouple the occlusion layout features into edit sets (Tokens) and consistency sets (Tokens). The decoupled edit sets (Tokens) and natural language text instructions are deeply fused through a conditional feature decoupling network to generate strong conditional features that fuse the editing intent and structural priors. The diffusion network uses the latent variables of the original image to be edited with added random Gaussian noise as the initial diffusion state. The strong conditional features that fuse the editing intent and structural priors, the consistency sets (Tokens), and the natural language text instructions are injected into the denoising process as guiding signals. Through multi-step noise prediction and gradual restoration of image pixels, a high-quality edited image that conforms to the text instructions and has a reasonable spatial structure is finally generated at the initial time step.

[0039] Experimental Analysis 1.1 Experimental Conditions The experimental software environment is based on the Linux operating system, and the model algorithm is built using the Python programming language and the PyTorch deep learning framework. The hardware environment is equipped with an NVIDIA 4090 to meet the memory and computing power requirements of the image editing model. The experimental data uses the image editing dataset constructed in step 1.1.

[0040] 1.2 Experiment Content To verify the effectiveness of the proposed occlusion layout-aware image consistency editing model, a qualitative comparative experiment was designed with the current mainstream instruction-driven image editing methods InstructPix2Pix and MGIE. The experiment primarily focused on editing tasks involving changes in occlusion relationships. During the experiment, consistent input conditions were maintained; the same source image and natural language editing instructions were input into both the proposed model and the comparative model, and the generated results were used for visual quality evaluation.

[0041] 1.3 Experimental Results Figure 6 shows the qualitative comparison results of the experiment. In the first task, "Remove the book placed on the teddy bear's legs," InstructPix2Pix failed to correctly understand the geometric structure of the image, resulting in severe damage to the teddy bear's leg and abdominal structure while removing the book, and introducing messy artifacts. Although MGIE preserved the main outline, it failed to effectively execute the removal instruction, and the texture and shape of the book were still obviously left behind. In contrast, the image consistency editing model based on occlusion layout awareness proposed in this invention not only accurately removed the occluded object (the book), but also successfully inferred and completed the originally occluded abdominal and leg areas of the teddy bear. The generated texture was natural and highly integrated with the surrounding environment, maintaining the integrity of the occluded object's structure.

[0042] In the second task, "replace the book on the chair with a small cup," InstructPix2Pix exhibited semantic understanding errors, failing to remove the original book and instead incorrectly overlaying the cup onto the image, causing visual confusion. While MGIE generated the target object (the cup), it performed poorly in handling the interaction between the object and its environment, resulting in a lack of reasonable spatial occlusion between the teddy bear's arm and the newly generated cup, leading to a visually abrupt effect. In contrast, the proposed image consistency editing model based on occlusion layout awareness not only successfully replaced the book with the cup but also reconstructed a reasonable interactive pose, allowing the teddy bear to naturally "hold" the cup while perfectly preserving the detail consistency between the background chair and the unedited areas of the teddy bear.

[0043] Therefore, the image consistency editing model based on occlusion layout awareness proposed in this invention can effectively solve the problems of artifacts such as structural distortion and texture breakage in the occluded re-revealed area, and can meet the requirements of maintaining spatial logic and physical authenticity in occluded scenes.

[0044] The present invention also provides a text-guided image editing system with scene layout occlusion awareness, comprising: The dataset preprocessing module is used to perform preprocessing of the occlusion layout dataset and image editing dataset in step 1; The occlusion layout-aware image consistency editing model construction module is used to implement the construction of the occlusion layout-aware image consistency editing model in step 2 for image editing in occlusion scenarios; it includes an occlusion layout generation module and an instruction layout-aware diffusion module. The occlusion layout-aware image consistency editing model training module is used to train the occlusion layout-aware image consistency editing model constructed in step 2 using the occlusion layout dataset and image editing dataset preprocessed in step 1 in step 3. The image editing module is used to load the occlusion layout-aware image consistency editing model trained in step 3 in step 4 and perform image editing.

[0045] The present invention also provides a text-guided image editing device with scene layout occlusion awareness, comprising: Memory: A computer program storing the above-described text-guided image editing method for scene layout occlusion perception, which is a computer-readable device; Processor: Used to implement the scene layout occlusion perception text-guided image editing method when executing the computer program.

[0046] The present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, enables the implementation of the aforementioned text-guided image editing method based on scene layout occlusion perception.

Claims

1. A text-guided image editing method based on scene layout occlusion awareness, characterized in that, Includes the following steps: Step 1: Preprocess the occlusion layout dataset and the image editing dataset; Step 2: Construct an image consistency editing model based on occlusion layout awareness for image editing in occlusion scenarios; This includes an occlusion layout generation module and an instruction layout perception and diffusion module; Step 3: Use the occlusion layout dataset and image editing dataset preprocessed in Step 1 to train the image consistency editing model based on occlusion layout awareness constructed in Step 2; Step 4: Load the occlusion layout-aware image consistency editing model trained in Step 3 and perform image editing.

2. The text-guided image editing method based on scene layout occlusion perception according to claim 1, characterized in that, The specific method of step 1 includes: Step 1.1: Construct the occlusion layout dataset and image editing dataset and determine the annotation information; The occlusion layout dataset includes the original RGB image and the corresponding occlusion scene annotations. The annotation information of the occlusion scene includes the visible mask of the object, the full-view mask, and the relative depth relationship between the objects. The image editing dataset includes the original RGB image, natural language text instructions, and the corresponding edited real image; Step 1.2: Perform standardization preprocessing on the occlusion layout dataset and image editing dataset constructed in Step 1.1 using an image processing library; Using an image processing library, the original RGB images from the occlusion layout dataset constructed in step 1.1 and the original RGB images and edited real images from the image editing dataset are read. The image resolution is uniformly adjusted using a bilinear interpolation algorithm and converted into floating-point Tensor format. At the same time, the three color channels of the image are normalized using specified mean and standard deviation as distribution parameters to map the range of image pixel values ​​to fixed values, thus obtaining the preprocessed occlusion layout dataset and image editing dataset. Step 1.3: Use the annotation parsing tool to perform feature processing on the annotation information; For mask-type data, the visible mask and full-view mask of the objects in the annotation information of the occlusion scene in the occlusion layout dataset constructed in step 1.1 are adjusted to a resolution that matches the model feature map and binarized to be converted into floating-point Tensors to obtain the feature-processed visible mask and full-view mask. For depth data, the relative depth relationship between objects is transformed into an antisymmetric matrix, and the numerical values ​​are standardized to represent the front and back occlusion order between objects, thus obtaining the depth adjacency matrix. For the natural language text instructions in the image editing dataset constructed in step 1.1, a pre-trained word embedding device is used to transform the natural language text instructions into high-dimensional word embedding vectors, thus obtaining the text instruction word embedding vectors.

3. The text-guided image editing method based on scene layout occlusion perception according to claim 1, characterized in that, The specific method for step 2 includes: Step 2.1, construct the occlusion layout generation module; Based on the Transformer architecture, an occlusion layout generation module is constructed. The occlusion layout generation module is used to extract the visual features and spatial relationships of potential objects from the occlusion layout dataset constructed in step 1.1, predict the overall geometric structure and depth order of objects in the scene, and output an occlusion layout containing complete occlusion information. The occlusion layout generation module is composed of an occlusion layout decoding network and a layout prediction output network connected in sequence. Step 2.1.1: Initialize image features and query vectors. The occlusion layout generation module receives image features extracted by the pre-trained SwinTransformer network and a set of randomly initialized learnable query vectors to represent potential object instances in the scene. Step 2.1.2: Construct the occlusion layout decoding network. The occlusion layout decoding network consists of three stacked decoder layers. Each decoder layer consists of an embedding layer, a cross-attention layer, a layer normalization layer, a cross-attention layer, a layer normalization layer, a feedforward network layer, a depth-sensing attention layer, a layer normalization layer, a self-attention layer, a layer normalization layer, and a feedforward network layer. The input is the image features and learnable query vector initialized in step 2.1.

1. First, the image features and learnable query vectors initialized in step 2.1.1 are input into the embedding layer for feature mapping, resulting in vectors mapped from the image features and vectors mapped from the learnable query vectors. Then, a cross-attention layer is used to interactively compute the vector mapped from the learnable query vectors as the query term and the vector mapped from the image features as the key term, aggregating the image's appearance information. The output features are then processed by layer normalization. Next, the features output after layer normalization are again fed into the cross-attention layer and layer normalization, and then fused through a feedforward network layer. Subsequently, the fused features are input into a depth-aware attention layer, where depth constraints are embedded during feature interaction to correct spatial geometric representations. After layer normalization, a self-attention layer and layer normalization are used to establish global dependencies between features. Finally, after nonlinear mapping by the feedforward network layer, a feature sequence containing the geometric features of the occluded region is output. Step 2.1.3: Construct the layout prediction output network. The layout prediction output network is connected to the end of the occlusion layout decoding network. It consists of a linear layer and a multilayer perceptron in parallel. The feature sequence output from Step 2.1.2 is input into the two branches respectively. The linear layer is used to predict the depth level of the object, and the multilayer perceptron is used to generate the full-view mask of the object. The two together constitute the occlusion layout features, which are used as the output of the occlusion layout generation module. Step 2.2, construct the instruction layout awareness diffusion module; The instruction layout-aware diffusion module is used to semantically align and decouple the occlusion layout features output by the occlusion layout generation module in step 2.1 with the natural language text instructions in the image editing dataset constructed in step 1.1 to obtain conditional features, and then inject the processed conditional features into the generation process; The instruction layout-aware diffusion module is composed of an instruction layout alignment network, a conditional feature decoupling network and an image editing network connected in sequence. Step 2.2.1: Construct the instruction layout alignment network. The instruction layout alignment network, as a feature interaction and splitting model, consists of an embedding layer, a cross-attention layer, a multilayer perceptron, and a feature splitting layer. First, the natural language text instructions in the image editing dataset constructed in Step 1.1 are processed through the embedding layer to obtain text embeddings. Then, the cross-attention layer is used to calculate the correlation between the occlusion layout features output by the occlusion layout generation module in Step 2.1 and the text embeddings. After being mapped to alignment scores by the multilayer perceptron, the feature splitting layer divides the occlusion layout features into an edit set and a consistency set according to the alignment scores, and outputs the decoupled edit set Tokens and consistency set Tokens. Step 2.2.2: Construct a conditional feature decoupling network. As a feature fusion model, the conditional feature decoupling network consists of a concatenation layer, a self-attention layer, a layer normalization layer, and a feedforward network layer. The conditional feature decoupling network concatenates the text embeddings and edit set tokens obtained in Step 2.2.1 in a dimensional manner. Through deep processing by the self-attention layer, the layer normalization layer, and the feedforward network layer, it generates strong conditional features that fuse editing intent and structural priors. Step 2.2.3: Construct an image editing network. The image editing network is based on the Stable Diffusion v1.5 architecture and includes a diffusion network. The diffusion network receives the original RGB images from the image editing dataset constructed in Step 1.1 as input, the text embeddings obtained in Step 2.2.1, the consistency set tokens generated in Step 2.2.1, and the strong conditional features generated in Step 2.2.2 as mixed conditional features. During the denoising process of the diffusion network, the mixed conditional features are used to guide the generation direction. While preserving the structure of the non-edited region, the target region of the input image is edited, and the edited image is finally output.

4. The text-guided image editing method based on scene layout occlusion perception according to claim 1, characterized in that, The specific method for step 3 includes: Step 3.1: Configure the training parameters and inputs for the occlusion layout generation module in the occlusion layout-aware image consistency editing model, and perform training: The original RGB images from the occlusion layout dataset preprocessed in step 1.2 and the visible mask after feature processing in step 1.3 are used as inputs to the occlusion layout generation module constructed in step 2.

1. The corresponding full-view mask and the depth adjacency matrix after feature processing in step 1.3 are used as the real labels. The occlusion layout generation module is trained iteratively with parameters through backpropagation, and finally the weight file of the trained occlusion layout generation module is obtained. Step 3.2: Configure the training parameters and inputs for the occlusion layout-aware image consistency editing model, and perform training: The original RGB images from the image editing dataset after preprocessing in step 1.2 and the embedded vectors of the text instruction words after feature processing in step 1.3 are used as inputs to the image consistency editing model based on occlusion layout awareness constructed in step 2. The edited real images from the image editing dataset after preprocessing in step 1.2 are used as training targets. The weight file of the occlusion layout generation module trained in step 3.1 is loaded and frozen as the parameters of the occlusion layout generation module in the image consistency editing model based on occlusion layout awareness. The weights of the pre-trained Stable Diffusion v1.5 model are loaded and frozen as the parameters of the diffusion network of the instruction layout awareness diffusion module in the image consistency editing model based on occlusion layout awareness. The parameters are updated only for the instruction layout alignment network constructed in step 2.2.1 and the conditional feature decoupling network constructed in step 2.2.

2. Through backpropagation and parameter iteration, the weight file of the trained image consistency editing model based on occlusion layout awareness is finally obtained.

5. The text-guided image editing method based on scene layout occlusion perception according to claim 3, characterized in that, In step 2.2.3, the specific method for using hybrid conditional features to guide the generation direction during the denoising process of the diffusion network, while preserving the structure of the non-edited region, to edit the target region of the input image and finally output a high-quality scene image includes: Random Gaussian noise is added to the original RGB images in the image editing dataset constructed in step 1.

1. The original RGB images in the image editing dataset constructed in step 1.1 with added random Gaussian noise, the mixture conditional features, and the current time step are used as input to the diffusion network. The noise added between the current time step and the previous time step is predicted. This predicted noise is then subtracted from the original RGB image with added random Gaussian noise to obtain the predicted image for the previous time step. The diffusion network is then used again as input to this predicted image, the mixed conditional features, and the next time step to predict the noise added between the current time step and the previous time step, thus obtaining the predicted image for the previous time step again. This process is repeated until the initial time step is reached, resulting in the final edited image.

6. The text-guided image editing method based on scene layout occlusion awareness according to claim 4, characterized in that, During the training process in step 3.1, the model parameters of the occlusion layout generation module in step 3.1 are optimized using a hybrid supervised loss function; the hybrid supervised loss function includes two parts: full-view mask prediction loss and depth ranking loss. For the overall mask prediction loss, a combination of binary cross-entropy loss and Dice loss is used, and the calculation formula is as follows: in, These are the pixel values ​​of the full-view mask obtained in step 1.3 after feature processing. These are the mask pixel values ​​predicted by the model. Total number of pixels; To address the depth ranking loss, an ordinal depth ranking loss is introduced to supervise the model's correct inference of the object occlusion order. The calculation formula is as follows: in, Representing objects and The predicted depth value; As an indicator variable, if the object In objects The back (which is obscured) is... ,otherwise Depth ranking loss penalizes depth predictions that violate true occlusion relationships through logistic regression. Combining the overall mask prediction loss and the depth ranking loss, the total loss function of the occlusion layout generation module is obtained as follows: in, and These are the preset loss weighting coefficients; During the training process in step 3.2, a noise prediction loss based on latent space is adopted, and the optimization objective is defined as minimizing the mean square error between the predicted noise and the actual added noise: in, For noise sampled from a standard Gaussian distribution, For the noise predicted by the diffusion network, For diffusion time step, Encoding of the image in the latent space, and These are the consensus set Tokens after decoupling in step 2.2.1 and the strongly conditional features of the fusion editing intent and structural priors output in step 2.2.

2.

7. The text-guided image editing method based on scene layout occlusion perception according to claim 1, characterized in that, The specific method for step 4 includes: Step 4.1: Read the weight file of the image consistency editing model based on occlusion layout awareness trained in Step 3, load it into the network structure of the image consistency editing model based on occlusion layout awareness, set the state of the image consistency editing model based on occlusion layout awareness to inference mode, fix all parameters of each network layer, and turn off gradient calculation to meet inference requirements. Step 4.2 involves inputting the original image to be edited and the natural language text instructions into the occlusion layout-aware image consistency editing model to execute the editing process; specifically including: The occlusion layout generation module encodes the features of the original image to be edited. A pre-trained SwingTransformer network extracts the image features of the original image and infers the overall mask and relative depth relationship of objects in the scene, outputting the occlusion layout features. The instruction layout alignment network in the instruction layout perception diffusion module receives the occlusion layout features and the input natural language text instructions. Attention scores are calculated to decouple the occlusion layout features into edit sets (Tokens) and consistency sets (Tokens). The decoupled edit sets (Tokens) and natural language text instructions are deeply fused through a conditional feature decoupling network to generate strong conditional features that fuse the editing intent and structural priors. The diffusion network uses the latent variables of the original image to be edited with added random Gaussian noise as the initial diffusion state. The strong conditional features that fuse the editing intent and structural priors, the consistency sets (Tokens), and the natural language text instructions are injected into the denoising process as guiding signals. Through multi-step noise prediction and gradual restoration of image pixels, an edited image that conforms to the text instructions and has a reasonable spatial structure is finally generated at the initial time step.

8. A text-guided image editing system based on scene layout occlusion perception according to claim 1, characterized in that, include: The dataset preprocessing module is used to preprocess the occlusion layout dataset and the image editing dataset; The occlusion layout-aware image consistency editing model construction module is used to build an occlusion layout-aware image consistency editing model for image editing in occlusion scenarios. This includes an occlusion layout generation module and an instruction layout perception and diffusion module; The occlusion layout-aware image consistency editing model training module is used to train the occlusion layout-aware image consistency editing model using a preprocessed occlusion layout dataset and an image editing dataset. The image editing module is used to load the trained image consistency editing model based on occlusion layout awareness and perform image editing.

9. A text-guided image editing device with scene layout occlusion perception, characterized in that, include: Memory: A computer program for a text-guided image editing method with scene layout occlusion awareness as described in any one of claims 1-7, which is a computer-readable device; Processor: Used to implement the text-guided image editing method for scene layout occlusion perception as described in any one of claims 1-7 when executing the computer program.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, enables the implementation of a text-guided image editing method for scene layout occlusion awareness as described in any one of claims 1-7.

Images