Indoor scene image generation method and system based on dual guidance of structure and layout

Abstract

The application provides an indoor scene image generation method and system based on dual guidance of structure and layout, and belongs to the technical field of computer vision; the application splits the image semantics in a complex space scene into geometric structure semantics and continuous layout semantics through a double-flow architecture of a "frozen layout semantic backbone network + trainable copy module", models the two types respectively, realizes decoupling control of structure locking and layout rendering, and finally generates a target indoor scene image which inherits the geometric structure semantics of a structure reference image and the continuous layout semantics of a layout reference image; the application effectively solves the problem of serious coupling of structure semantics and layout semantics in the prior art, and the problem of mutual involvement of structure adjustment and style change, supports flexible style migration and structure-layout semantic combination generation across images under the premise of keeping the space structure stable and consistent, and has the advantages of stable training, high structure fidelity and strong engineering feasibility.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a method and system for generating indoor scene images based on dual guidance of structure and layout. Background Technology

[0002] With the development of deep learning and generative artificial intelligence technologies, image generation methods represented by diffusion models have become an important technical route for high-quality image synthesis. Among them, the latent diffusion model, by modeling noise and progressively denoising in the latent space, not only significantly reduces computational overhead, but also can combine various conditional information such as text, segmentation maps, and edge maps to achieve image generation. Therefore, it has been widely used in tasks such as image reconstruction, image editing, style transfer, and conditional generation.

[0003] However, existing image generation models still have significant limitations in application scenarios with strong structural constraints, such as interior design and architectural space generation. These scenarios typically contain two types of semantic information: one is geometric structural semantics, such as the outlines of walls, doors and windows, furniture, and perspective relationships, which requires the generated result to remain stable in spatial position, shape, and proportion; the other is continuous layout semantics, such as lighting gradations, color temperature distribution, material texture, and environmental atmosphere, which requires the generated result to be visually natural and smooth with good overall consistency. Because existing latent diffusion models usually adopt a highly integrated end-to-end modeling approach, they often encode geometric structure, spatial layout, and texture style into a unified implicit latent distribution, resulting in strong coupling between different types of semantic information.

[0004] Under this coupling mechanism, when users attempt to adjust only the scene style, lighting atmosphere, or local visual effects, the model often alters the spatial structure that should be strictly preserved, easily leading to problems such as wall deformation, furniture outline drift, perspective distortion, and accidental modification of background areas. Especially in practical tasks such as home decoration design, space renovation previews, and partial editing, users typically want to adjust only the style, materials, colors, or lighting while maintaining the original structure. However, current technology struggles to simultaneously meet both the requirements of "strictly locked structure" and "flexible style transfer."

[0005] Existing technologies primarily focus on conditional image generation and controllable image editing based on deep generative models, especially latent diffusion models. These technologies typically rely on pre-trained diffusion models, guiding them to generate target images that meet conditional constraints by inputting textual prompts, edge maps, segmentation maps, reference images, or other control signals. While existing conditional image generation techniques based on latent diffusion models have achieved image editing, style transfer, and conditional control to some extent, they still suffer from the following limitations in complex spatial scenarios such as interior design:

[0006] (1) The high coupling between structural semantics and layout semantics leads to insufficient control precision. Existing technologies typically use an end-to-end approach to uniformly model images, encoding information such as geometric structure, spatial layout, lighting atmosphere, and material texture in the same latent distribution, lacking an explicit separation mechanism for different semantic components. When users only want to adjust the image style, lighting, or local visual effects, the model often simultaneously changes the original geometric structure such as wall positions, furniture outlines, and spatial perspective, making it difficult to achieve flexible editing while maintaining the structure. The fundamental reason is that existing models lack independent structural control links and continuous layout control links, causing different semantic information to interfere with each other during the generation process.

[0007] (2) Insufficient ability to maintain structure in complex spatial scenes, easily leading to structural drift or perspective distortion. In interior design scenarios, walls, doors, windows, cabinets, furniture, and vanishing point relationships together constitute a strict spatial skeleton, requiring high consistency in the generated results. Even when introducing conditional signals such as edge maps and segmentation maps, existing methods often only provide shallow, local contour constraints, making it difficult to stably express spatial hierarchy, regional connectivity, and global perspective relationships in complex scenes. Therefore, during generation or editing, problems such as boundary misalignment, local deformation, abnormal furniture proportions, and chaotic occlusion relationships are prone to occur. The reason for this problem is that existing structural conditions are usually too sparse or low-level, unable to provide sufficiently rich semantics for mid-to-high-level structures.

[0008] (3) The injection of external control signals is relatively crude and easily disrupts the stable distribution of the pre-trained backbone. To enhance controllability, existing methods directly splice or superimpose edge, segmentation, or reference image features into the backbone network features, or directly fine-tune the entire pre-trained diffusion model. Although such approaches can introduce external conditions, due to the lack of a flexible and progressive access mechanism, they are prone to causing significant disturbances to the stable feature distribution already formed by the backbone model in the early stages of training, thereby leading to gradient oscillations, training instability, or even catastrophic forgetting.

[0009] (4) Insufficient generalization and combination capabilities make it difficult to support free combination of structure and style across images. Most existing technologies are better at completing generation tasks under single input conditions, such as reconstruction based on a single reference image or condition constraints based on a single control map; however, when users want to combine the spatial structure in one image with the lighting style and material atmosphere in another image, existing technologies often struggle to reliably complete cross-semantic fusion. This is because most existing methods do not clearly define the division of labor between "structure locking" and "layout rendering" at the model level, causing control signals from different sources to compete with each other in the feature space, ultimately affecting the generation quality and control accuracy.

[0010] Therefore, there is an urgent need for an indoor scene image generation method and system based on dual guidance of structure and layout to solve the technical problem that the high coupling between structural semantics and layout semantics in existing image generation technologies makes it impossible to maintain the consistency of geometric structure when adjusting style, lighting or atmosphere. Summary of the Invention

[0011] To address the aforementioned technical issues, this invention provides an indoor scene image generation method and system based on dual guidance of structure and layout. The aim is to effectively decouple structural semantics from continuous layout semantics during the image generation process, and establish a controllable generation technology that can stably preserve the spatial geometric skeleton while flexibly transferring environmental atmosphere and visual style, thereby improving the structural consistency, layout continuity, and overall controllability of the generated results in complex spatial scenes.

[0012] To achieve the above objectives, this application proposes an indoor scene image generation method based on dual guidance of structure and layout, including:

[0013] Obtain the structure flow image and layout flow image;

[0014] The layout flow image is input into the frozen layout semantic backbone network (KANDiff), which performs forward propagation processing on the layout flow image to extract layout features used to describe continuous layout semantics, which include at least lighting, color and material information.

[0015] The structure flow image is input to the structure feature extraction branch, which processes the structure flow image to extract multi-scale structural features for describing geometric structure semantics. The geometric structure semantics include at least object contours, edges, and spatial positional relationships.

[0016] The multi-scale structural features are input into the trainable replica module, which performs multi-scale encoding on the multi-scale structural features to generate multi-scale structural control signals corresponding to multiple decoding levels in the layout semantic backbone network.

[0017] A progressive injection mechanism is adopted to inject the multi-scale structural control signals into the corresponding decoding layers of the frozen layout semantic backbone network by adding residuals.

[0018] The frozen layout semantic backbone network decodes the layout features and the injected multi-scale structure control signal to generate a target indoor scene image, so that the target indoor scene image inherits the geometric structure semantics of the structure flow image and fuses the continuous layout semantics of the layout flow image.

[0019] As a further solution, the progressive injection mechanism is a dual zero-convolution injection mechanism, which includes:

[0020] Before inputting the multi-scale structural features into the trainable replica module, the multi-scale structural features are first dimension-mapped through a first zero convolutional layer and then fused with the input features of the layout semantic backbone network. The fused features are then gating through a second zero convolutional layer to make the increment of the structural control signal zero in the early stage of training. After that, the processed features are input into the trainable replica module.

[0021] As a further solution, the structural feature extraction branch includes a frozen visual perception network; the extraction of multi-scale structural features for describing geometric structural semantics includes:

[0022] The structured flow image is input into the frozen visual perception network, and activation features of multiple intermediate layers at different depths in the visual perception network are extracted as the multi-scale structural features.

[0023] Shallow activation features are used to describe edge and texture structures, while deep activation features are used to describe object outlines and spatial perspective relationships.

[0024] As a further solution, the visual perception network is a VGG network, and the multiple intermediate layers of different depths include at least a first intermediate layer, a second intermediate layer, a third intermediate layer, and a fourth intermediate layer, which correspond to the activation outputs of the relu1_1, relu2_1, relu3_1, and relu4_1 layers in the VGG network, respectively.

[0025] As a further solution, the frozen layout semantic backbone network is the U-Net backbone network of the pre-trained latent space diffusion model; the trainable copy module replicates the topology of the encoder part of the U-Net backbone network, and its parameters are updated independently of the parameters of the frozen layout semantic backbone network during training.

[0026] As a further solution, the trainable copy module performs multi-scale encoding on the multi-scale structural features to generate multi-scale structural control signals, including:

[0027] The trainable replica module extracts features layer by layer along its downsampling path, and uses the feature maps output by each downsampling level as the multi-scale structure control signal to be injected into the corresponding upsampling decoding level of the frozen layout semantic backbone network.

[0028] As a further solution, the step of injecting the multi-scale structural control signals into the corresponding decoding layers of the frozen layout semantic backbone network by residual addition includes:

[0029] The multi-scale structure control signals of each level are injected into the feature map of the corresponding decoding level in an element-wise addition manner.

[0030] As a further solution, during the model training phase, the layout flow image is a noisy image, and its corresponding input condition is an empty mask placeholder; the structure flow image is an indoor scene value image corresponding to the noisy image.

[0031] The method updates the parameters of the structural feature extraction branch, the trainable copy module, and the progressive injection mechanism by performing loss calculations on the generated target indoor scene image and the indoor scene value image, while keeping the parameters of the frozen layout semantic backbone network unchanged.

[0032] As a further solution, the structure flow image and the layout flow image are the same indoor scene image, so that the generated target indoor scene image can achieve style transfer or local editing of continuous layout semantics while keeping the geometric structure semantics unchanged in the structure flow image.

[0033] On the other hand, the present invention also provides an indoor scene image generation system based on dual guidance of structure and layout, for implementing the indoor scene image generation method based on dual guidance of structure and layout as described in any of the preceding claims, comprising:

[0034] At least one processor; and

[0035] A memory communicatively connected to the at least one processor; wherein,

[0036] The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enable the system to perform the following operations:

[0037] Obtain the structure flow image and layout flow image;

[0038] The layout flow image is input into the frozen layout semantic backbone network, and the frozen layout semantic backbone network performs forward propagation processing on the layout flow image to extract layout features used to describe continuous layout semantics, which include at least lighting, color and material information.

[0039] The structure flow image is input to the structure feature extraction module, which processes the structure flow image to extract multi-scale structural semantic features for describing geometric structure semantics. The geometric structure semantics include at least object contours, edges, and spatial positional relationships.

[0040] The multi-scale structural semantic features are input into the trainable replica module, which performs multi-scale encoding on the multi-scale structural semantic features to generate multi-scale structural control signals. The topology of the trainable replica module corresponds to at least a portion of the encoder structure of the layout semantic backbone network.

[0041] The multi-scale structural control signal is progressively injected into the corresponding decoding layer of the layout semantic backbone network through the progressive injection module, so that the generated target indoor scene image inherits the geometric structural semantics of the structural flow image and the continuous layout semantics of the layout flow image.

[0042] Compared with related technologies, the indoor scene image generation method and system based on dual guidance of structure and layout provided by the present invention has the following advantages:

[0043] 1. This invention decomposes the image semantics in complex spatial scenes into two categories: geometric structure semantics and continuous layout semantics. By constructing a dual-stream architecture of a "frozen layout semantic backbone network + trainable copy module," structural semantics and layout semantics are modeled separately and work collaboratively within the model. Specifically, the frozen layout semantic backbone network is responsible for modeling and rendering continuous layout semantics such as lighting, color, and materials, while the trainable copy module is responsible for extracting and encoding geometric structure semantics such as object contours, edges, and spatial relationships. Structural constraints are injected into the decoding process through multi-scale structural control signals. Therefore, this invention avoids the problem of mutual interference between different semantic information in existing end-to-end models, and can flexibly adjust scene style, lighting, and environmental atmosphere while maintaining spatial structural stability, achieving a division of labor and collaboration between "structure locking" and "layout rendering."

[0044] 2. This invention freezes the visual perception network in the structural feature extraction branch, extracting activation features from multiple intermediate layers of varying depths as multi-scale structural features. Shallow activation features describe edge and texture structures, while deep activation features describe object contours and spatial perspective relationships. Compared to existing technologies that rely solely on low-level contour constraints such as Canny edge maps, this invention provides richer mid-to-high-level structural semantic constraints for the generation process, including spatial hierarchy, regional connectivity, and global perspective relationships. This effectively reduces problems such as furniture deformation, boundary misalignment, and spatial drift, enhancing the structural fidelity of the generated results for complex interior scenes.

[0045] 3. Before inputting multi-scale structural features into the trainable copy module, this invention first performs dimension mapping through a first zero-convolutional layer and then adds and fuses them with the input features of the layout semantic backbone network. A second zero-convolutional layer then performs gating processing on the fused features. Since the parameters of the second zero-convolutional layer are initially zero, the increment of the structural control signal is zero in the early stages of training, preventing abrupt disturbances to the frozen layout semantic backbone network. As training progresses, the zero-convolutional parameters gradually learn an appropriate injection intensity, achieving a gradual and stable access to the structural control signal. This design effectively avoids the gradient oscillation and catastrophic forgetting problems caused by traditional direct splicing or fine-tuning methods, steadily enhancing the model's structural control capability while maintaining the original high-quality generation prior.

[0046] 4. During the inference phase, this invention can receive both structure flow images and layout flow images. The furniture geometry, spatial position, and perspective relationships from the structure flow image are used as geometric structural semantic input, while the lighting atmosphere, color temperature style, and material texture from the layout flow image are used as continuous layout semantic input. A trainable copy module generates multi-scale structural control signals, which are then injected into the decoding process of the frozen layout semantic backbone network. This ultimately generates a target interior scene image that simultaneously inherits the structural features of the former and the stylistic atmosphere of the latter. This cross-image semantic recombination capability enables this invention not only to generate and edit images under single-image conditions but also to support the free combination of structures and styles from images of different sources, expanding the model's application space in scenarios such as interior design preview, style transfer, and interactive assisted design.

[0047] 5. In this invention, only the parameters of the structural feature extraction branch (including the KANDiff trainable copy module and the dual zero convolution module) are updated during training, while the parameters of the frozen layout semantic backbone network (KANDiff) remain unchanged. This design achieves functional enhancement by simply adding control branches and feature injection modules without requiring overall retraining of the large-scale diffusion model. This helps reduce training costs and deployment difficulty, and also facilitates modular expansion based on existing generative models. Attached Figure Description

[0048] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0049] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0050] Figure 1This is a schematic diagram illustrating the steps of a controllable image generation method for indoor scenes provided by the present invention.

[0051] Figure 2 A diagram illustrating the dual-guided model structure provided in this embodiment of the invention;

[0052] Figure 3 A schematic diagram of intermediate activation features provided in an embodiment of the present invention;

[0053] Figure 4 This is an experimental result diagram of cross-semantic combination generation provided in an embodiment of the present invention;

[0054] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0056] Before providing a detailed description of the various embodiments of the present invention, the technical terms used in each embodiment are explained as follows:

[0057] Structural semantics refers to the semantics that describe information such as the geometric contours, spatial skeletons, edge relationships, and perspective relationships of objects in an image. It mainly includes wall positions, door and window outlines, furniture shapes, vanishing point relationships, and regional occlusion relationships.

[0058] Continuous layout semantics: refers to the semantics that describe the continuous visual attributes of an image, such as lighting atmosphere, color temperature tone, material texture and overall spatial style. It mainly includes lighting gradation, color distribution, material transition and environmental atmosphere.

[0059] Frozen Layout Semantic Backbone Network (KANDiff): refers to a pre-trained generative network whose parameters are not updated during training. It is used to retain existing generative capabilities and is responsible for modeling and rendering continuous layout semantics in this invention.

[0060] Trainable copy module: refers to a control branch that replicates the topology of the main branch but whose parameters are trained independently, and is used to carry out the extraction, encoding and injection of structural semantics.

[0061] Zero Convolution (ZeroConv): refers to a convolutional layer with initial parameters set to zero, used to introduce external control signals with minimal perturbation during the early stages of training.

[0062] Dual zero-convolutional modules: These refer to feature alignment and injection mechanisms consisting of at least two zero-convolutional layers and an intermediate pre-fusion process, used to achieve progressive and stable injection of structural features. The first zero-convolutional layer is used for feature space mapping and preprocessing, while the second zero-convolutional layer is used for cold-start gating.

[0063] Multi-scale residual injection: refers to injecting structural control signals into the corresponding decoding level feature maps of the backbone network in the form of element-wise addition residuals at different resolution levels, thereby affecting the final generation result at different scales.

[0064] Cross-image semantic recombination refers to the method of extracting structural semantics and continuous layout semantics from different input images and recombining them in the same generation process to output a new image, thereby achieving decoupled combination generation where "structure comes from image A and layout comes from image B".

[0065] Example 1

[0066] Please see Figure 1 This embodiment provides an indoor scene image generation method based on dual guidance of structure and layout. By constructing a dual-stream architecture of "frozen backbone + trainable copy" and combining perceptual structural feature extraction and zero-convolution progressive injection mechanism, it achieves decoupled control and collaborative generation of geometric structural semantics and continuous layout semantics in complex spatial scenes, including:

[0067] Obtain the structure flow image and layout flow image;

[0068] The layout flow image is input into the frozen layout semantic backbone network KANDiff, which performs forward propagation processing on the layout flow image to extract layout features used to describe continuous layout semantics, which include at least lighting, color and material information.

[0069] The structure flow image is input to the structure feature extraction branch, which processes the structure flow image to extract multi-scale structural features for describing geometric structure semantics. The geometric structure semantics include at least object contours, edges, and spatial positional relationships.

[0070] The multi-scale structural features are input into the trainable replica module, which performs multi-scale encoding on the multi-scale structural features to generate multi-scale structural control signals corresponding to multiple decoding levels in the layout semantic backbone network KANDiff.

[0071] A progressive injection mechanism is adopted to inject the multi-scale structural control signals into the corresponding decoding layers of the frozen layout semantic backbone network KANDiff by adding residuals respectively.

[0072] The frozen layout semantic backbone network KANDiff decodes the layout features and the injected multi-scale structure control signals to generate a target indoor scene image, so that the target indoor scene image inherits the geometric structure semantics of the structure flow image and fuses the continuous layout semantics of the layout flow image.

[0073] Specifically, this embodiment addresses the problems existing in the prior art, such as severe coupling between structural semantics and layout semantics, insufficient ability to maintain structure in complex spatial scenes, unstable expression of continuous light and shadow and material semantics, easy disruption of pre-trained model distribution by external control signal injection, and easy background drift during local editing. It provides an image generation method, system, and model architecture guided by both structure and layout to improve the structural controllability, layout continuity, and overall generation quality in the process of generating images of complex indoor scenes.

[0074] Specifically, the technical problems to be solved by this invention mainly include the following aspects:

[0075] (1) Solving the problem of the difficulty in independently controlling the mixed modeling of structural information and style and layout information in existing image generation models. Existing technologies usually encode the geometric structural semantics such as walls, doors, windows, and furniture outlines together with the continuous layout semantics such as lighting, color temperature, and material texture in a unified latent space, which causes the original structure to change when the user adjusts the style, atmosphere, or local effects. The primary technical problem to be solved by this invention is how to effectively decouple the geometric structural semantics and continuous layout semantics in image generation, so as to achieve the division of labor and cooperation between "structure locking" and "layout rendering".

[0076] (2) Solving the problem of unstable structure and easy deformation and perspective drift in complex spatial scenes. In view of the high requirements for the stability of spatial skeleton in interior design scenes, this invention aims to solve how to stably constrain the furniture shape, wall boundary, spatial position relationship and perspective relationship during the image generation or editing process, so that the generated image can maintain the consistency and accuracy of the original geometric structure while the style changes or the atmosphere shifts.

[0077] (3) To address the problem of unstable training and catastrophic forgetting when external control information is incorporated into the diffusion model. Existing technologies often use direct splicing, superposition, or overall fine-tuning when introducing external control conditions such as edge, segmentation, or perceptual features, which can easily disrupt the stable generative priors already formed by the pre-trained backbone. This invention aims to solve how to effectively inject structural control information into the pre-trained generative model in a non-intrusive and progressive manner, enhancing control capabilities while maintaining the original high-quality generative capabilities of the backbone network as much as possible.

[0078] (4) Solving the problem that existing technologies cannot support the free combination of cross-image structure and layout semantics. The present invention also solves the problem of how to enable the model to accept heterogeneous semantic input from different reference images at the same time during the inference stage, effectively integrate the geometric structure in one image with the lighting style and color temperature atmosphere in another image, and thus realize the decoupled combination generation of structure and layout across images, thereby improving the flexibility and generalization ability of the model.

[0079] Furthermore, the progressive injection mechanism is a dual zero-convolution injection mechanism, which includes:

[0080] Before inputting the multi-scale structural features into the trainable replica module, the multi-scale structural features are first dimension-mapped through a first zero convolutional layer and then fused with the input features of the layout semantic backbone network KANDiff. Then, the fused features are gated through a second zero convolutional layer to make the increment of the structural control signal zero in the early stage of training. After that, the processed features are input into the trainable replica module.

[0081] Specifically, in this embodiment, before inputting the multi-scale structural features into the trainable copy module, the structural features are first mapped to a dimension using a first zero-convolutional layer. This maps the structural features from the original feature space to a space with the same dimension as the input features of the layout semantic backbone network KANDiff. Then, the mapped features are added and fused with the input features of the layout semantic backbone network KANDiff to complete feature pre-fusion. Next, the fused features are gated using a second zero-convolutional layer to ensure that the increment of the structural control signal is zero in the early stages of training. After that, the processed features are input into the trainable copy module for multi-scale encoding.

[0082] The technical effectiveness of this dual zero-convolution injection mechanism is reflected in two aspects: the first zero-convolutional layer performs feature space alignment and preliminary mapping, enabling multi-scale structural features from the structural feature extraction branch to be dimensionally compatible with the input features of the layout semantic backbone network KANDiff, laying the foundation for subsequent fusion; the feature pre-fusion step then brings structural and layout information together for the first time on the input side; the second zero-convolutional layer performs cold-start gating. Since the convolutional parameters are initially zero, the increment of the entire structural branch on the frozen layout semantic backbone network KANDiff is zero in the early stages of training, preventing abrupt disturbances to the stable feature distribution already formed by the pre-trained backbone. As training progresses, the parameters of the second zero-convolutional layer gradually learn an appropriate injection strength, achieving a gradual and stable access to the structural control signal.

[0083] Compared to the existing approach of directly injecting control conditions into the backbone network through splicing or superposition, the dual zero-convolution progressive injection mechanism in this embodiment effectively avoids gradient oscillation and catastrophic forgetting problems in the early stage of training. Under the premise of preserving the original high-quality generation prior of the frozen layout semantic backbone network KANDiff, it gradually establishes the ability of structural semantics to constrain the generation process, which is a key technical means to achieve "non-intrusive control".

[0084] Furthermore, the structural feature extraction branch includes a frozen visual perception network; the extraction of multi-scale structural features for describing geometric structural semantics includes:

[0085] The structured flow image is input into the frozen visual perception network, and activation features of multiple intermediate layers at different depths in the visual perception network are extracted as the multi-scale structural features.

[0086] Shallow activation features are used to describe edge and texture structures, while deep activation features are used to describe object outlines and spatial perspective relationships.

[0087] Specifically, a structural reference image is input into a frozen visual perception network, and activation features from multiple intermediate layers of varying depths within this network are extracted. These features are then directly used as multi-scale structural features to describe the semantics of the geometric structure, instead of relying on explicit control maps such as single Canny edge maps or semantic segmentation maps used in traditional schemes. Shallow activation features are used to describe edge and texture structures, while deep activation features are used to describe object contours and spatial perspective relationships.

[0088] This technical solution based on multi-layer feature extraction from visual perception networks makes the information abundance of structural control signals significantly better than existing technologies.

[0089] Shallow layers of visual perception networks (such as the first and second intermediate layers) respond strongly to local geometric information such as edges, corners, and texture primitives in images, providing fine-grained contour constraints. Deeper layers (such as the third and fourth intermediate layers) have a larger receptive field, capturing mid-to-high-level structural semantics such as overall object contours, spatial layout relationships, and regional connectivity. Compared to traditional Canny edge detection, which only provides binary contour lines, this scheme, by combining multi-layered perceptual features from shallow and deep layers, forms a multi-scale structural semantic representation encompassing everything from local edges to global spatial relationships, providing a richer structural information foundation for subsequent encoding of trainable copy modules.

[0090] Furthermore, in this embodiment, the visual perception network remains frozen during feature extraction, meaning its network parameters are not updated during training. This design ensures that structural feature extraction is always based on stable pre-trained visual priors, maintaining consistency and predictability of the semantic space of structural semantic representation throughout the training process. This avoids structural semantic drift caused by changes in feature extraction network parameters, thus providing a stable and reliable information source for "strictly locked structure."

[0091] Furthermore, the visual perception network is a VGG network, and the multiple intermediate layers of different depths include at least a first intermediate layer, a second intermediate layer, a third intermediate layer, and a fourth intermediate layer, which correspond to the activation outputs of the relu1_1, relu2_1, relu3_1, and relu4_1 layers in the VGG network, respectively.

[0092] Specifically, this embodiment selects the VGG network as the visual perception network because the VGG network has good generalization and stability in image perception feature extraction tasks, and its hierarchical feature representation has been widely verified to effectively capture multi-scale visual information from low-level texture to high-level semantics.

[0093] Specifically, ReLU1_1 and ReLU2_1 layers, as shallower intermediate layers, have smaller receptive fields and respond strongly to local geometric information such as edges, corners, and texture primitives in the image. ReLU3_1 and ReLU4_1 layers, as deeper intermediate layers, have progressively larger receptive fields and can capture the overall outline of objects, spatial layout relationships, and occlusion and connectivity relationships between regions. By using the activation outputs of these four layers together as multi-scale structural features, a complete structural semantic representation from fine-grained edges to the global spatial skeleton can be formed.

[0094] Therefore, this embodiment sets up a visual perception network using a VGG network and defines its extraction layers as four specific intermediate layers. This ensures that the multi-scale structural features output by the structural feature extraction branch have clear technical boundaries in terms of information granularity and semantic level. This avoids both the weakening of contour constraints due to excessive abstraction of structural input information and the insufficient expression of spatial relationships due to excessive reliance on low-level textures. As a result, it provides a rich and stable structural semantic foundation for subsequent multi-scale encoding of the trainable copy module and residual injection into the decoding layer.

[0095] Furthermore, the frozen layout semantic backbone network KANDiff is the U-Net backbone network of the pre-trained latent space diffusion model; the trainable copy module replicates the topology of the encoder part of the U-Net backbone network, and its parameters are updated independently of the parameters of the frozen layout semantic backbone network KANDiff during training.

[0096] Specifically, in this embodiment, the U-Net backbone network of the pre-trained latent space diffusion model is selected as the layout semantic backbone network KANDiff because the U-Net structure, with its symmetrical encoder-decoder design and skip connection mechanism, can effectively integrate multi-scale features in the stepwise denoising generation process, and has been widely verified as the core backbone architecture of the diffusion model.

[0097] The U-Net backbone network has already learned rich prior knowledge of continuous layout semantics, such as lighting transitions, material textures, and color distribution, during the pre-training phase. By keeping its parameters frozen, this invention fully preserves this generative prior, avoiding degradation of generative capabilities when fine-tuning on small sample indoor scene data. The trainable copy module obtains a control link that is structurally isomorphic to the backbone encoding path but has independent parameters by replicating the topology of the encoder part of the U-Net backbone network. This link is specifically used for layer-by-layer encoding of multi-scale structural features from the structural feature extraction branch.

[0098] The key to this embodiment lies in the design principle of "identical topology and independent parameters." The trainable copy module and the encoder part of the frozen layout semantic backbone network KANDiff have the same hierarchical structure. This ensures that the feature maps output at each downsampling level naturally match the corresponding decoding levels of the backbone in terms of spatial resolution and channel dimension, facilitating direct injection via residual addition. Independent parameter updates mean that the trainable copy module only needs to learn how to transform structural semantics into control signal forms suitable for injection into the backbone decoding process during training, while the frozen layout semantic backbone network KANDiff focuses solely on rendering continuous layout semantics. This division of labor reduces the convergence difficulty of the trainable part and avoids interference from structural control learning on layout generation capabilities, forming a crucial structural foundation for achieving the synergistic function of "structure locking" and "layout rendering."

[0099] Furthermore, the trainable copy module performs multi-scale encoding on the multi-scale structural features to generate multi-scale structural control signals, including:

[0100] The trainable copy module extracts features layer by layer along its downsampling path, and uses the feature maps output by each downsampling level as the multi-scale structure control signal to be injected into the upsampling decoding level of the frozen layout semantic backbone network KANDiff.

[0101] Specifically, this embodiment provides a multi-scale correspondence mechanism of "layer-by-layer extraction and layer-by-layer injection".

[0102] The trainable copy module replicates the topology of the encoder portion of the frozen layout semantic backbone network KANDiff, and therefore also has multiple downsampling levels from shallow to deep.

[0103] At each level, the trainable copy module encodes the multi-scale structural features of the input at different granularities: the feature maps output by shallow downsampling layers retain more local edge and texture details, while the feature maps output by deep downsampling layers abstract higher-level geometric contours and spatial relationships. Therefore, the structural control signals output by the trainable copy module are not single-scale control information, but rather a set of multi-resolution control signals corresponding one-to-one with each decoding layer of the backbone network.

[0104] This embodiment enables structural constraints to function simultaneously at different resolution levels. During the decoding phase of the frozen layout semantic backbone network KANDiff, shallower decoding levels receive structural control signals containing rich local edge information, which are used to constrain the accuracy of the local contours and boundaries of the generated results; deeper decoding levels receive structural control signals containing spatial hierarchy and global perspective information, which are used to constrain the consistency of the overall spatial skeleton and object layout.

[0105] This embodiment injects a hierarchical structure from fine-grained to global, which effectively avoids the problems of "insufficient high-level semantic constraints leading to spatial drift" or "overly rigid low-level constraints leading to stiff textures" that may occur when relying solely on a single-scale control signal. It is an important technical means to achieve stable generation of structures in complex indoor scenes.

[0106] Furthermore, the step of injecting the multi-scale structural control signals into the corresponding decoding layers of the frozen layout semantic backbone network KANDiff in a residual addition manner includes:

[0107] The multi-scale structure control signals of each level are injected into the feature map of the corresponding decoding level in an element-wise addition manner.

[0108] Specifically, residual injection refers to the element-wise addition of the structural control signal output by the trainable replica module at any downsampling level, after necessary dimensional alignment, to the original feature map of the corresponding upsampling decoding level in the frozen layout semantic backbone network KANDiff. This injection method means that the structural control signal does not replace or cover the decoding features of the backbone network itself, but rather applies a structure-guided correction term to it. Since residual injection is essentially an additive operation, it has relatively little disturbance to the backbone feature distribution, and can smoothly introduce geometric constraints into the decoding process without destroying the existing semantic rendering capabilities of the frozen layout semantic backbone network KANDiff.

[0109] Compared to commonly used injection methods such as convolutional mapping after channel-dimensional concatenation or direct full replacement, residual addition injection has two technical advantages:

[0110] First, the addition operation preserves the original channel dimension and semantic space of the feature map, making the information fusion between the backbone decoding features and the structural control features more transparent and controllable, and facilitating the backbone network to maintain its original continuous layout semantic rendering capability. Second, under the multi-scale injection architecture, the structural control signals at each level are injected in the form of residuals, so that the strength of the structural constraints presents a natural hierarchical distribution at different resolution levels. The shallow residual signals strengthen the consistency of local contours, and the deep residual signals correct the global spatial skeleton, thereby achieving comprehensive structural constraints from local to global.

[0111] Furthermore, during the model training phase, the layout flow image is a noisy image, and its corresponding input condition is an empty mask placeholder; the structure flow image is an indoor scene value image corresponding to the noisy image.

[0112] The method updates the parameters of the structural feature extraction branch, the trainable copy module, and the progressive injection mechanism by performing loss calculations on the generated target indoor scene image and the indoor scene value image, while keeping the parameters of the frozen layout semantic backbone network KANDiff unchanged.

[0113] Specifically, training conventional diffusion models typically involves learning to complete randomly occluded regions. However, this embodiment employs an empty mask placeholder, providing a completely empty mask condition to the frozen layout semantic backbone network KANDiff. This means the model no longer learns to complete occlusions in local areas, but instead directly performs global continuous layout semantic reasoning on the indoor scene image. Simultaneously, the structural feature extraction branch extracts multi-scale structural features from the corresponding indoor scene value image, encodes them into multi-scale structural control signals via a trainable copy module, and then injects them into the backbone network. Since both the layout stream and structural stream inputs originate from the same real image, the model can learn the inherent correspondence between structural semantics and layout semantics within the image during training.

[0114] Another important feature of the training strategy in this embodiment is the selectivity of parameter updates. Throughout the training process, only the structural feature extraction branch, the trainable copy module, and the parameters related to the progressive injection mechanism participate in gradient updates, while the parameters of the frozen layout semantic backbone network KANDiff remain unchanged.

[0115] The "frozen backbone, adaptive control" training paradigm provided in this embodiment, on the one hand, fully preserves the high-quality continuous layout semantic generation capability of the pre-trained diffusion model and avoids the generation prior degradation caused by fine-tuning of small sample indoor scene data; on the other hand, by forcing the trainable part to learn to inject structural information into the decoding process of the frozen backbone, a deep coupling relationship between the structure flow and the layout flow is established without repeatedly training the basic semantic extraction capability of the backbone.

[0116] Furthermore, the structure flow image and the layout flow image are the same indoor scene image, so that the generated target indoor scene image can achieve style transfer or local editing of continuous layout semantics while keeping the geometric structure semantics unchanged in the structure flow image.

[0117] Specifically, this embodiment provides a "same image, different flow" reasoning mode. When the structural reference image and the layout reference image are the same image, this invention does not degenerate into traditional end-to-end unified modeling simply because the information sources are the same. Instead, the image is simultaneously fed into two processing chains with different functions:

[0118] On one hand, it serves as a structural reference image input to the structural feature extraction branch, where a frozen visual perception network extracts multi-scale structural features. These features are then processed by a dual zero-convolution module and a trainable copy module to generate multi-scale structural control signals, specifically responsible for locking geometric structural semantics such as wall and furniture outlines and perspective relationships. On the other hand, the same image serves as a layout reference image input to the frozen layout semantic backbone network KANDiff, which extracts layout features of continuous layout semantics such as lighting, color, and material. The two semantic paths are recombined and fused during the decoding stage of the backbone network through residual injection.

[0119] This embodiment enables the decoupled control of structural semantics and layout semantics even in a single image input scenario. When a user wishes to perform style transfer or local material adjustment only on the original indoor scene image, additional style guidance conditions can be applied to the layout semantic backbone network KANDiff, or specific preprocessing can be performed on the layout stream input to change the lighting atmosphere, color temperature tone, or material texture of the final output. At the same time, the structural flow ensures that the object outlines, spatial positions, and perspective relationships in the target indoor scene image are strictly consistent with the original image.

[0120] Example 2

[0121] This embodiment also provides an indoor scene image generation system based on dual guidance of structure and layout, used to implement the indoor scene image generation method based on dual guidance of structure and layout as described in any one of Embodiment 1, including:

[0122] At least one processor; and

[0123] A memory communicatively connected to the at least one processor; wherein,

[0124] The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enable the system to perform the following operations:

[0125] Obtain the structure flow image and layout flow image;

[0126] The layout flow image is input into the frozen layout semantic backbone network KANDiff, which performs forward propagation processing on the layout flow image to extract layout features used to describe continuous layout semantics, which include at least lighting, color and material information.

[0127] The structure flow image is input to the structure feature extraction module, which processes the structure flow image to extract multi-scale structural semantic features for describing geometric structure semantics. The geometric structure semantics include at least object contours, edges, and spatial positional relationships.

[0128] The multi-scale structural semantic features are input into the trainable replica module, which encodes the multi-scale structural semantic features at multiple scales to generate multi-scale structural control signals. The topology of the trainable replica module corresponds to at least a portion of the encoder structure of the layout semantic backbone network KANDiff.

[0129] The multi-scale structural control signal is progressively injected into the corresponding decoding layer of the layout semantic backbone network KANDiff through the progressive injection module, so that the generated target indoor scene image inherits the geometric structural semantics of the structure flow image and the continuous layout semantics of the layout flow image.

[0130] Furthermore, Figure 2 This embodiment illustrates the overall architecture of the image generation system guided by both structure and layout principles. The system includes:

[0131] Layout semantic backbone module: A pre-trained diffusion backbone network is used, preferably a latent space diffusion model backbone containing a U-Net structure, to be responsible for modeling and rendering continuous layout semantics such as lighting, color temperature, material texture, and spatial atmosphere. This layout semantic backbone network KANDiff is kept frozen in this invention.

[0132] Structural feature extraction module: used to extract multi-level structure-aware features from the structure flow image. In a preferred embodiment, a frozen VGG visual perception network is used to extract activation features from multiple intermediate layers of different depths. Figure 3The diagram illustrates the extraction process, in which activation features of the first, second, third, and fourth intermediate layers are extracted, corresponding to the activation outputs of the ReLU1_1, ReLU2_1, ReLU3_1, and ReLU4_1 layers, respectively, to obtain multi-scale structural features from shallow edge textures to deep geometric contours.

[0133] The dual zero-convolution alignment module maps multi-scale structural features to a feature space compatible with the backbone input and progressively controls the injection strength of structural features to avoid excessive perturbation to the pre-trained backbone distribution. This module includes a first zero-convolutional layer and a second zero-convolutional layer. The first zero-convolutional layer is used for dimension mapping and feature alignment, while the second zero-convolutional layer is used for cold-start gating.

[0134] Trainable copy module: The encoder topology of the layout semantic backbone network KANDiff is copied to form a trainable control link, which is responsible for multi-scale encoding of multi-scale structural features to obtain structural control signals suitable for different resolution levels.

[0135] Multi-scale residual injection module: The structural control signals output by the trainable copy module are injected into the corresponding decoding layers of the frozen layout semantic backbone network KANDiff by adding the residuals, so as to constrain the geometric structure of the generated result during the decoding stage.

[0136] Output image generation module: Based on the fused multi-scale features, output the final target indoor scene image so that the result simultaneously meets structural constraints and layout style requirements.

[0137] Example 3

[0138] Based on Example 1, this example provides specific steps of the present invention in the training phase. The core of the training phase is: without changing the optimization objective of the original pre-trained generative model, by constructing a specific self-supervised data stream and input strategy, the trainable copy module is forced to learn how to inject structural information into the frozen layout semantic backbone network KANDiff, thereby establishing a deep coupling relationship between structure and layout.

[0139] Step S1: Input data construction.

[0140] Acquire training images and construct two types of inputs from the same real indoor scene image: layout flow input to provide continuous layout semantic information; and structure flow input to provide structural semantic information.

[0141] For the layout stream input, an indoor scene image and a null mask placeholder are used (required for the KANDiff model input with frozen parameters). Specifically, the null mask placeholder strategy is adopted so that the frozen layout semantic backbone network KANDiff no longer repeats the local occlusion completion process, but directly performs global layout semantic reasoning on the entire noisy image.

[0142] For the structure stream input, the same indoor scene image is used as the structure stream image and input into the structure feature extraction branch to obtain multi-scale structure features through the frozen visual perception network (VGG).

[0143] Step S2: Freeze the layout semantic backbone.

[0144] The parameters of the pre-trained layout semantic backbone network (preferably the KANDiff model backbone network) are fixed and not updated, retaining only its forward propagation capability. This frozen layout semantic backbone network, KANDiff, is responsible for learning and maintaining continuous layout semantic representations, including changes in lighting gradients, color temperature transitions, material texture continuity, and overall consistency of spatial atmosphere. By freezing the backbone, the original generation capability is avoided from being compromised on small sample home decoration data.

[0145] Step S3: Extract structure-aware features.

[0146] The structured flow image is input into the structured feature extraction branch to obtain multi-level structural semantic features. These structural features mainly describe geometric structural information such as furniture outlines, wall boundaries, perspective skeletons, occlusion relationships, and edge and corner distribution, without directly including semantic information about continuous layout such as color, lighting, and material.

[0147] In a preferred embodiment, the structural feature extraction branch includes a frozen VGG visual perception network. The structure flow image is input into this VGG network, and the activation outputs of the first intermediate layer (relu1_1), the second intermediate layer (relu2_1), the third intermediate layer (relu3_1), and the fourth intermediate layer (relu4_1) are extracted as multi-scale structural features. The shallow activation features (relu1_1, relu2_1) are used to describe edge and texture structures, while the deep activation features (relu3_1, relu4_1) are used to describe object contours and spatial perspective relationships.

[0148] Step S4: Trainable copies are used for multi-scale structure encoding.

[0149] Before inputting multi-scale structural features into the trainable copy module, double zero-convolution feature alignment is performed:

[0150] The first zero-convolutional layer performs dimensionality mapping on the multi-scale structural features, mapping them to a space with the same dimension as the input features of the layout semantic backbone network KANDiff. Then, the mapped features are added and fused with the input features of the layout semantic backbone network KANDiff to complete feature pre-fusion. The second zero-convolutional layer performs gating processing on the pre-fused features to make the increment of the structural control signal zero in the early stage of training.

[0151] The features processed by the dual zero-convolution module are input into the trainable copy module. This trainable copy module is topologically identical to the encoder of the frozen layout semantic backbone network KANDiff, extracting structural control features at different scales layer by layer along the downsampling path. At each layer, the trainable copy module outputs a structural control signal at the corresponding resolution, used to represent the structural semantics from shallow edge textures to deep object contours.

[0152] Step S5: Injection of multi-scale structural control signal residuals.

[0153] The structural control signals obtained in step S4 are injected element-wise into the feature maps of the corresponding upsampled decoding layers in the frozen layout semantic backbone network KANDiff. This multi-level, multi-resolution injection scheme enables the model to simultaneously obtain shallow contour constraints and deep spatial structure constraints at different resolution levels, achieving continuous guidance and layer-by-layer correction of the generation process by structural constraints.

[0154] Step S6: Reconstruction and Optimization.

[0155] The target indoor scene image is reconstructed based on the fused feature output, and the loss is calculated by comparing the generated target indoor scene image with the ground truth indoor scene image from step S1. During training, only the parameters of the trainable copy modules, the parameters of the dual zero convolution modules, and the additional parameters related to structure control are updated; the parameters of the frozen layout semantic backbone network KANDiff are always left unchanged.

[0156] Through the aforementioned empty mask occupancy strategy, the frozen layout semantic backbone network KANDiff directly performs global layout semantic reasoning on indoor scene images. The structural feature extraction branch extracts structural features from the same indoor scene image and participates in reconstruction. As a result, the model is forced to learn how to inject structural information into layout semantics, establishing a deep coupling relationship between structural flow and layout flow, without having to repeatedly train the model's existing basic semantic extraction capabilities.

[0157] Example 4

[0158] Based on Example 1, this example presents a flowchart of the specific steps in the inference stage of the present invention. The core of the inference stage lies in: providing a geometric skeleton using a structure flow image and providing environmental atmosphere and continuous visual semantics using a layout flow image, and then fusing the two within the model to generate a new image that simultaneously inherits the structural features of the former and the stylistic atmosphere of the latter.

[0159] Step T1: Receive two reference images.

[0160] The system receives two reference images from different sources: a structure flow image to provide object outlines, spatial skeletons, perspective, and edge constraints; and a layout flow image to provide ambient lighting, color temperature, material atmosphere, and overall spatial texture. The structure flow image and layout flow image can be different images to achieve cross-image structure-layout semantic combination generation; or they can be the same indoor scene image to achieve style transfer or local editing while preserving the structure of the same image.

[0161] Step T2: Extract semantic information from two channels in parallel.

[0162] The structured flow image is processed by the structure feature extraction branch to extract multi-scale structural features; the multi-scale structural features are then processed by the dual zero convolution alignment module for dimension mapping, pre-fusion and gating, and then input into the trainable copy module; the trainable copy module extracts features layer by layer along the downsampling path to generate multi-scale structural control signals.

[0163] The layout flow image is fed into the frozen layout semantic backbone network KANDiff, and layout features of continuous layout semantics are extracted through forward propagation.

[0164] Step T3: Generation of cross-image semantic reorganization.

[0165] The multi-scale structural control signals generated by the trainable copy module are injected into the corresponding decoding layers of the frozen layout semantic backbone network KANDiff in an element-wise additive residual manner. The frozen layout semantic backbone network KANDiff decodes the layout features and the injected structural control signals to generate the final target indoor scene image.

[0166] The target indoor scene image inherits the furniture geometry, spatial position and perspective relationship from the structure flow image, as well as the lighting atmosphere, color temperature style and material texture from the layout flow image, to achieve controllable recombination and generation of "structure from image A, layout from image B".

[0167] Example 5

[0168] Based on Example 1, this example adds an editable area mask, allowing structural control and layout reorganization only in user-specified areas, while unedited areas remain unchanged. This variation can be used for furniture replacement, soft furnishing renovation, partial style migration, and partial material adjustment, with the common goal of maintaining a natural overall layout while ensuring controllable local structures.

[0169] Example 6

[0170] This embodiment combines Embodiment 1 with the retrieval module, asset library module, and interactive design system to automatically generate multiple layout schemes based on floor plans, user preferences, and the furniture asset library. The common goal remains the same: to leverage the decoupling control capability between structure and layout to ensure that the generated results both meet spatial framework constraints and possess diverse visual styles.

[0171] Example 7

[0172] This embodiment provides an alternative to the structural information extraction method in Embodiment 1. The structural feature extraction branch can be implemented using various alternative methods, including: using the resulting edge map as the structural control input based on explicit edge extraction methods such as Canny and HED; expressing spatial structure based on semantic segmentation maps, region label maps, or instance segmentation results using different region categories and positional relationships; expressing spatial structural relationships based on geometric priors such as depth maps, surface normal maps, and vanishing line maps; expressing object position and contour relationships based on keypoints, skeleton maps, detection boxes, and layout boxes; and extracting multi-scale structural features using other visual perception networks such as ResNet, HRNet, and SwinTransformer. The common feature of these alternative methods is that they all obtain richer hierarchical spatial information than simple edge maps through pre-trained visual networks.

[0173] Example 8

[0174] This embodiment addresses an alternative to the structural conditional injection method in Embodiment 1. The progressive injection mechanism of double zero convolution can also be replaced by other methods, including: direct convolutional mapping after splicing, element-wise addition or residual injection, lightweight adapter methods, cross-attention methods, low-rank adaptation or parameter-efficient fine-tuning methods, FiLM-style feature modulation methods, and spatial adaptive normalization methods such as SPADE. The commonality among these alternative methods is that they all belong to the technical route of "modulating intermediate activations using external conditions," aiming to introduce new control capabilities through additional modules with minimal modification to the main structure.

[0175] Example 9

[0176] This embodiment addresses an alternative implementation of the training paradigm in Embodiment 1. The dual-stream training paradigm of "frozen trunk + trainable copy" can also be implemented in other ways, including: freezing the trunk and training only lightweight control branches, partially freezing and partially fine-tuning, full fine-tuning with constraint regularization, distillation, or two-stage training methods. These methods differ in their implementation paths, but none change the overall idea of ​​this invention.

[0177] Test Implementation Examples

[0178] This embodiment is generated and tested based on the above embodiments. Figure 4 The qualitative experimental results of cross-semantic combination generation are shown. Column (a) is the layout flow input (layout flow image), column (b) is the generated target indoor scene image, and column (c) is the structure flow input (structure flow image).

[0179] observe Figure 4 As can be seen from the generated results, this invention demonstrates significant structure-locking capabilities. In all test samples, the generated target indoor scene images not only accurately reproduced the object categories in the structural input but also rigorously preserved their specific geometric shapes and spatial positions. Figure 4 Taking the fourth row of samples as an example, the two orange armchairs and the semi-transparent partition behind them in the input structure are precisely replicated in the generated image, with pixel-level accuracy in terms of curvature, placement angle, and mutual occlusion. Similarly, in the third row, although the layout input is an open natural landscape, the generated image still strictly adheres to the rigid lines and cabinet layout of a modern kitchen defined by the structural input. This high fidelity demonstrates that in the dual-stream architecture of this invention, the trainable copy module successfully extracts and injects strong geometric constraints, effectively preventing the structural drift phenomenon common in style transfer tasks.

[0180] While ensuring structural stability, this invention also demonstrates flexible layout semantic transfer capabilities, enabling the re-rendering of continuous lighting and atmospheric features extracted from the layout input onto the locked structural skeleton. As shown in line 2, an extreme heavy lighting case demonstrates how the model successfully transferred the bright, luxurious warm-toned atmosphere from the layout input to the originally dim industrial-style bedroom structure, completely altering the scene's emotional tone. Furthermore, the model exhibits clear mechanistic characteristics when handling more complex semantic conflicts. As shown in line 5, when a strong demand for a bright, minimalist layout conflicts with the rich wood and stone textures in the original structure, the generated result prioritizes the consistency of the global atmosphere, blending the entire scene into a bright, minimalist style. This priority response to globally continuous layout semantics precisely verifies that the frozen layout semantic backbone network KANDiff focuses on processing continuously distributed semantic information such as lighting and color, and this capability can operate independently of the geometric structure.

[0181] In summary, the qualitative experimental results demonstrate that this invention successfully achieves a controllable cross-semantic decoupling and fusion. It not only theoretically proves the feasibility of decoupling indoor scenes into structural semantics and continuously refined layout semantics, but also practically demonstrates the powerful ability to create entirely new high-fidelity scenes by combining semantic information from different sources. Although there is still room for improvement in material details under certain extreme lighting conditions, its core structural and layout decoupling control mechanism has achieved its intended goals.

[0182] The above are only some embodiments of this application and do not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A method for generating indoor scene images based on dual guidance of structure and layout, characterized in that, include: Obtain the structure flow image and layout flow image; The layout flow image is input into the frozen layout semantic backbone network, and the frozen layout semantic backbone network performs forward propagation processing on the layout flow image to extract layout features used to describe continuous layout semantics, which include at least lighting, color and material information. The structure flow image is input to the structure feature extraction branch, which processes the structure flow image to extract multi-scale structural features for describing geometric structure semantics. The geometric structure semantics include at least object contours, edges, and spatial positional relationships. The multi-scale structural features are input into the trainable replica module, which performs multi-scale encoding on the multi-scale structural features to generate multi-scale structural control signals corresponding to multiple decoding levels in the layout semantic backbone network. A progressive injection mechanism is adopted to inject the multi-scale structural control signals into the corresponding decoding layers of the frozen layout semantic backbone network by adding residuals. The frozen layout semantic backbone network decodes the layout features and the injected multi-scale structure control signal to generate a target indoor scene image, so that the target indoor scene image inherits the geometric structure semantics of the structure flow image and fuses the continuous layout semantics of the layout flow image.

2. The indoor scene image generation method based on dual guidance of structure and layout according to claim 1, characterized in that, The progressive injection mechanism is a dual zero-convolution injection mechanism, which includes: Before inputting the multi-scale structural features into the trainable replica module, the multi-scale structural features are first dimension-mapped through a first zero convolutional layer and then fused with the input features of the layout semantic backbone network. The fused features are then gating through a second zero convolutional layer to make the increment of the structural control signal zero in the early stage of training. After that, the processed features are input into the trainable replica module.

3. The indoor scene image generation method based on dual guidance of structure and layout according to claim 1, characterized in that, The structural feature extraction branch includes a frozen visual perception network; The extraction of multi-scale structural features for describing the semantics of geometric structures includes: The structured flow image is input into the frozen visual perception network, and activation features of multiple intermediate layers at different depths in the visual perception network are extracted as the multi-scale structural features. Shallow activation features are used to describe edge and texture structures, while deep activation features are used to describe object outlines and spatial perspective relationships.

4. The indoor scene image generation method based on dual guidance of structure and layout according to claim 3, characterized in that, The visual perception network is a VGG network, and the multiple intermediate layers of different depths include at least a first intermediate layer, a second intermediate layer, a third intermediate layer, and a fourth intermediate layer, which correspond to the activation outputs of the relu1_1, relu2_1, relu3_1, and relu4_1 layers in the VGG network, respectively.

5. The indoor scene image generation method based on dual guidance of structure and layout according to claim 1, characterized in that, The frozen layout semantic backbone network is the U-Net backbone network of the pre-trained latent space diffusion model; the trainable copy module replicates the topology of the encoder part of the U-Net backbone network, and its parameters are updated independently of the parameters of the frozen layout semantic backbone network during training.

6. The indoor scene image generation method based on dual guidance of structure and layout according to claim 5, characterized in that, The trainable replica module performs multi-scale encoding on the multi-scale structural features to generate multi-scale structural control signals, including: The trainable replica module extracts features layer by layer along its downsampling path, and uses the feature maps output by each downsampling level as the multi-scale structure control signal to be injected into the corresponding upsampling decoding level of the frozen layout semantic backbone network.

7. The method for generating indoor scene images based on dual guidance of structure and layout according to claim 1, characterized in that, The step of injecting the multi-scale structural control signals into the corresponding decoding layers of the frozen layout semantic backbone network by adding residuals includes: The multi-scale structure control signals of each level are injected into the feature map of the corresponding decoding level in an element-wise addition manner.

8. The method for generating indoor scene images based on dual guidance of structure and layout according to claim 1, characterized in that, During the model training phase, the layout flow image is a noisy image, and its corresponding input condition is an empty mask placeholder; the structure flow image is an indoor scene value image corresponding to the noisy image. The method updates the parameters of the structural feature extraction branch, the trainable copy module, and the progressive injection mechanism by performing loss calculations on the generated target indoor scene image and the indoor scene value image, while keeping the parameters of the frozen layout semantic backbone network unchanged.

9. The method for generating indoor scene images based on dual guidance of structure and layout according to claim 1, characterized in that, The structure flow image and the layout flow image are the same indoor scene image, so that the generated target indoor scene image can achieve style transfer or local editing of continuous layout semantics while keeping the geometric structure semantics unchanged in the structure flow image.

10. An indoor scene image generation system based on dual guidance of structure and layout, used to implement the indoor scene image generation method based on dual guidance of structure and layout as described in any one of claims 1 to 9, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enable the system to perform the following operations: Obtain the structure flow image and layout flow image; The layout flow image is input into the frozen layout semantic backbone network, and the frozen layout semantic backbone network performs forward propagation processing on the layout flow image to extract layout features used to describe continuous layout semantics, which include at least lighting, color and material information. The structure flow image is input to the structure feature extraction module, which processes the structure flow image to extract multi-scale structural semantic features for describing geometric structure semantics. The geometric structure semantics include at least object contours, edges, and spatial positional relationships. The multi-scale structural semantic features are input into the trainable replica module, which performs multi-scale encoding on the multi-scale structural semantic features to generate multi-scale structural control signals. The topology of the trainable replica module corresponds to at least a portion of the encoder structure of the layout semantic backbone network. The multi-scale structural control signal is progressively injected into the corresponding decoding layer of the layout semantic backbone network through the progressive injection module, so that the generated target indoor scene image inherits the geometric structural semantics of the structural flow image and the continuous layout semantics of the layout flow image.

Images