A scale-aware indoor home design graph generation system and method

Abstract

The application provides a scale-aware indoor home design drawing generation system and method, the system comprises a global 3D space processing module, a local object space processing module, a cross attention module and a feature fusion module. By introducing the fusion mechanism of global and local features, and using the Multi-controlNet strategy combined with the diffusion model, the scale information of the objects in the indoor scene can be accurately captured, the global space constraint and the local geometric details can be fully utilized, and the control ability of the proportion coordination in the generation process can be enhanced. The method of comprehensively considering the global space characteristics and the local scale difference can significantly improve the size perception and proportion accuracy of the indoor design drawing generation, and can provide a more real, accurate and efficient solution for the design generation task of the related industry.

Description

Technical Field

[0001] This invention relates to the field of design drawing generation technology, and in particular to a scale-aware interior design drawing generation system and method. Background Technology

[0002] As consumers increasingly demand personalized and high-precision design results, the interior design industry is placing higher demands on the realism and accuracy of design renderings. Traditional design models typically rely solely on floor plans or simple 3D models to generate renderings, methods that have significant limitations in expressing spatial proportions and scale. For example, furniture or spaces in a design rendering may appear visually harmonious, but in actual placement, discrepancies in size and proportion can lead to design deviations.

[0003] Most current generation techniques focus on the visual appeal of the rendering while neglecting the expression of spatial scale. This approach can easily lead to discrepancies between the design and the actual scene, increasing the cost of design modifications and implementation. Therefore, how to achieve a true sense of spatial scale during the design rendering process has become an urgent problem to be solved.

[0004] As consumers increasingly demand personalized and high-precision design results, the interior design industry is placing higher demands on the realism, accuracy, and spatial proportions of design renderings. However, existing methods for generating design renderings still have many limitations and cannot fully meet these needs.

[0005] In traditional design processes, designers typically rely on floor plans or simple 3D models to generate renderings. While these methods can convey design intent to some extent, they fall short in expressing spatial proportions and scale. For example, furniture or decorations in a design rendering may appear visually harmonious, but in reality, discrepancies in size and proportion can lead to discrepancies in the design's appearance. This discrepancy not only increases the workload of design modifications but also leads to increased costs and reduced efficiency during project implementation.

[0006] In recent years, the diffusion model (Ho J, Jain A, Abbeel P. Denoising diffusion probabilistic models[J]. Advances in neural information processing systems, 2020, 33: 6840-6851.) has been widely used as a generative model in design drawing generation tasks. The diffusion model can generate beautiful images through efficient probabilistic modeling, showing significant advantages in rendering effects. However, these methods focus more on the visual aesthetics of the design drawings, while being relatively weak in expressing spatial dimensions and proportions, making it difficult to meet the real-world requirements for physical dimensions in actual scenarios.

[0007] To enhance the controllability of design generation, some existing methods attempt to guide the generation process using control conditions such as ControlNet (Zhang L, Rao A, Agrawala M. Adding conditional control to text-to-image diffusion models[C] / / Proceedings of the IEEE / CVF International Conference on Computer Vision.2023: 3836-3847.). These methods improve the orientation capability of generation to some extent by introducing control information such as 2D images, sketches, or depth maps. However, since these guidance methods are limited to 2D image information, they cannot fully integrate 3D space and physical dimensions, resulting in insufficient spatial perception and dimensional representation of real scenes. For example, although 2D guidance can constrain the contour and layout of the generated result, it still has significant shortcomings in expressing the true geometric relationships, spatial scale, and positional consistency of objects.

[0008] Therefore, how to combine three-dimensional spatial information in the generation of design drawings to achieve accurate perception of the size and proportion of objects has become a technical problem that urgently needs to be solved. Therefore, we propose an interior home design drawing generation system and method based on scale perception. Summary of the Invention

[0009] This invention proposes a scale-aware interior design drawing generation system and method to solve the problems mentioned in the background art.

[0010] To achieve the above objectives, the present invention adopts the following technical solution: A scale-aware interior design drawing generation system includes a global 3D space processing module, a local object space processing module, a cross-attention module, and a feature fusion module. The global 3D space processing module and the local object space processing module are both connected to the cross-attention module, and the cross-attention module is also connected to the feature fusion module. The global 3D space processing module introduces a U-Net feature extraction network to encode the 3D space model and its corresponding viewpoint information, extract its global features, and send them to the cross-attention module. The local object space processing module embeds each object in the indoor scene using an object encoder and sends it to the cross-attention module. The cross-attention module uses a cross-attention mechanism to fuse global features and local object embeddings. The cross-attention module processes through a multi-layer network to ensure effective interaction between global and local information. The feature fusion module incorporates the Multi-ControlNet mechanism, which combines global and local dimensional features with the corresponding layers in the Diffusion model that generates the design drawings. This enables the introduction of dimensional information during the design drawing generation process, thereby generating interior design drawings with accurate dimensional information.

[0011] Preferably, the cross-attention module also incorporates global size and layout information into the local object embedding to further optimize the local size information.

[0012] This invention also proposes a method for generating scale-aware interior design drawings, comprising the following steps: S1: Data preparation, as detailed below: S101: Input global 3D spatial data G={M,V}, where M represents the 3D model and V represents the rendering view information of the design drawing; S102: Local object data is represented as L={(l i ,g i ,t i ,p i ,v i )} N i=1 , where l i For semantic labels of objects, such as "sofa", g i For the geometric data of the object, t i For the material texture of an object, p i Let v be the spatial position of the object (x, y, z). i This refers to the object's perspective information; S2: Global Feature Extraction: Extracting global features using a deep convolutional neural network. : in Indicates a global encoder; S3: Local object feature extraction. For each object i, an object encoder is used to extract feature embeddings and obtain local features. : Among them, object encoder E object include: 3D Variational Autoencoder: Extracting Low-Dimensional Features from Geometry and Material: F gi , F ti =VAE 3d (g i ,t i ), Where F gi and F ti These represent the geometric and material characteristics of an object, respectively. Text encoder: Converts semantic labels of objects into embedding vectors. F li =E text (l i ); Position feature encoding: using the three-dimensional position p of the object i and perspective v i Generate spatial relationship features: F pi,vi =E pos (p i ,v i ); The final output local feature is: F local, i =F gi +F ti +F li +F pi,vi ; S4: The cross-attention module fuses global and local spatial information to ensure effective interaction between global and local information; S5: Through the Multi-ControlNet mechanism, efficient fusion of global and local size features is achieved in each generation layer of the Diffusion model; S6: Introducing the Multi-ControlNet mechanism, global and local dimensional features are combined with the corresponding layers in the Diffusion model that generates the design drawings, thereby introducing dimensional information during the design drawing generation process and generating interior design drawings with accurate dimensional information.

[0013] Preferably, the specific steps of S4 are as follows: S401: Transfer global feature F global and the set of local object features {F local,i} N i=1 Input cross-attention mechanism; S402: Calculate the similarity matrix between global features and local features, using the following formula: A = softmax( ), Q=F global , Where Q and K are Query and Key respectively, and d is the feature dimension of K; S403: Generate joint features by weighted summation of local features based on weight matrix A: in Value.

[0014] Preferably, the specific steps of S5 are as follows: S501: Global Features and local features Each layer's feature map is obtained by processing through different control branches. and : , S502: Mapping the two feature maps to the intermediate features of the Diffusion model's generation layer. Combining these, the formula is as follows: , Where α and β are the fusion weights.

[0015] Preferably, the specific steps of S6 are as follows: S601: Noise Initialization: Input initial Gaussian noise image ,in This represents the number of time steps in the diffusion process. S602: Global and Local Feature Fusion: Integrating global features and local features The Multi-ControlNet control branch is combined with each generation layer of the diffusion model: in , The weight adjustment parameters at time step t; and These are the control outputs for global and local features, respectively; These are intermediate features of the t-th layer in the diffusion model; S603: Stepwise noise reduction generation: Based on the reverse process formula of the diffusion model, the initial noise is... Denoising is performed step by step to generate the design image: in ϵ is the denoising network for the diffusion model, where ϵ is the added sampling noise used to prevent mode collapse; S604: Final output: After T-step noise reduction, a design drawing with true proportions and dimensions is generated.

[0016] Compared with existing technologies, this invention introduces a fusion mechanism of global and local features and adopts a Multi-controlNet strategy combined with a diffusion model. This enables it to accurately capture the scale information of objects in indoor scenes, make full use of global spatial constraints and local geometric details, and enhance the control over proportional coordination during the generation process. This method, which comprehensively considers global spatial characteristics and local scale differences, significantly improves the size perception and proportional accuracy of indoor design drawing generation, and can provide a more realistic, accurate and efficient solution for design generation tasks in related industries. Attached Figure Description

[0017] Figure 1 This is a connection block diagram of a scale-aware interior home design drawing generation system proposed in this invention; Figure 2 This is a schematic diagram of the object encoder structure in a scale-aware interior home design drawing generation system proposed in this invention. Detailed Implementation

[0018] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0019] Reference Figure 1-2 A scale-aware interior design drawing generation system includes a global 3D space processing module, a local object space processing module, a cross attention module, and a feature fusion module. The global 3D space processing module and the local object space processing module are both connected to the cross attention module, and the cross attention module is also connected to the feature fusion module. The global 3D space processing module introduces a U-Net feature extraction network to encode the 3D space model and its corresponding viewpoint information, extract its global features, and send them to the cross-attention module. The local object space processing module embeds each object in the indoor scene using an object encoder and sends it to the cross-attention module. The cross-attention module uses a cross-attention mechanism to fuse global features and local object embedding. The cross-attention module ensures effective interaction between global and local information through multi-layer network processing. The cross-attention module also introduces global size and layout information in local object embedding to further optimize local size information. The feature fusion module incorporates the Multi-ControlNet mechanism, which combines global and local dimensional features with the corresponding layers in the Diffusion model that generates the design drawings. This enables the introduction of dimensional information during the design drawing generation process, thereby generating interior design drawings with accurate dimensional information.

[0020] This embodiment also proposes a method for generating scale-aware interior design drawings, including the following steps: S1: Data preparation, as detailed below: S101: Input global 3D spatial data G={M,V}, where M represents the 3D model and V represents the rendering view information of the design drawing; S102: Local object data is represented as L={(l i ,g i ,t i ,p i ,v i )} N i=1 , where l i For semantic labels of objects, such as "sofa", g i For the geometric data of the object, t i For the material texture of an object, p i Let v be the spatial position (x, y, z) of the object. i This refers to the object's viewpoint information. S2: Global Feature Extraction: Extracting global features using a deep convolutional neural network. :

[0021] in Indicates a global encoder; S3: Local object feature extraction. For each object i, an object encoder is used to extract feature embeddings and obtain local features. :

[0022] Among them, object encoder E object include: 3D Variational Autoencoder: Extracting Low-Dimensional Features from Geometry and Material: F gi , F ti =VAE3d (g i ,t i ), Where F gi and F ti These represent the geometric and material characteristics of an object, respectively. Text encoder: Converts semantic labels of objects into embedding vectors. F li =E text (l i ); Position feature encoding: using the three-dimensional position p of the object i and perspective v i Generate spatial relationship features: F pi,vi =E pos (p i ,v i ); The final output local feature is: F local, i =F gi +F ti +F li +F pi,vi ; S4: The cross-attention module fuses global and local spatial information to ensure effective interaction between global and local information. The specific steps are as follows: S401: Transfer global feature F global and the set of local object features {F local,i} N i=1 Input cross-attention mechanism; S402: Calculate the similarity matrix between global features and local features, using the following formula: A = softmax( ), Q=F global ,

[0023] Where Q and K are Query and Key respectively, and d is the feature dimension of K; S403: Generate joint features by weighted summation of local features based on weight matrix A:

[0024] in Value; S5: Through the Multi-ControlNet mechanism, efficient fusion of global and local size features is achieved in each generation layer of the Diffusion model. The specific steps are as follows: S501: Global Features and local features Each layer's feature map is obtained by processing through different control branches. and : ,

[0025] S502: Mapping the two feature maps to the intermediate features generated by the Diffusion model layer. Combining these, the formula is as follows: , Where α and β are the fusion weights; S6: Introducing the Multi-ControlNet mechanism, global and local dimensional features are combined with the corresponding layers in the Diffusion model that generates the design drawing, thereby incorporating dimensional information during the design drawing generation process. The specific steps are as follows: S601: Noise Initialization: Input initial Gaussian noise image ,in This represents the number of time steps in the diffusion process. S602: Global and Local Feature Fusion: Integrating global features and local features The Multi-ControlNet control branch is combined with each generation layer of the diffusion model: , in , The weight adjustment parameters at time step t; and These are the control outputs for global and local features, respectively; These are intermediate features of the t-th layer in the diffusion model; S603: Stepwise noise reduction generation: Based on the reverse process formula of the diffusion model, the initial noise is... Denoising is performed step by step to generate the design image: in ϵ is the denoising network for the diffusion model, where ϵ is the added sampling noise used to prevent mode collapse; S604: Final output: After T-step noise reduction, a design drawing with true proportions and dimensions is generated.

[0026] It should be noted that this embodiment, by introducing a fusion mechanism of global and local features and adopting the Multi-controlNet strategy combined with a diffusion model, can accurately capture the scale information of objects in indoor scenes, make full use of global spatial constraints and local geometric details, and enhance the control of proportional coordination during the generation process. This method, which comprehensively considers global spatial characteristics and local scale differences, significantly improves the size perception and proportional accuracy of indoor design drawing generation, and can provide a more realistic, accurate and efficient solution for design generation tasks in related industries.

[0027] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A scale-aware interior design drawing generation system, characterized in that, It includes a global 3D space processing module, a local object space processing module, a cross attention module, and a feature fusion module. The global 3D space processing module and the local object space processing module are both connected to the cross attention module, and the cross attention module is also connected to the feature fusion module. The global 3D space processing module introduces a U-Net feature extraction network to encode the 3D space model and its corresponding viewpoint information, extract its global features, and send them to the cross-attention module. The local object space processing module embeds each object in the indoor scene using an object encoder and sends it to the cross-attention module. The cross-attention module uses a cross-attention mechanism to fuse global features and local object embeddings. The cross-attention module processes through a multi-layer network to ensure effective interaction between global and local information. The feature fusion module incorporates the Multi-ControlNet mechanism, which combines global and local dimensional features with the corresponding layers in the Diffusion model that generates the design drawings. This enables the introduction of dimensional information during the design drawing generation process, thereby generating interior design drawings with accurate dimensional information.

2. The scale-aware interior design drawing generation system according to claim 1, characterized in that, The cross-attention module also incorporates global size and layout information into local object embedding to further optimize local size information.

3. A method for generating scale-aware interior design drawings, applied to the system described in any of claims 1-2, characterized in that, Includes the following steps: S1: Data preparation, as detailed below: S101: Input global 3D spatial data G={M,V}, where M represents the 3D model and V represents the rendering view information of the design drawing; S102: Local object data is represented as L={(l i ,g i ,t i ,p i ,v i )} N i=1 , where l i For semantic labels of objects, such as "sofa", g i For the geometric data of the object, t i For the material texture of an object, p i Let v be the spatial position of the object (x, y, z). i This refers to the object's viewpoint information. S2: Global Feature Extraction: Extracting global features using a deep convolutional neural network. : in Indicates a global encoder; S3: Local object feature extraction. For each object i, an object encoder is used to extract feature embeddings and obtain local features. : Among them, object encoder E object include: 3D Variational Autoencoder: Extracting Low-Dimensional Features from Geometry and Material: F gi , F ti =VAE 3d (g i ,t i ), Where F gi and F ti These represent the geometric and material characteristics of an object, respectively. Text encoder: Converts semantic labels of objects into embedding vectors. F li =E text (l i ); Position feature encoding: using the three-dimensional position p of the object i and perspective v i Generate spatial relationship features: F pi,vi =E pos (p i ,v i ); The final output local feature is: F local, i =F gi +F ti +F li +F pi,vi ; S4: The cross-attention module fuses global and local spatial information to ensure effective interaction between global and local information; S5: Through the Multi-ControlNet mechanism, efficient fusion of global and local size features is achieved in each generation layer of the Diffusion model; S6: Introducing the Multi-ControlNet mechanism, global and local dimensional features are combined with the corresponding layers in the Diffusion model that generates the design drawings, thereby introducing dimensional information during the design drawing generation process and generating interior design drawings with accurate dimensional information.

4. The method for generating scale-aware interior design drawings according to claim 3, characterized in that, The specific steps of S4 are as follows: S401: Transfer global feature F global and the set of local object features {F local,i } N i=1 Input cross-attention mechanism; S402: Calculate the similarity matrix between global features and local features, using the following formula: A = softmax( ), Q=F global , Where Q and K are Query and Key respectively, and d is the feature dimension of K; S403: Generate joint features by weighted summation of local features based on weight matrix A: in Value.

5. The method for generating scale-aware interior design drawings according to claim 4, characterized in that, The specific steps of S5 are as follows: S501: Global Features and local features Each layer's feature map is obtained by processing through different control branches. and : ， S502: Mapping the two feature maps to the intermediate features generated by the Diffusion model layer. Combining these, the formula is as follows: ， Where α and β are the fusion weights.

6. The method for generating scale-aware interior design drawings according to claim 5, characterized in that, The specific steps of S6 are as follows: S601: Noise Initialization: Input initial Gaussian noise image ,in This represents the number of time steps in the diffusion process. S602: Global and Local Feature Fusion: Integrating global features and local features The Multi-ControlNet control branch is combined with each generation layer of the diffusion model: in , The weight adjustment parameters at time step t; and These are the control outputs for global and local features, respectively; These are intermediate features of the t-th layer in the diffusion model; S603: Stepwise noise reduction generation: Based on the reverse process formula of the diffusion model, the initial noise is... Denoising is performed step by step to generate the design image: in ϵ is the denoising network for the diffusion model, where ϵ is the added sampling noise used to prevent mode collapse; S604: Final output: After T-step noise reduction, a design drawing with true proportions and dimensions is generated.

Images