A method and device for generating a three-dimensional model of a building space layout

Abstract

The present application provides a method and device for generating a three-dimensional model of a building space layout, the method comprising: obtaining a plane structure diagram of a target building and text description information for the layout design requirements of the target building; generating a layout fusion feature vector based on the plane layout features of the plane structure diagram and the text layout features of the text description information; inputting the layout fusion feature vector into a pre-trained space layout generation model to obtain the space layout of the target building, and generating a scene function matrix based on the space layout, the layout fusion feature vector and the text layout features; inputting the scene function matrix into a pre-trained three-dimensional generative adversarial network to generate a three-dimensional model of the space layout of the target building. Through the method and device, a three-dimensional space layout model with reasonable space, complete functions and meeting the requirements is generated on the basis of retaining the basic structure of the building.

Description

Technical Field

[0001] The present application relates to the field of space planning technology, and in particular to a method and device for generating a three-dimensional model of a building space layout. Background Art

[0002] The three-dimensional model of building space layout is an important tool in the process of architectural design and planning. It can help designers visualize and analyze the layout and functional zoning of buildings in three-dimensional space.

[0003] Traditional architectural space planning and design often relies on inherent indoor floor plans, which have basic structural information but cannot flexibly respond to users' needs for indoor layout, resulting in the difficulty of the 3D model generated based on the original floor plan to meet the real needs of space design. Therefore, how to generate an architectural layout design that can simultaneously meet the diverse needs of users and structural rationality is a technical problem that needs to be solved urgently. Summary of the invention

[0004] In view of this, the purpose of the present application is to provide a method and a device for generating a three-dimensional model of a building space layout, by integrating the layout features of the plan structure diagram and the layout requirement features of the text description, using the plan layout features and the text layout features to reconstruct the indoor layout, and using a three-dimensional adversarial network to generate a three-dimensional model of the spatial layout that is reasonable, functionally complete, and meets the requirements on the basis of retaining the basic structure of the building, so as to serve as the base framework for subsequent building interior design.

[0005] In a first aspect, an embodiment of the present application provides a method for generating a three-dimensional model of a building space layout, the generating method comprising:

[0006] Obtaining a plan view of a target building and textual description information of layout design requirements for the target building;

[0007] Generate a layout fusion feature vector based on the plan layout features of the plan structure diagram and the text layout features of the text description information;

[0008] Inputting the layout fusion feature vector into a pre-trained spatial layout generation model to obtain the spatial layout of the target building, and generating a scene function matrix based on the spatial layout, the layout fusion feature vector and the text layout feature;

[0009] The scene function matrix is ​​input into a pre-trained three-dimensional generative adversarial network to generate a three-dimensional model of the spatial layout of the target building.

[0010] Furthermore, the generating of a layout fusion feature vector based on the plan layout feature of the plan structure diagram and the text layout feature of the text description information includes:

[0011] Extracting spatial geometric features of the planar structure graph using a convolutional neural network, determining spatial layout information of different scales in the spatial geometric features using a multi-scale convolution kernel, and constructing the planar graph layout features based on the spatial layout information;

[0012] Encoding the text description information using a Transformer model to determine spatial layout semantic features, and constructing the text layout features based on the spatial layout semantic features;

[0013] The layout differential relationship features between the plan view layout features and the text layout features are extracted, and multimodal feature fusion is performed on the plan view layout features, the text layout features and the layout differential relationship features to obtain the layout fusion feature vector.

[0014] Furthermore, the step of inputting the layout fusion feature vector into a pre-trained spatial layout generation model to obtain the spatial layout of the target building includes:

[0015] Inputting the layout fusion feature vector into the encoder of the spatial layout generation model, and encoding the layout fusion feature vector by using the encoder to obtain a spatial layout latent variable;

[0016] The spatial layout latent variables are input into a decoder of the spatial layout generation model to obtain the spatial layout.

[0017] Furthermore, the spatial layout generation model is trained through the following steps:

[0018] Acquire a training sample set; wherein the training sample set includes a layout fusion sample feature vector and a spatial sample layout corresponding to the layout fusion sample feature vector;

[0019] Inputting the layout fusion sample feature vector into the encoder in the original generation model of the spatial layout to obtain a spatial layout sample variable;

[0020] Inputting the spatial layout sample variables into a decoder in the spatial layout original generation model to obtain a spatial prediction layout;

[0021] A first loss value of the decoder is calculated based on the spatial sample layout and the spatial prediction layout, and the decoder is trained based on the first loss value of the decoder to obtain the spatial layout generation model.

[0022] Furthermore, the three-dimensional generative adversarial network is trained by the following steps:

[0023] Inputting a first scene function matrix sample into a generator of the three-dimensional generative adversarial network to obtain a first three-dimensional model;

[0024] Inputting the real three-dimensional model and the first three-dimensional model into the discriminator of the three-dimensional generative adversarial network, fixing the parameters of the generator, training the discriminator using the second loss value of the discriminator, and continuously optimizing the parameters of the discriminator;

[0025] Input a second scene function matrix sample into the generator to obtain a second three-dimensional model, input the second three-dimensional model into the discriminator, calculate a third loss value of the generator based on a discrimination result of the discriminator, and update the parameters of the generator through error back propagation;

[0026] Return to the step of inputting the first scene function matrix sample into the generator of the three-dimensional generative adversarial network, repeatedly and alternately train the generator and the discriminator until the generator and the discriminator reach a balanced state, and obtain the three-dimensional generative adversarial network.

[0027] In a second aspect, an embodiment of the present application further provides a device for generating a three-dimensional model of a building space layout, the generating device comprising:

[0028] An acquisition module, used to acquire a plan view of a target building and textual description information of layout design requirements for the target building;

[0029] A fusion feature vector generation module, used to generate a layout fusion feature vector based on the plan layout features of the plan structure diagram and the text layout features of the text description information;

[0030] A scene function matrix generation module, used for inputting the layout fusion feature vector into a pre-trained space layout generation model to obtain the spatial layout of the target building, and generating a scene function matrix based on the spatial layout, the layout fusion feature vector and the text layout feature;

[0031] The three-dimensional model generation module is used to input the scene function matrix into a pre-trained three-dimensional generative adversarial network to generate a three-dimensional model of the spatial layout of the target building.

[0032] Furthermore, when the fused feature vector generation module is used to generate a layout fused feature vector based on the plan layout features of the plan structure diagram and the text layout features of the text description information, the fused feature vector generation module is also used to:

[0033] Extracting spatial geometric features of the planar structure graph using a convolutional neural network, determining spatial layout information of different scales in the spatial geometric features using a multi-scale convolution kernel, and constructing the planar graph layout features based on the spatial layout information;

[0034] Encoding the text description information using a Transformer model to determine spatial layout semantic features, and constructing the text layout features based on the spatial layout semantic features;

[0035] The layout differential relationship features between the plan view layout features and the text layout features are extracted, and multimodal feature fusion is performed on the plan view layout features, the text layout features and the layout differential relationship features to obtain the layout fusion feature vector.

[0036] Furthermore, when the scene function matrix generation module is used to input the layout fusion feature vector into a pre-trained space layout generation model to obtain the space layout of the target building, the scene function matrix generation module is also used to:

[0037] Inputting the layout fusion feature vector into the encoder of the spatial layout generation model, and encoding the layout fusion feature vector by using the encoder to obtain a spatial layout latent variable;

[0038] The spatial layout latent variables are input into a decoder of the spatial layout generation model to obtain the spatial layout.

[0039] In a third aspect, an embodiment of the present application further provides an electronic device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the memory through the bus, and when the machine-readable instructions are executed by the processor, the steps of the method for generating a three-dimensional model of the architectural space layout as described above are performed.

[0040] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the method for generating a three-dimensional model of a building space layout as described above are executed.

[0041] A method and device for generating a three-dimensional model of a building space layout provided in an embodiment of the present application, first, obtain a plan structure diagram of a target building and text description information for the layout design requirements of the target building; then, generate a layout fusion feature vector based on the plan layout features of the plan structure diagram and the text layout features of the text description information; input the layout fusion feature vector into a pre-trained space layout generation model to obtain the space layout of the target building, and generate a scene function matrix based on the space layout, the layout fusion feature vector and the text layout features; finally, input the scene function matrix into a pre-trained three-dimensional generative adversarial network to generate a three-dimensional model of the space layout of the target building.

[0042] This application reconstructs the interior layout by integrating the layout features of the plan structure diagram and the layout requirement features described in the text, and uses the plan layout features and text layout features to reconstruct the interior layout. On the basis of retaining the basic structure of the building, a three-dimensional adversarial network is used to generate a three-dimensional model of the spatial layout that is reasonable, functionally complete, and meets the requirements, which serves as the base framework for subsequent architectural interior design.

[0043] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0045] Figure 1 A flowchart of a method for generating a three-dimensional model of a building space layout provided in an embodiment of the present application;

[0046] Figure 2 One of the structural schematic diagrams of a device for generating a three-dimensional model of a building space layout provided in an embodiment of the present application;

[0047] Figure 3 A second structural schematic diagram of a device for generating a three-dimensional model of a building space layout provided in an embodiment of the present application;

[0048] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0049] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application usually described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, each other embodiment obtained by those skilled in the art without making creative work belongs to the scope of protection of the present application.

[0050] First, the application scenarios to which the present application is applicable are introduced. The present application can be applied in the field of space planning technology.

[0051] The three-dimensional model of building space layout is an important tool in the process of architectural design and planning. It can help designers visualize and analyze the layout and functional zoning of buildings in three-dimensional space.

[0052] Research has found that traditional architectural space planning and design often relies on inherent indoor floor plans, which have basic structural information but cannot flexibly respond to users' needs for indoor layouts, resulting in the 3D model generated based on the original floor plan being difficult to meet the real needs of space design. Therefore, how to generate an architectural layout design that can simultaneously meet the diverse needs of users and structural rationality is a technical problem that needs to be solved urgently.

[0053] Based on this, the embodiment of the present application provides a method and a device for generating a three-dimensional model of a building space layout, so as to generate a three-dimensional model of a space layout that is reasonable in space, complete in functions, and meets the needs while retaining the basic structure of the building.

[0054] See also Figure 1 , Figure 1 The following is a flow chart of a method for generating a three-dimensional model of a building space layout provided in an embodiment of the present application. Figure 1 As shown in, the generation method provided in the embodiment of the present application includes:

[0055] S101, obtaining a plan structure diagram of a target building and text description information of layout design requirements for the target building.

[0056] It should be noted that the plan structure diagram of the target building is a plan view in architectural design, which is used to show the layout and structure of the building on the horizontal plane. It is usually drawn from a bird's-eye view of the building, showing the location and size of each room, corridor, staircase, door, window, wall, column, equipment and other structural elements of the building. The text description information refers to the text description input by the user for the layout design requirements of the target building.

[0057] In the above step S101, when implementing the above step, the plan structure diagram of the target building is obtained. P And textual description information for the layout design requirements of the target building T .

[0058] S102: Generate a layout fusion feature vector based on the plan layout features of the plan structure diagram and the text layout features of the text description information.

[0059] Regarding the above step S102, in the specific implementation, first determine the plane structure diagram P Floor plan layout features E P , and text description information T Text layout features E T Then the floor plan layout features E P and text layout features E T Perform multimodal feature fusion to obtain the layout fusion feature vector E F Here, according to the embodiment provided by the present application, a multi-stream neural network architecture is used to fuse the embedded text layout requirement features and the floor plan structure features to reconstruct the indoor layout. The multi-stream neural network processes different modal information by selecting appropriate neural networks and multi-stream weights to solve the conflict problem of inconsistent requirements in multi-modal fusion.

[0060] As an optional embodiment, with respect to the above step S102, generating a layout fusion feature vector based on the plan layout features of the plan structure diagram and the text layout features of the text description information includes:

[0061] Step 1021, using a convolutional neural network to extract the spatial geometric features of the planar structure diagram, determining spatial layout information of different scales in the spatial geometric features through a multi-scale convolution kernel, and constructing the planar diagram layout features based on the spatial layout information.

[0062] Regarding the above step 1021, in the specific implementation, firstly, a convolutional neural network is used to extract a planar structure diagram. P Spatial geometric features Then, the spatial geometric features are determined by multi-scale convolution kernels. Different scales Spatial layout information , and based on spatial layout information Build floor plan layout features E P . Specifically, the floor plan layout features E P It is expressed by the following formula:

[0063]

[0064] here, E P Refers to the plan structure diagram P The floor plan layout features, H Refers to the height of each area in the plan structure diagram. W Refers to the width of each area in the plane structure diagram. j Refers to the number of areas in the floor plan.

[0065] Step 1022: Encode the text description information using a Transformer model to determine the spatial layout semantic features, and construct the text layout features based on the spatial layout semantic features.

[0066] Regarding the above step 1022, in the specific implementation, firstly, the text description information T Input into the pre-trained Transformer model to describe the text information T Encode to obtain the spatial layout semantic features related to the indoor layout , and then use the spatial layout semantic features Constructing text description information T Text layout features E T . Specifically, text layout features E T It is expressed by the following formula:

[0067]

[0068] in, E T Refers to the text description T The text layout features of n Refers to the number of words related to interior layout in the text description information.

[0069] Step 1023, extracting the layout differential relationship features between the plan view layout features and the text layout features, and performing multimodal feature fusion on the plan view layout features, the text layout features and the layout differential relationship features to obtain the layout fusion feature vector.

[0070] Regarding the above step 1023, in the specific implementation, firstly extract the plan layout features E P With text layout features E T The layout difference relationship characteristics between E C Specifically, in order to identify the inconsistency between the apartment type and the text description requirements, the layout difference relationship features between the text description and the apartment type are extracted, and a cross-modal attention mechanism is introduced to form E P and E T The key vector Perform dot product calculation to generate attention weights ,use right E T The value vector of Perform weighted summation and get E C Specifically, layout differential relationship features E C It is expressed by the following formula:

[0071]

[0072] Layout differential relationship features E C After determining, the floor plan layout features E P , text layout features E T And layout differential relationship characteristics E C Perform multimodal feature fusion to obtain the layout fusion feature vector E F Specifically, a multi-layer perceptron is used to perform multimodal feature fusion, and the resulting layout fusion feature vector E F It is expressed by the following formula:

[0073]

[0074] S103, inputting the layout fusion feature vector into a pre-trained spatial layout generation model to obtain the spatial layout of the target building, and generating a scene function matrix based on the spatial layout, the layout fusion feature vector and the text layout feature.

[0075] Here, the spatial layout generation model adopts a generation model based on conditional variational autoencoder. The conditional variational autoencoder can generate diversified content that meets specific conditions by modeling the potential distribution of data, thereby enhancing the flexibility and personalization of generated content.

[0076] Regarding the above step S103, in the specific implementation, the layout fusion feature vector obtained in the above step is Input into the pre-trained spatial layout generation model to generate the spatial layout of the target building Then, by analyzing the spatial layout , layout fusion feature vector and text layout features extracted from text description information , generate a specific scenario function matrix . Specifically, the scenario function matrix It is expressed by the following formula:

[0077]

[0078] As an optional embodiment, with respect to the above step S103, the step of inputting the layout fusion feature vector into a pre-trained spatial layout generation model to obtain the spatial layout of the target building includes:

[0079] Step 1031: input the layout fusion feature vector into the encoder of the spatial layout generation model, and use the encoder to encode the layout fusion feature vector to obtain spatial layout latent variables.

[0080] For the above step 1031, in the specific implementation, the layout fusion feature vector E F As an input condition, the encoder of the spatial layout generation model is used to fusion the layout feature vector E F Encode to obtain the spatial layout latent variable Z. Specifically, Z=Encoder(E F ).

[0081] Step 1032: input the spatial layout latent variables into the decoder of the spatial layout generation model to obtain the spatial layout.

[0082] For the above step 1032, in the specific implementation, the spatial layout latent variable Z output by the encoder is input into the decoder Decoder of the spatial layout generation model, and the decoder Decoder is used to generate the indoor layout Y∈R according to the indoor layout latent variable Z. h×w×k ,Specifically, Y=Decoder(Z), where h is the height of each region in the ,spatial layout, w is the width of each region in the ,spatial layout, and k is the number of label channels of the ,indoor layout.

[0083] Specifically, according to the generation method provided in the embodiment of the present application, the spatial layout generation model is trained through the following steps:

[0084] A: Get a training sample set.

[0085] The training sample set includes a layout fusion sample feature vector and a spatial sample layout corresponding to the layout fusion sample feature vector.

[0086] B: Input the layout fusion sample feature vector into the encoder in the original spatial layout generation model to obtain the spatial layout sample variable.

[0087] C: Inputting the spatial layout sample variables into the decoder in the original generation model of the spatial layout to obtain a spatial prediction layout.

[0088] For the above steps A to C, in the specific implementation, a training sample set is obtained, which includes a layout fusion sample feature vector and a spatial sample layout corresponding to the layout fusion sample feature vector. Then, the layout fusion sample feature vector is input into the encoder in the original generation model of the spatial layout to obtain a spatial layout sample variable. Then, the spatial layout sample variable is input into the decoder in the original generation model of the spatial layout to obtain a spatial prediction layout.

[0089] D: Calculating a first loss value of the decoder based on the spatial sample layout and the spatial prediction layout, and training the decoder based on the first loss value of the decoder to obtain the spatial layout generation model.

[0090] For the above step D, in the specific implementation, the spatial sample layout is compared with the spatial prediction layout, and the accuracy of the decoder output is determined by comparing the spatial sample layout with the spatial prediction layout. At this time, it is necessary to calculate the first loss value of the decoder in the current state. Here, according to the embodiment provided by the present application, the loss function of the decoder includes reconstruction loss and KL divergence, and the first loss value of the decoder is calculated by the following formula:

[0091]

[0092] in, is the first loss value of the decoder, is the weight coefficient, is the KL divergence.

[0093] Then, based on the first loss value of the decoder, the decoder is trained and the parameters of the decoder are continuously adjusted. The decoder will continuously minimize the loss through iteration. At each step of the iteration, the first loss value of the decoder will be calculated. When the first loss value cannot reach the loss threshold, the parameters of the decoder are continuously updated. The new parameters will calculate a new loss value, so that the loss value shows a fluctuating downward trend during the iteration. Finally, when the loss value reaches smoothness, that is, when the loss value of the trained decoder is not greater than the loss threshold, that is, when the loss value is not significantly reduced compared with the loss value calculated last time, it is considered that the decoder in the original spatial layout generation model has reached a convergence state. At this time, the training is terminated to obtain the spatial layout generation model.

[0094] S104, inputting the scene function matrix into a pre-trained three-dimensional generative adversarial network to generate a three-dimensional model of the spatial layout of the target building.

[0095] Here, the 3D generative adversarial network improves the authenticity of the generated model through an adversarial learning mechanism to meet the layout and functional requirements. This application uses a 3D generative adversarial network to generate a 3D model of the spatial layout that matches the input features. Q As input, the generator of the generative adversarial network , generate a 3D model of the spatial layout . Specifically, the spatial layout 3D model It is expressed by the following formula:

[0096]

[0097] Specifically, according to the generation method provided in the embodiment of the present application, the three-dimensional generative adversarial network is trained through the following steps:

[0098] I: Input the first scene function matrix sample into the generator of the three-dimensional generative adversarial network to obtain a first three-dimensional model.

[0099] II: Input the real three-dimensional model and the first three-dimensional model into the discriminator of the three-dimensional generative adversarial network, fix the parameters of the generator, use the second loss value of the discriminator to train the discriminator, and continuously optimize the parameters of the discriminator.

[0100] For the above steps I-II, in the specific implementation, first obtain the first scene function matrix sample, and then input the first scene function matrix sample into the generator of the three-dimensional generative adversarial network to obtain the first three-dimensional model generated by the generator. The first stage of training is to input the real three-dimensional model and the first three-dimensional model generated by the generator into the discriminator. This process only updates the parameters of the discriminator and is used to train the discriminator to distinguish between true and false signals. Specifically, the first scene function matrix sample is output as a first three-dimensional model after passing through the generator. Then the real three-dimensional model and the first three-dimensional model are fed to the discriminator as input, the parameters of the generator are fixed, the second loss value of the discriminator is calculated, and the discriminator is trained based on the second loss value of the discriminator, and the parameters of the discriminator are continuously optimized. Through the above process, a two-class discriminator is trained. At this time, the discriminator is easier to distinguish between the real three-dimensional model and the generated three-dimensional model because the initial quality of the generated three-dimensional model is poor.

[0101] Here, the discriminator of the 3D generative adversarial network To evaluate the similarity between the generated spatial layout and the real layout, the loss function of the discriminator is:

[0102]

[0103] in, represents the probability that the real 3D model is judged as real, represents the probability that the first 3D model is judged as false. By continuously optimizing the loss function, the discriminator can improve its ability to identify the generated 3D spatial layout model and promote the generator to generate a more realistic 3D spatial layout model.

[0104] III: Input the second scene function matrix sample into the generator to obtain a second three-dimensional model, input the second three-dimensional model into the discriminator, calculate the third loss value of the generator based on the discrimination result of the discriminator, and update the parameters of the generator through error back propagation.

[0105] For the above step III, in the specific implementation, the parameters of the discriminator are fixed in the second stage of training to train the generator. The second scene function matrix sample is input into the generator to obtain the second three-dimensional model. Then the second three-dimensional model is input into the discriminator, and the discriminator outputs the discrimination result, which is the discriminator's judgment of whether the real three-dimensional model and the second three-dimensional model are true or false. At the same time, the third loss value of the generator is calculated based on the discrimination result of the discriminator, and the parameters of the generator are updated through error back propagation.

[0106] Specifically, according to the embodiment provided in this application, the generator of the three-dimensional generative adversarial network The loss function is:

[0107]

[0108] in, It is the L1 norm between the generated layout and the real layout, which is used to ensure the accuracy of the generated layout; λ is a balance term, which is used to weigh the confrontation between the generator and the discriminator.

[0109] IV: Return to the step of inputting the first scene function matrix sample into the generator of the three-dimensional generative adversarial network, repeatedly and alternately train the generator and the discriminator until the generator and the discriminator reach a equilibrium state, and obtain the three-dimensional generative adversarial network.

[0110] Here, the equilibrium state means that the discriminator cannot determine whether the 3D model generated by the generator is real or fake.

[0111] For the above step IV, in the specific implementation, the above steps I-III are only a cycle of adversarial training, and the above process will be repeated later, returning to the step of inputting the first scene function matrix sample into the generator of the three-dimensional generative adversarial network in the above step I, and repeatedly training the generator and the discriminator alternately. In the training of the generative adversarial network, the generator and the discriminator compete with each other and continuously optimize their respective parameters. When the three-dimensional model generated by the generator gradually approaches the real three-dimensional model, and the discriminator finds it difficult to distinguish the generated three-dimensional model from the real three-dimensional model, an equilibrium state is reached. In this state, the signal generated by the generator is of high quality, and the discriminator cannot accurately judge the true or false, so that the generated signal is more realistic. In summary, this embodiment can effectively train the generative adversarial network by generating model input discriminator, Nash equilibrium state and iterative adversarial training and other operations, so that it generates a realistic spatial local three-dimensional model.

[0112] The method for generating a three-dimensional model of the architectural space layout provided in the embodiment of the present application first obtains the plane structure diagram of the target building and the text description information of the layout design requirements of the target building; then, generates a layout fusion feature vector based on the plane layout features of the plane structure diagram and the text layout features of the text description information; inputs the layout fusion feature vector into a pre-trained spatial layout generation model to obtain the spatial layout of the target building, and generates a scene function matrix based on the spatial layout, the layout fusion feature vector and the text layout features; finally, inputs the scene function matrix into a pre-trained three-dimensional generative adversarial network to generate a three-dimensional model of the spatial layout of the target building. The present application reconstructs the indoor layout by integrating the layout features of the plane structure diagram and the layout requirement features of the text description, and uses the plane layout features and the text layout features to generate a three-dimensional spatial layout model that is reasonable in space, complete in function and meets the requirements on the basis of retaining the basic structure of the building, so as to serve as the base framework for subsequent architectural interior design.

[0113] See also Figure 2 and Figure 3 , Figure 2 This is one of the structural schematic diagrams of a device for generating a three-dimensional model of a building space layout provided in an embodiment of the present application. Figure 3 This is a second structural diagram of a device for generating a three-dimensional model of a building space layout provided in an embodiment of the present application. Figure 2 As shown in , the generating device 200 includes:

[0114] An acquisition module 201 is used to acquire a plan view of a target building and textual description information of layout design requirements of the target building;

[0115] A fusion feature vector generating module 202 is used to generate a layout fusion feature vector based on the plan layout features of the plan structure diagram and the text layout features of the text description information;

[0116] A scene function matrix generation module 203 is used to input the layout fusion feature vector into a pre-trained spatial layout generation model to obtain the spatial layout of the target building, and generate a scene function matrix based on the spatial layout, the layout fusion feature vector and the text layout feature;

[0117] The three-dimensional model generation module 204 is used to input the scene function matrix into a pre-trained three-dimensional generative adversarial network to generate a three-dimensional model of the spatial layout of the target building.

[0118] Furthermore, when the fused feature vector generating module 202 is used to generate a layout fused feature vector based on the plan layout features of the plan structure diagram and the text layout features of the text description information, the fused feature vector generating module 202 is also used to:

[0119] Extracting spatial geometric features of the planar structure graph using a convolutional neural network, determining spatial layout information of different scales in the spatial geometric features using a multi-scale convolution kernel, and constructing the planar graph layout features based on the spatial layout information;

[0120] Encoding the text description information using a Transformer model to determine spatial layout semantic features, and constructing the text layout features based on the spatial layout semantic features;

[0121] The layout differential relationship features between the plan view layout features and the text layout features are extracted, and multimodal feature fusion is performed on the plan view layout features, the text layout features and the layout differential relationship features to obtain the layout fusion feature vector.

[0122] Furthermore, when the scene function matrix generation module 203 is used to input the layout fusion feature vector into a pre-trained spatial layout generation model to obtain the spatial layout of the target building, the scene function matrix generation module 203 is also used to:

[0123] Inputting the layout fusion feature vector into the encoder of the spatial layout generation model, and encoding the layout fusion feature vector by using the encoder to obtain a spatial layout latent variable;

[0124] The spatial layout latent variables are input into a decoder of the spatial layout generation model to obtain the spatial layout.

[0125] See also Figure 3 The generating device 200 further includes a first training module 205, and the first training module 205 is used to train the spatial layout generation model through the following steps:

[0126] Acquire a training sample set; wherein the training sample set includes a layout fusion sample feature vector and a spatial sample layout corresponding to the layout fusion sample feature vector;

[0127] Inputting the layout fusion sample feature vector into the encoder in the original generation model of the spatial layout to obtain a spatial layout sample variable;

[0128] Inputting the spatial layout sample variables into a decoder in the spatial layout original generation model to obtain a spatial prediction layout;

[0129] A first loss value of the decoder is calculated based on the spatial sample layout and the spatial prediction layout, and the decoder is trained based on the first loss value of the decoder to obtain the spatial layout generation model.

[0130] See also Figure 3 The generating device 200 further includes a second training module 206, and the second training module 206 is used to train the three-dimensional generative adversarial network through the following steps:

[0131] Inputting a first scene function matrix sample into a generator of the three-dimensional generative adversarial network to obtain a first three-dimensional model;

[0132] Inputting the real three-dimensional model and the first three-dimensional model into the discriminator of the three-dimensional generative adversarial network, fixing the parameters of the generator, training the discriminator using the second loss value of the discriminator, and continuously optimizing the parameters of the discriminator;

[0133] Input a second scene function matrix sample into the generator to obtain a second three-dimensional model, input the second three-dimensional model into the discriminator, calculate a third loss value of the generator based on a discrimination result of the discriminator, and update the parameters of the generator through error back propagation;

[0134] Return to the step of inputting the first scene function matrix sample into the generator of the three-dimensional generative adversarial network, repeatedly and alternately train the generator and the discriminator until the generator and the discriminator reach a balanced state, and obtain the three-dimensional generative adversarial network.

[0135] See also Figure 4 , Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 4 As shown in , the electronic device 400 includes a processor 410 , a memory 420 and a bus 430 .

[0136] The memory 420 stores machine-readable instructions executable by the processor 410. When the electronic device 400 is running, the processor 410 communicates with the memory 420 via the bus 430. When the machine-readable instructions are executed by the processor 410, the above-mentioned Figure 1 The steps of the method for generating the three-dimensional model of the architectural space layout in the method embodiment shown, the specific implementation method can be found in the method embodiment, and will not be repeated here.

[0137] The present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the computer program can execute the above-mentioned Figure 1 The steps of the method for generating the three-dimensional model of the architectural space layout in the method embodiment shown, the specific implementation method can be found in the method embodiment, and will not be repeated here.

[0138] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0139] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0140] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0141] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0142] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program codes.

[0143] It should be noted that similar numbers and letters represent similar items in the following figures. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In addition, the terms "first", "second", "third", etc. are only used to distinguish the description and are not to be understood as indicating or implying relative importance.

[0144] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The protection scope of the present application is not limited thereto. Although the present application is described in detail with reference to the above-mentioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-mentioned embodiments within the technical scope disclosed in the present application, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A method for generating a three-dimensional model of a building space layout, characterized in that: The generation method comprises: Obtaining a plan view of a target building and textual description information of layout design requirements for the target building; Generate a layout fusion feature vector based on the plan layout features of the plan structure diagram and the text layout features of the text description information; Inputting the layout fusion feature vector into a pre-trained spatial layout generation model to obtain the spatial layout of the target building, and generating a scene function matrix based on the spatial layout, the layout fusion feature vector and the text layout feature; Inputting the scene function matrix into a pre-trained three-dimensional generative adversarial network to generate a three-dimensional model of the spatial layout of the target building; The generating of a layout fusion feature vector based on the plan layout feature of the plan structure diagram and the text layout feature of the text description information includes: Extracting spatial geometric features of the planar structure graph using a convolutional neural network, determining spatial layout information of different scales in the spatial geometric features using a multi-scale convolution kernel, and constructing the planar graph layout features based on the spatial layout information; Encoding the text description information using a Transformer model to determine spatial layout semantic features, and constructing the text layout features based on the spatial layout semantic features; Extracting layout differential relationship features between the plan view layout features and the text layout features, and performing multimodal feature fusion on the plan view layout features, the text layout features, and the layout differential relationship features to obtain the layout fusion feature vector; The step of inputting the layout fusion feature vector into a pre-trained spatial layout generation model to obtain the spatial layout of the target building includes: Inputting the layout fusion feature vector into the encoder of the spatial layout generation model, and encoding the layout fusion feature vector by using the encoder to obtain a spatial layout latent variable; Inputting the spatial layout latent variables into a decoder of the spatial layout generation model to obtain the spatial layout; The spatial layout generation model is trained by the following steps: Acquire a training sample set; wherein the training sample set includes a layout fusion sample feature vector and a spatial sample layout corresponding to the layout fusion sample feature vector; Inputting the layout fusion sample feature vector into the encoder in the original generation model of the spatial layout to obtain a spatial layout sample variable; Inputting the spatial layout sample variables into a decoder in the spatial layout original generation model to obtain a spatial prediction layout; Calculating a first loss value of the decoder based on the spatial sample layout and the spatial prediction layout, and training the decoder based on the first loss value of the decoder to obtain the spatial layout generation model; The three-dimensional generative adversarial network is trained by the following steps: Inputting a first scene function matrix sample into a generator of the three-dimensional generative adversarial network to obtain a first three-dimensional model; Inputting the real three-dimensional model and the first three-dimensional model into the discriminator of the three-dimensional generative adversarial network, fixing the parameters of the generator, training the discriminator using the second loss value of the discriminator, and continuously optimizing the parameters of the discriminator; Input a second scene function matrix sample into the generator to obtain a second three-dimensional model, input the second three-dimensional model into the discriminator, calculate a third loss value of the generator based on a discrimination result of the discriminator, and update the parameters of the generator through error back propagation; Return to the step of inputting the first scene function matrix sample into the generator of the three-dimensional generative adversarial network, repeatedly and alternately train the generator and the discriminator until the generator and the discriminator reach a balanced state, and obtain the three-dimensional generative adversarial network.

2. A device for generating a three-dimensional model of a building space layout, characterized in that: The generating device comprises: An acquisition module, used to acquire a plan view of a target building and textual description information of layout design requirements for the target building; A fusion feature vector generation module, used to generate a layout fusion feature vector based on the plan layout features of the plan structure diagram and the text layout features of the text description information; A scene function matrix generation module, used for inputting the layout fusion feature vector into a pre-trained space layout generation model to obtain the spatial layout of the target building, and generating a scene function matrix based on the spatial layout, the layout fusion feature vector and the text layout feature; A three-dimensional model generation module, used to input the scene function matrix into a pre-trained three-dimensional generative adversarial network to generate a three-dimensional model of the spatial layout of the target building; When the fused feature vector generation module is used to generate a layout fused feature vector based on the plan layout features of the plan structure diagram and the text layout features of the text description information, the fused feature vector generation module is further used to: Extracting spatial geometric features of the planar structure graph using a convolutional neural network, determining spatial layout information of different scales in the spatial geometric features using a multi-scale convolution kernel, and constructing the planar graph layout features based on the spatial layout information; Encoding the text description information using a Transformer model to determine spatial layout semantic features, and constructing the text layout features based on the spatial layout semantic features; Extracting layout differential relationship features between the plan view layout features and the text layout features, and performing multimodal feature fusion on the plan view layout features, the text layout features, and the layout differential relationship features to obtain the layout fusion feature vector; When the scene function matrix generation module is used to input the layout fusion feature vector into a pre-trained space layout generation model to obtain the space layout of the target building, the scene function matrix generation module is also used to: Inputting the layout fusion feature vector into the encoder of the spatial layout generation model, and encoding the layout fusion feature vector by using the encoder to obtain a spatial layout latent variable; Inputting the spatial layout latent variables into a decoder of the spatial layout generation model to obtain the spatial layout; The generating device further includes a first training module, and the first training module is used to train the spatial layout generation model through the following steps: Acquire a training sample set; wherein the training sample set includes a layout fusion sample feature vector and a spatial sample layout corresponding to the layout fusion sample feature vector; Inputting the layout fusion sample feature vector into the encoder in the original generation model of the spatial layout to obtain a spatial layout sample variable; Inputting the spatial layout sample variables into a decoder in the spatial layout original generation model to obtain a spatial prediction layout; Calculating a first loss value of the decoder based on the spatial sample layout and the spatial prediction layout, and training the decoder based on the first loss value of the decoder to obtain the spatial layout generation model; The generating device further includes a second training module, and the second training module is used to train the three-dimensional generative adversarial network through the following steps: Inputting a first scene function matrix sample into a generator of the three-dimensional generative adversarial network to obtain a first three-dimensional model; Inputting the real three-dimensional model and the first three-dimensional model into the discriminator of the three-dimensional generative adversarial network, fixing the parameters of the generator, training the discriminator using the second loss value of the discriminator, and continuously optimizing the parameters of the discriminator; Input a second scene function matrix sample into the generator to obtain a second three-dimensional model, input the second three-dimensional model into the discriminator, calculate a third loss value of the generator based on a discrimination result of the discriminator, and update the parameters of the generator through error back propagation; Return to the step of inputting the first scene function matrix sample into the generator of the three-dimensional generative adversarial network, repeatedly and alternately train the generator and the discriminator until the generator and the discriminator reach a balanced state, and obtain the three-dimensional generative adversarial network.

3. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are run by the processor, the steps of the method for generating a three-dimensional model of the architectural space layout as described in claim 1 are executed.

4. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method for generating a three-dimensional model of a building space layout as claimed in claim 1 are executed.

Images