A method for automatically generating three-dimensional layout of ship cabin furniture

Abstract

The present invention belongs to the field of ship design and computer-aided design technology, and relates to a method for automatically generating the three-dimensional layout of ship cabin furniture. The method constructs a high-level ship cabin scene graph by extracting graphic element information from the ship cabin CAD drawings; relies on a knowledge retrieval and specification extraction mechanism to call classification society standards, industry regulations, and empirical rules to form layout constraints; uses a large language model to semantically enhance the scene graph, comprehensively improving the integrity and contextual relevance of the graph structure expression; and realizes the joint generation of three-dimensional furniture positions and shapes through a joint shape-layout generation network, and introduces a conditional diffusion model and VQ-VAE structure to ensure the fidelity and diversity of shape generation. Therefore, the method of the present invention constructs a closed-loop linkage and automated modeling process of "drawing-knowledge-model" linkage, which has good technical versatility, system intelligence, and industrial application prospects.

Description

Technical Field

[0001] The present invention relates to the technical field of ship design and computer-aided design, and in particular to a method for automatically generating three-dimensional layout of ship cabin furniture. Background Art

[0002] As a crucial component of a ship's interior functional areas, ship accommodations (such as crew bedrooms, galleys, dining rooms, and offices) face multiple challenges in 3D modeling and furniture layout, including complex spatial structures, diverse drawing representations, and intensive layout logic. Traditional methods rely on manual CAD drawing recognition, table-based selection of furniture templates, and manual adjustment of model size and position. This approach is not only inefficient but also highly dependent on designer experience, prone to errors and unable to adapt to the large-scale, high-precision cabin layout requirements of modern ships.

[0003] On the other hand, cabin furniture layout is not just a geometric modeling problem; it also involves the precise incorporation of heterogeneous knowledge, including classification society regulations, corporate standards, and design experience. Currently, there is a lack of efficient mechanisms to organically integrate drawing information with complex design knowledge, which is a core difficulty in automated modeling.

[0004] Recent progress in graph-based layout-shape co-generation methods in the field of 3D scene synthesis has been significant. Large language models (LLMs) have demonstrated strong capabilities in graph semantic understanding and knowledge generation. By introducing a large model enhancement mechanism, a global understanding of the elements, annotations, and structures in cabin drawings can be achieved, achieving semantic alignment with design specifications. Furthermore, a knowledge retrieval mechanism dynamically extracts constraints from a ship design rule base, guiding the automated generation and optimal placement of furniture models.

[0005] Therefore, there is an urgent need for an automated method that integrates drawing analysis and large-scale model enhanced knowledge retrieval to achieve high-precision analysis of cabin drawings, intelligent understanding of design knowledge and automatic generation of three-dimensional models, breaking through the current bottlenecks in efficiency, intelligence and compliance of ship cabin furniture layout. Summary of the Invention

[0006] In view of the problems and shortcomings in the prior art, the present invention aims to provide a method for automatically generating a three-dimensional layout of ship cabin furniture.

[0007] To achieve the purpose of the invention, the technical solution adopted by the present invention is as follows:

[0008] A first aspect of the present invention provides a method for automatically generating a three-dimensional layout of ship cabin furniture, comprising the following steps:

[0009] S1: Extracting primitive information from the ship cabin CAD drawing and constructing the initial scene graph of the ship cabin based on the primitive information GThe graphic element information includes boundary information of the ship cabin, graphic element type information, furniture category information, furniture layout position information, furniture geometry information and annotation information;

[0010] S2: the initial scene graph G Perform semantic enhancement processing ， Get enhanced scene graph ;

[0011] S3: Scene graph Input pre-trained joint shape-layout generation network, which consists of scene encoder and scene decoder; scene graph First, the scene encoder is used to extract features and obtain the scene graph The full-image feature vector of And output, the layout decoder decodes the full image feature vector to obtain the layout position mean of the furniture object , mean rotation angle , layout position variance , rotation angle variance , splicing to obtain the distribution parameters ; Sample the distribution parameters to obtain the potential layout vector , using the latent layout vector Collection Replacing the scene graph The furniture layout position information in the scene graph is obtained ;

[0012] S4: Scene encoder using pre-trained joint shape-layout generation network to generate scene graphs Perform feature extraction to obtain a scene graph The full-image feature vector of , the whole image feature vector Input the layout decoder and shape decoder of the scene decoder respectively, decode and process to obtain the scene graph The spatial position information and three-dimensional shape information of each furniture object in the scene decoder are input into the VQ-VAE decoder to reconstruct the three-dimensional layout of the ship cabin furniture.

[0013] According to the above-mentioned method for automatically generating a three-dimensional layout of ship cabin furniture, preferably, the layout decoder is composed of two sets of parallel stacked multi-layer perceptrons, and the first set of multi-layer perceptrons is used to predict the three-dimensional bounding box information of each furniture object. b The second set of multi-layer perceptrons is used to predict the rotation angle of each furniture object in the vertical axis direction. ; The layout position information generated by the layout decoder is ,symbol Represents vector concatenation.

[0014] According to the above-mentioned method for automatically generating the three-dimensional layout of ship cabin furniture, preferably, the shape decoder is composed of a conditional diffusion model and a group of multi-layer perceptrons; the multi-layer perceptron is used to extract the shape latent variable encoding from the full image feature vector c And output, the input of the multi-layer perceptron is the full image feature vector output by the scene encoder; the conditional diffusion model encodes the shape latent variable output by the multi-layer perceptron c Generate the real 3D layout of furniture objects in the cabin as a guide condition - shape encoding .

[0015] According to the above-mentioned method for automatically generating a three-dimensional layout of ship cabin furniture, preferably, the conditional diffusion model is a Latent Diffusion Model.

[0016] More preferably, the latent diffusion model encodes the shape latent variable output by the multi-layer perceptron c Generate the real 3D layout of furniture objects in the cabin as a guide condition - shape encoding The specific operations are:

[0017] (1) Gradually add random noise to the shape latent variable encoding c , get any time step The noise representation :

[0018] ,

[0019] in, , , is the diffusion scheduling sequence; It represents the original signal retention coefficient at each step, which is used to describe the k The proportion of the original signal retained in the step; represents the random noise added;

[0020] (2) From Gaussian noise First, use 3D-UNet network to restore and get the true 3D layout-shape encoding .

[0021] According to the above-mentioned method for automatically generating a three-dimensional layout of ship cabin furniture, preferably, the scene encoder is a graph convolutional network (GCN).

[0022] According to the above-mentioned method for automatically generating a three-dimensional layout of ship cabin furniture, preferably, the specific operation of step S1 is: inputting the ship cabin CAD drawing into a pre-trained primitive-text dual-channel fusion recognition model, using the primitive-text dual-channel fusion recognition model to recognize and extract the primitive information of the ship cabin in the CAD drawing, and then using the primitive-text dual-channel fusion recognition model to construct an initial scene map of the ship cabin based on the primitive information. G , ,in, Furniture objects identified in CAD drawings, The spatial constraint relationship between furniture objects (the spatial constraint relationship includes "close", "aligned", and "perpendicular to the wall"), each furniture object and spatial constraints is embedded as the initial feature vector: ,in represents the feature dimension, is the number of furniture objects, M The number of spatial constraint relationship types.

[0023] According to the above-mentioned method for automatically generating the three-dimensional layout of ship cabin furniture, preferably, the primitive-text dual-channel fusion recognition model is composed of a text encoder, a visual encoder, a large language model, a hidden layer and a large model Head prediction layer; wherein the text encoder is used to convert the text information in the ship cabin CAD drawing into computer-recognizable vector data, the input of the text encoder is the text information in the ship cabin CAD drawing, and the output is a computer-recognizable numerical vector; the visual encoder is used to convert the primitive information in the ship cabin CAD drawing into computer-recognizable vector data, the input of the visual encoder is the primitive information in the ship cabin CAD drawing, and the output is a computer-recognizable numerical vector; the computer-recognizable numerical vector data output by the text encoder and the visual encoder are processed by vector addition. The large language model is used to unify the merged vector into the same semantic space, obtain a semantic vector and output it. The input of the large language model is the merged vector, and the output is the semantic vector. The hidden layer is used to extract features from the semantic vector output by the large language model to obtain an intermediate feature numerical vector and output it. The input of the hidden layer is the semantic vector, and the output is the intermediate feature numerical vector. The large model Head prediction layer is used to complete classification prediction for the intermediate feature numerical vector. The input of the large model Head prediction layer is the intermediate feature numerical vector output by the hidden layer, and the output is the graphic element information in the CAD drawing of the ship cabin. More preferably, the large prediction model is the large language model is GPT-4o.

[0024] According to the above-mentioned method for automatically generating a three-dimensional layout of ship cabin furniture, preferably, the specific operations of step S2 are:

[0025] S201: Based on the initial scene graph , retrieved from the standardized knowledge base consisting of specification documents related to ship interior design, and obtained The design specifications and empirical constraints related to the corresponding cabin type, furniture category, and spatial constraint relationship between furniture objects are obtained, and the semantic knowledge of each design specification and empirical constraint is defined in the form of triples to obtain the triple semantic knowledge of each design specification and empirical constraint. , ,in, Indicates the subject object , Indicates the object of the sentence. Represents predicates or / and relationships; combines all retrieved design specifications with triple semantic knowledge of empirical constraints Collection, get the initial scene graph G Scenario standard knowledge set

[0026] S202: From the scene standard knowledge collection Each triplet of semantic knowledge in Extract text information from the text, and convert the extracted text information into a natural language subject-verb-object text input visual language model for text encoding processing, and obtain the semantic knowledge of each triple Semantic vector embedding ( );in, Representing triple semantic knowledge The semantic vector of the overall relationship, Represents the semantic vector of the subject, The semantic vector representing the object;

[0027] S203: Gathering scene standard knowledge All triple semantic knowledge in As a structured input to the large language model, the initial scene graph As structured text input into the large language model, the large language model is used to extract semantic features to obtain the initial scene graph Scene-level global vector embedding And output;

[0028] S204: Semantic knowledge of each triple obtained in step S202 Semantic vector embedding ( ) and the scene-level global vector embedding obtained in step S203 Initial scene graph Perform semantic enhancement to obtain an enhanced scene graph , ,in, Represents each furniture object and relationships is embedded as the initial feature vector, Represented as furniture layout position information embedded from CAD drawings, represents the semantic vector embedding obtained by text encoding using the visual language model, represents the semantic vector embedding of the overall relationship obtained by using the visual language model for the text encoder, Represents the initial scene graph The scene-level global vector embedding of Represents the scene graph The initial eigenvector set of the relation in , symbol Refers to vector concatenation.

[0029] More preferably, in step S202, the visual language model is the CLIP visual language model; in step S203, the large language model is GPT-4o.

[0030] According to the above-mentioned method for automatically generating a three-dimensional layout of ship cabin furniture, preferably, in step S1, the furniture geometric information includes furniture size and furniture shape information.

[0031] According to the above-mentioned method for automatically generating the three-dimensional layout of ship cabin furniture, preferably, the scene encoder is used to extract features from the enhanced scene graph, obtain and output the full-image feature vector of the enhanced scene graph; the scene decoder is used to decode the full-image feature vector output by the scene encoder, and generate the spatial position and three-dimensional shape of the furniture objects in the cabin in the enhanced scene graph.

[0032] According to the above-mentioned method for automatically generating a 3D layout of ship cabin furniture, preferably, the training of the joint shape-layout generation network includes the training of a scene encoder and the training of a scene decoder. The scene encoder is a graph convolutional neural network (GCN), and the graph convolutional neural network GCN is trained using KL divergence loss. Training, KL divergence loss The calculation formula is as follows:

[0033]

[0034] in, For a given The posterior distribution under the input; Prior distribution; represents the Kullback-Leibler divergence.

[0035] The training goal of graph neural network (GCN) is to minimize the posterior distribution With prior distribution The KL (Kullback-Leibler) divergence between .

[0036] The scene decoder is trained as follows: the layout decoder and shape decoder are trained using layout loss , shape loss Conduct training. The calculation formula is as follows

[0037]

[0038] Where, is the layout reconstruction loss, is the regularization loss based on intersection-over-union ratio, This paper introduces a regularization loss based on the intersection-over-union (IoU) , regularization The loss solves the problem of using only reconstruction loss by encouraging the similarity between the predicted layout and the real layout, avoiding layout overlap and improving the rationality of the layout. When training the layout decoder, the layout decoder can usually only handle simple scenes and has the defect of not being realistic enough in predicting the spatial layout of complex scenes.

[0039] The calculation formula is as follows:

[0040]

[0041] Where, is the layout position information of the object, For the The 3D bounding box of an object, is the rotation angle, is the number of divisions of the rotation space; layout reconstruction loss The first term is the 3D bounding box regression loss, and the second term is the rotation classification loss.

[0042] The calculation formula is as follows:

[0043] .

[0044] Shape loss The calculation formula is as follows:

[0045]

[0046] Where, t is a time step that changes from a discrete value set mid-sampling; represents the real noise in the diffusion process, Indicates that the denoising model (such as a neural network) is used according to the current state , time step t and 3D layout-shape guidance conditions The predicted noise, Indicates in Noise signal after step diffusion.

[0047] Finally, the overall training loss of the joint shape-layout generation network is The calculation formula is as follows:

[0048]

[0049] Where, 、 and They are 、 、 The weight coefficient of the loss.

[0050] The present invention adopts this joint training method to enable the model to simultaneously learn structurally sound layouts and realistic object shapes, thereby generating more realistic 3D scenes.

[0051] The second aspect of the present invention provides a computer device, preferably, the computer device includes: a processor and a memory communicatively connected to the processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement the method for automatically generating the three-dimensional layout of ship cabin furniture as described in the first aspect above.

[0052] A third aspect of the present invention provides a computer-readable storage medium. Preferably, the computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method for automatically generating a three-dimensional layout of ship cabin furniture as described in the first aspect above.

[0053] Compared with the prior art, the present invention has the following positive and beneficial effects:

[0054] (1) Based on the graphics and annotations in CAD drawings, the present invention uses a graphics-text dual-channel fusion recognition model to extract key parameters such as the type, size, and orientation of furniture in CAD drawings, and dynamically matches and reconstructs them with a parametric furniture model library to achieve automatic generation and flexible configuration of furniture models. Furthermore, the present invention introduces a large language model (LLM) into the graphics-text dual-channel fusion recognition model to enhance the semantic parsing and knowledge retrieval capabilities of the drawings. LLM leverages its global understanding of the graph structure and text content to generate a hierarchical semantic representation of the cabin scene. Therefore, through the graphics-text dual-channel fusion recognition and automatic cabin boundary extraction technology, a high-precision cabin scene diagram can be constructed.

[0055] (2) The present invention combines knowledge retrieval and specification fusion technology to dynamically obtain constraint rules and design logic from massive design specifications and corporate standards, which are used to guide furniture type selection, layout logic judgment and spatial relationship constraints, significantly improving the compliance and rationality of automatic layout results.

[0056] (3) The present invention can construct a high-precision cabin scene graph through the dual-channel fusion recognition of primitives and texts and the automatic extraction of cabin boundaries. Relying on the knowledge retrieval and specification extraction mechanism, it automatically calls the classification society standards, industry regulations and empirical rules to form structured layout constraints. With the help of CLIP and the large language model, the scene graph is enhanced at the node level, edge level and global semantic level, which comprehensively improves the integrity and context relevance of the graph structure expression. On this basis, the joint generation of three-dimensional furniture positions and shapes is realized through the joint shape-layout generation network, and the conditional diffusion model and VQ-VAE structure are introduced to ensure the fidelity and diversity of shape generation.

[0057] (4) The method for automatically generating three-dimensional layout of ship cabin furniture in the present invention innovatively integrates CAD drawing semantic analysis, graph structure modeling, design specification knowledge retrieval and large language model enhancement mechanism, and constructs a closed-loop linkage and automated modeling process of "drawing-knowledge-model". It effectively breaks through the technical bottlenecks of traditional furniture modeling process such as heavy reliance on design experience, low efficiency of specification calling and poor layout compliance. It has strong adaptability and high scalability, and is particularly suitable for large-scale ship design tasks with a large number of cabins, complex structures and inconsistent drawing specifications.

[0058] (5) The present invention significantly improves the degree of automation, modeling accuracy and compliance reliability of cabin furniture layout modeling, reduces the burden of manual design, and is suitable for various engineering scenarios such as new ship design, cabin renovation, digital twin system construction, virtual simulation, etc. It has good technical versatility, system intelligence and industrial application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1This is a flowchart of scene graph semantic knowledge enhancement based on a large language model and a visual language model in the present invention;

[0060] Figure 2 Generate scene-level global vector embeddings using a large language model in this invention Tip template for

[0061] Figure 3 Schematic diagram of the architecture of the joint shape-layout generation network in the present invention;

[0062] Figure 4 Schematic diagram of the network architecture of the image-text dual-channel fusion recognition model in the present invention;

[0063] Figure 5 This is a rendering of a three-dimensional layout of ship cabin furniture generated based on a ship cabin CAD drawing using the method for automatically generating a three-dimensional layout of ship cabin furniture described in Example 1 of the present invention. DETAILED DESCRIPTION

[0064] Example 1:

[0065] A method for automatically generating a three-dimensional layout of ship cabin furniture, comprising the following specific steps:

[0066] S1: Input the ship cabin CAD drawing into the pre-trained primitive-text dual-channel fusion recognition model, and use the primitive-text dual-channel fusion recognition model to recognize and extract the primitive information in the CAD drawing; the primitive information includes the boundary information of the ship cabin, primitive type information, furniture category information, furniture layout position information, furniture geometry information and annotation information; wherein the furniture layout position information is represented as , is the 3D bounding box information of the furniture, For furniture facing angle, It is an embedded representation of furniture layout position information in CAD drawings by the dual-channel fusion recognition model of graphics and text. represents the feature dimension, is the number of furniture objects; furniture geometric information includes furniture size and furniture shape information;

[0067] S2: Based on the ship cabin boundary information, furniture category, furniture layout location information and furniture geometry information extracted by the primitive-text dual-channel fusion recognition model, the initial scene map of the ship cabin is constructed using the primitive-text dual-channel fusion recognition model. , ,in, Furniture objects identified in CAD drawings, The spatial constraint relationship between furniture objects (the spatial constraint relationship includes "close", "aligned", and "perpendicular to the wall"), each furniture object and spatial constraints is embedded as the initial feature vector: ,in represents the feature dimension, is the number of furniture objects, M is the number of spatial constraint relationship types;

[0068] S3: Based on the initial scene graph , retrieved from a standardized knowledge base consisting of regulatory documents related to ship interior design (e.g. Figure 1 As shown), get the initial scene graph The design specifications and empirical constraints related to the corresponding cabin type, furniture category, and spatial constraint relationship between furniture objects are obtained, and the semantic knowledge of each design specification and empirical constraint is defined in the form of triples to obtain the triple semantic knowledge of each design specification and empirical constraint. , ,in, Indicates the subject object (Starting node), such as the furniture object chair in the scene graph; Represents the object (target node), such as the furniture object table in the scene graph; Represents predicates and / or relationships (edges), such as: next to...; triple semantic knowledge that combines all retrieved design specifications with empirical constraints Collection, get the initial scene graph Scenario standard knowledge set , , M is the number of spatial constraint relationship types;

[0069] S4: From the scene standard knowledge collection Each triplet of semantic knowledge in Extract text information from the text, and convert the extracted text information into a natural language subject-verb-object text input CLIP visual language model for text encoding processing to obtain the semantic knowledge of each triple Semantic vector embedding ( ),in, Representing triple semantic knowledge The semantic vector of the overall relationship, Represents the semantic vector of the subject, The semantic vector representing the object;

[0070] S5: Collecting scene standard knowledge All triple semantic knowledge in As the structured input to the large language model GPT-4o, the initial scene graph Input the structured text into the large language model, use the large language model GPT-4o to extract semantic features, and generate the initial scene graph Scene-level global vector embedding And output, Represents feature dimensions; among them, the large language model GPT-4o is used to extract semantic features and generate the initial scene graph Scene-level global vector embedding The prompt template is as follows Figure 2 As shown;

[0071] S6: Semantic knowledge of each triple obtained in step S4 Semantic vector embedding And the scene-level global vector embedding obtained in step S5 Initial scene graph Perform semantic enhancement to obtain the enhanced scene graph , ,in, Represents each furniture object and relationships is embedded as the initial feature vector, Represented as furniture layout position information embedded from CAD drawings, represents the object semantic vector embedding obtained by text encoding using the CLIP visual language model, Represents the semantic vector embedding of the overall relationship of objects obtained by using the CLIP visual language model for text encoder, Represents the initial scene graph The scene-level global vector embedding of Represents the scene graph The initial eigenvector set of the relation in , symbol Refers to vector splicing;

[0072] S7: The enhanced scene graph Input a pre-trained joint shape-layout generation network (such as Figure 3As shown); the joint shape-layout generation network consists of a scene encoder and a scene decoder, wherein the scene encoder is used to extract features from the enhanced scene graph, obtain and output the full-image feature vector of the enhanced scene graph, and the scene decoder is used to decode the full-image feature vector output by the scene encoder, and generate the spatial position and three-dimensional shape of the furniture objects in the cabin in the enhanced scene graph; the scene decoder is composed of a layout decoder and a shape decoder; the layout decoder is used to decode the input full-image feature vector and generate the layout position information of each furniture object in the cabin; the shape decoder is used to decode the input full-image feature vector and generate the three-dimensional shape of each furniture object in the cabin; the enhanced scene graph of the input joint shape-layout generation network First, feature extraction is performed through the scene encoder to obtain the enhanced scene graph The full-image feature vector of And output, the layout decoder outputs the full image feature vector of the scene encoder Perform decoding processing to obtain the layout position mean of furniture objects , mean rotation angle , layout position variance , rotation angle variance , the distribution parameters are obtained by splicing , and then sample from the distribution parameters through the reparameterization technique to obtain the potential layout vector , , the latent layout vector The collection of , , Represents a set of potential layout vectors; using Replace the enhanced scene graph in , get the scene graph , ;

[0073] S8: Scene encoder using pre-trained joint shape-layout generation network to generate scene graphs Perform feature extraction to obtain a scene graph The full-image feature vector of , the whole image feature vector Input the layout decoder and shape decoder of the scene decoder respectively, perform decoding processing, and obtain the scene graph The spatial position information and 3D shape information of each furniture object in the scene decoder are input into the pre-trained VQ-VAE decoder to reconstruct the 3D layout of the ship cabin furniture.

[0074] Among them, the primitive-text dual-channel fusion recognition model (such as Figure 4 As shown) consists of a text encoder, a visual encoder, a large language model GPT-4o, a hidden layer and a large model Head prediction layer; wherein, the text encoder is used to convert the text information in the ship cabin CAD drawing into computer-recognizable vector data, the input of the text encoder is the text information in the ship cabin CAD drawing, and the output is a computer-recognizable numerical vector; the visual encoder is used to convert the primitive information in the ship cabin CAD drawing into computer-recognizable vector data, the input of the visual encoder is the primitive information in the ship cabin CAD drawing, and the output is a computer-recognizable numerical vector; the computer-recognizable numerical vector data output by the text encoder and the visual encoder are processed by vector addition to obtain a merged vector; the The large language model GPT-4o is used to unify the merged vector into the same semantic space, obtain a semantic vector and output it. The input of the large language model GPT-4o is the merged vector, and the output is a semantic vector; the hidden layer is used to perform feature extraction on the semantic vector output by the large language model GPT-4o to obtain an intermediate feature numerical vector and output it. The input of the hidden layer is the semantic vector, and the output is an intermediate feature numerical vector; the large model Head prediction layer is used to complete classification prediction of the intermediate feature numerical vector. The input of the large model Head prediction layer is the intermediate feature numerical vector output by the hidden layer, and the output is the category information corresponding to the circles, points, arcs, and lines in the ship cabin CAD drawing.

[0075] The layout decoder consists of two sets of parallel stacked multi-layer perceptrons (MLPs). The first set of MLPs is used to predict the 3D bounding box information of each furniture object. b , b ， in, Indicates the center coordinates of the furniture object, Represent the width, length, and height of the furniture object respectively; the second group of multi-layer perceptrons is used to predict the rotation angle of each furniture object in the vertical axis direction ; The layout position information generated by the layout decoder is ,symbol Represents vector concatenation.

[0076] The shape decoder consists of a conditional diffusion model and a set of multi-layer perceptrons (MLPs) (denoted as ); the multi-layer perceptron is used to extract shape latent variable encoding from the full image feature vector c And output, the input of the multi-layer perceptron is the full image feature vector output by the scene encoder; the conditional diffusion model encodes the shape latent variable output by the multi-layer perceptron cGenerate the real 3D layout of furniture objects in the cabin as a guide condition - shape encoding .

[0077] Preferably, the conditional diffusion model is a Latent Diffusion Model. The Latent Diffusion Model uses the shape latent variable encoding output by the multi-layer perceptron c Generate the real 3D layout of furniture objects in the cabin as a guide condition - shape encoding The specific operations (reference paper: "High-Resolution Image Synthesis with Latent Diffusion Models") are:

[0078] (1) Gradually add random noise to the shape latent variable encoding c , get any time step The noise representation :

[0079]

[0080] in, , , is the diffusion scheduling sequence; It represents the original signal retention coefficient at each step, which is used to describe the k The proportion of the original signal retained in the step; represents the random noise added;

[0081] (2) From Gaussian noise First, use 3D-UNet network to restore and get the true 3D layout-shape encoding .

[0082] According to the above-mentioned method for automatically generating a 3D layout of ship cabin furniture, preferably, the training of the joint shape-layout generation network includes the training of a scene encoder and the training of a scene decoder. The scene encoder is a graph convolutional neural network (GCN), and the graph convolutional neural network GCN is trained using KL divergence loss. Training, KL divergence loss The calculation formula is as follows:

[0083]

[0084] in, For a given The posterior distribution under the input; Prior distribution; represents the Kullback-Leibler divergence.

[0085] The training goal of graph neural network (GCN) is to minimize the posterior distribution With prior distribution The KL (Kullback-Leibler) divergence between them. Prior distribution and posterior distribution are both well-known terms in this field. The prior distribution represents the prior knowledge of parameters before any data is available, obtained through real indoor scene data. The posterior distribution can be understood as the distribution of predicted shapes and layouts given the observed data.

[0086] The scene decoder is trained as follows: the layout decoder and shape decoder are trained using layout loss , shape loss Conduct training. The calculation formula is as follows

[0087] ,

[0088] Where, is the layout reconstruction loss, is the regularization loss based on intersection-over-union ratio, This paper introduces a regularization loss based on the intersection-over-union (IoU) , regularization The loss solves the problem of using only reconstruction loss by encouraging the similarity between the predicted layout and the real layout, avoiding layout overlap and improving the rationality of the layout. When training the layout decoder, the layout decoder can usually only handle simple scenes and has the defect of not being realistic enough in predicting the spatial layout of complex scenes.

[0089] The calculation formula is as follows:

[0090]

[0091] Where, is the layout position information of the object, For the The 3D bounding box of an object, is the rotation angle, is the number of divisions of the rotation space; layout reconstruction loss The first term is the 3D bounding box regression loss, and the second term is the rotation classification loss.

[0092] The calculation formula is as follows:

[0093] .

[0094] Shape loss The calculation formula is as follows:

[0095]

[0096] Where, is a time step that changes from a discrete value set mid-sampling; represents the real noise in the diffusion process, Indicates that the denoising model (such as a neural network) is used according to the current state , time step t and 3D layout-shape guidance conditions The predicted noise, Indicates in Noise signal after step diffusion.

[0097] Finally, the overall training loss of the joint shape-layout generation network is The calculation formula is as follows:

[0098]

[0099] Where, 、 and They are 、 、 The weight coefficient of the loss.

[0100] The three-dimensional layout effect diagram of ship cabin furniture reconstructed based on the ship cabin CAD drawing by using the automatic generation method of three-dimensional layout of ship cabin furniture described in Example 1 is as follows: Figure 5 shown. Figure 5 The two pictures above and below are the 3D layout renderings of ship cabin furniture reconstructed from two different ship cabin CAD drawings. Figure 5 It can be seen that the automatic generation method of the three-dimensional layout of ship cabin furniture of the present invention can realize the automatic generation of the layout of ship cabin furniture with high modeling accuracy and high fidelity.

[0101] Example 2:

[0102] A computer device comprising: a processor and a memory communicatively connected to the processor; the memory storing computer-executable instructions; and the processor executing the computer-executable instructions stored in the memory to implement the method for automatically generating a three-dimensional layout of ship cabin furniture as described in Example 1 above.

[0103] Example 3:

[0104] A computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method for automatically generating a three-dimensional layout of ship cabin furniture as described in the above-mentioned embodiment 1.

Claims

1. A method for automatically generating a three-dimensional layout of ship cabin furniture, characterized in that: The following steps are involved: S1: Extracting primitive information from the ship cabin CAD drawing and constructing the initial scene graph of the ship cabin based on the primitive information G The graphic element information includes boundary information of the ship cabin, graphic element type information, furniture category information, furniture layout position information, furniture geometry information and annotation information; S2: the initial scene graph G Perform semantic enhancement processing ， Get enhanced scene graph ; S3: Scene graph Input pre-trained joint shape-layout generation network, which consists of scene encoder and scene decoder; scene graph First, the scene encoder is used to extract features and obtain the scene graph The full-image feature vector of And output, the layout decoder for the full image feature vector Perform decoding processing to obtain the layout position mean of furniture objects , mean rotation angle , layout position variance , rotation angle variance , splicing to obtain the distribution parameters ; Sample the distribution parameters to obtain the potential layout vector , using the latent layout vector Collection Replacing the scene graph The furniture layout position information in the scene graph is obtained ; S4: Scene Encoder using a pre-trained joint shape-layout generation network to generate scene graphs Perform feature extraction to obtain a scene graph The full-image feature vector of , the whole image feature vector Input the layout decoder and shape decoder of the scene decoder respectively, decode and process to obtain the scene graph The spatial position information and three-dimensional shape information of each furniture object in the scene decoder are input into the VQ-VAE decoder to reconstruct the three-dimensional layout of the ship cabin furniture.

2. The method for automatically generating three-dimensional layout of ship cabin furniture according to claim 1, characterized in that: The layout decoder consists of two sets of parallel stacked multi-layer perceptrons. The first set of multi-layer perceptrons is used to predict the 3D bounding box information of each furniture object. b ; The second set of multi-layer perceptrons is used to predict the rotation angle of each furniture object in the vertical axis direction ; The layout position information generated by the layout decoder is ,symbol Represents vector concatenation.

3. The method for automatically generating three-dimensional layout of ship cabin furniture according to claim 1, characterized in that: The shape decoder consists of a conditional diffusion model and a set of multi-layer perceptrons; the multi-layer perceptron is used to extract the shape latent variable encoding from the full image feature vector c And output, the input of the multi-layer perceptron is the full image feature vector output by the scene encoder; the conditional diffusion model encodes the shape latent variable output by the multi-layer perceptron c Generate the real 3D layout of furniture objects in the cabin as a guide condition - shape encoding .

4. The method for automatically generating three-dimensional layout of ship cabin furniture according to claim 3, characterized in that: The conditional diffusion model is a Latent Diffusion Model diffusion model; the diffusion model is encoded with shape latent variables output by a multi-layer perceptron. c Generate the real 3D layout of furniture objects in the cabin as a guide condition - shape encoding The specific operations are: (1) Gradually add random noise to the shape latent variable encoding c , get any time step The noise representation : ， in, , , is the diffusion scheduling sequence; It represents the original signal retention coefficient at each step, which is used to describe the k The proportion of the original signal retained in the step; represents the random noise added; (2) From Gaussian noise First, use 3D-UNet network to restore and get the true 3D layout-shape encoding .

5. The method for automatically generating three-dimensional layout of ship cabin furniture according to claim 1, characterized in that: The specific operation of step S1 is as follows: input the ship cabin CAD drawing into the pre-trained primitive-text dual-channel fusion recognition model, use the primitive-text dual-channel fusion recognition model to recognize and extract the primitive information of the ship cabin in the CAD drawing, and then use the primitive-text dual-channel fusion recognition model to construct the initial scene map of the ship cabin based on the primitive information. G。 6. The method for automatically generating three-dimensional layout of ship cabin furniture according to claim 5, characterized in that: The primitive-text dual-channel fusion recognition model is composed of a text encoder, a visual encoder, a large language model, a hidden layer and a large model Head prediction layer; wherein the text encoder is used to convert the text information in the ship cabin CAD drawing into vector data that can be recognized by a computer, the input of the text encoder is the text information in the ship cabin CAD drawing, and the output is a numerical vector that can be recognized by a computer; the visual encoder is used to convert the primitive information in the ship cabin CAD drawing into vector data that can be recognized by a computer, the input of the visual encoder is the primitive information in the ship cabin CAD drawing, and the output is a numerical vector that can be recognized by a computer; the numerical vector data that can be recognized by a computer output by the text encoder and the visual encoder are processed by vector addition. The large language model is used to unify the merged vector into the same semantic space, obtain a semantic vector and output it, the input of the large language model is the merged vector, and the output is the semantic vector; the hidden layer is used to perform feature extraction on the semantic vector output by the large language model to obtain an intermediate feature numerical vector and output it, the input of the hidden layer is the semantic vector, and the output is the intermediate feature numerical vector; the large model Head prediction layer is used to complete classification prediction on the intermediate feature numerical vector, the input of the large model Head prediction layer is the intermediate feature numerical vector output by the hidden layer, and the output is the graphic element information in the ship cabin CAD drawing.

7. The method for automatically generating three-dimensional layout of ship cabin furniture according to claim 1, characterized in that: The specific operations of step S2 are: S201: Based on the initial scene graph , retrieved from the standardized knowledge base consisting of specification documents related to ship interior design, and obtained The design specifications and empirical constraints related to the corresponding cabin type, furniture category, and spatial constraint relationship between furniture objects are obtained, and the semantic knowledge of each design specification and empirical constraint is defined in the form of triples to obtain the triple semantic knowledge of each design specification and empirical constraint. , ,in, Indicates the subject object , Indicates the object. Represents predicates or / and relationships; combines all retrieved design specifications with triple semantic knowledge of empirical constraints Collection, get the initial scene graph G Scenario standard knowledge set ; S202: From the scene standard knowledge collection Each triple in the semantic knowledge Extract text information from the text, and convert the extracted text information into a natural language subject-verb-object text input visual language model for text encoding processing, and obtain the semantic knowledge of each triple Semantic vector embedding ( );in, Representing triple semantic knowledge The semantic vector of the overall relationship, Represents the semantic vector of the subject, The semantic vector representing the object; S203: Gathering scene standard knowledge All triple semantic knowledge in As a structured input to the large language model, the initial scene graph As structured text input into the large language model, the large language model is used to extract semantic features to obtain the initial scene graph Scene-level global vector embedding And output; S204: Semantic knowledge of each triple obtained in step S202 Semantic vector embedding ( ) and the scene-level global vector embedding obtained in step S203 Initial scene graph Perform semantic enhancement to obtain an enhanced scene graph , ,in, Represents each furniture object and the relationship is embedded as the initial feature vector, Represented as furniture layout position information embedded from CAD drawings, represents the semantic vector embedding obtained by text encoding using the visual language model, represents the semantic vector embedding of the overall relationship obtained by using the visual language model for the text encoder, Represents the initial scene graph The scene-level global vector embedding of Represents the scene graph The initial eigenvector set of the relation in , symbol Refers to vector concatenation.

8. The method for automatically generating three-dimensional layout of ship cabin furniture according to claim 7, characterized in that: The visual language model is the CLIP visual language model; the large language model is GPT-4o.

9. A computer device, characterized in that: include: a processor and a memory communicatively connected to the processor; The memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement the method for automatically generating three-dimensional layout of ship cabin furniture according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method for automatically generating a three-dimensional arrangement of ship cabin furniture according to any one of claims 1 to 8.

Images