A BIM modeling method and device based on a multi-modal modeling code generation model, an electronic device, and a storage medium

Abstract

The application discloses a BIM modeling method and device based on a multi-modal modeling code generation model, electronic equipment and a storage medium, and belongs to the field of digital modeling. The method comprises the following steps: acquiring a modeling requirement text and an image to be modeled; inputting the modeling requirement text and the image to be modeled into a multi-modal modeling code generation model, so that the multi-modal modeling code generates a BIM modeling code according to the modeling requirement text and the image to be modeled; correcting the BIM modeling code according to a preset regular expression to generate a corrected BIM modeling code; and calling a BIM modeling software, so that the BIM modeling software generates a BIM three-dimensional model according to the corrected BIM modeling code. Therefore, by implementing the application, the problem that the code deviates from the requirement due to the single modal input in the BIM modeling of the prior art can be solved.

Description

Technical Field

[0001] This invention relates to the field of digital modeling technology, specifically to a BIM modeling method, apparatus, electronic device, and storage medium based on multimodal modeling code generation model. Background Technology

[0002] As construction projects become increasingly complex, Building Information Modeling (BIM) technology is playing a crucial role in architectural design, construction, and operation and maintenance management. BIM not only enables 3D visualization modeling, collaborative design, and refined management, but also enhances information integration and delivery efficiency throughout the building lifecycle, becoming a core support for driving the digital transformation of the construction industry. A high-quality BIM modeling process can effectively improve the quality of architectural design schemes, reduce design changes and construction conflicts, and save project costs, demonstrating significant engineering value and economic benefits.

[0003] Despite the emergence of numerous BIM automation tools, most current mainstream solutions generate code using single-modal inputs (such as text or images only), lacking the ability to fuse and analyze multi-source heterogeneous data and perform cross-modal semantic alignment. This results in a significant gap between the generated code and actual requirements. Furthermore, the code correction process heavily relies on manual, one-by-one proofreading, which is inefficient and fails to meet the stringent requirements of rapid iteration in today's construction projects. Summary of the Invention

[0004] This invention provides a BIM modeling method, apparatus, electronic device, and storage medium based on a multimodal modeling code generation model, which can solve the problem of code deviation from requirements caused by the use of single modal input in existing BIM modeling.

[0005] An embodiment of the present invention provides a BIM modeling method based on a multimodal modeling code generation model, wherein the multimodal modeling code generation model includes a preprocessing layer, an encoder layer, a feature fusion layer, and a decoder layer; the BIM modeling method includes:

[0006] Obtain the modeling requirements text and the image to be modeled;

[0007] The modeling requirement text and the image to be modeled are input into the multimodal modeling code generation model. The multimodal modeling code generation model preprocesses the modeling requirement text and the image to be modeled through a preprocessing layer to generate text segmentation sequences and image block sequences. The encoder layer encodes the text segmentation sequences and image block sequences to generate text features and image features. The feature fusion layer fuses the text features and image features to generate text-image fusion features. The decoder layer decodes the text-image fusion features to generate BIM modeling code.

[0008] The BIM modeling code is modified according to a preset regular expression to generate modified BIM modeling code;

[0009] Call the BIM modeling software so that it can generate a BIM 3D model based on the modified BIM modeling code.

[0010] Furthermore, the preprocessing layer includes a text segmenter, a code segmenter, and an image blocker; the encoder layer includes a text encoder, a code encoder, and an image encoder.

[0011] The training of the multimodal modeling code generation model includes:

[0012] Obtain a text-code-image triplet dataset; wherein the text-code-image triplet dataset contains several triplets, each triplet containing training modeling requirement text, corresponding training BIM modeling code, and corresponding reference modeling image; wherein the training BIM modeling code is a parameterized code segment generated based on the training modeling requirement text, and calling BIM modeling software to execute the BIM modeling code can generate a BIM 3D model consistent with the reference modeling image;

[0013] The text-code-image ternary dataset is randomly divided into several batches of first training samples according to a preset number;

[0014] The first training samples of each batch are sequentially input into the multimodal modeling code generation model to train the text encoder, code encoder and image encoder in the multimodal modeling code generation model until the preset number of training times is reached.

[0015] Among them, the multimodal modeling code generation model segments the training modeling requirement text into words using a text segmenter when it receives the first training sample of each batch, and generates a text segmentation sequence.

[0016] The training BIM modeling code is segmented using a code segmenter to generate a code segmentation sequence;

[0017] The reference modeling image is divided into blocks using an image block divider to generate an image block sequence;

[0018] Text features are generated by encoding the segmented text sequence using a text encoder.

[0019] The code segmentation sequence is encoded using a code encoder to generate code features;

[0020] Image features are generated by encoding image block sequences using an image encoder.

[0021] Based on the InfoNCE loss function, the first loss function value is calculated through contrastive learning training, and the optimizer is used to update the text encoder, code encoder, and image encoder based on the first loss function value.

[0022] After training the text encoder, code encoder, and image encoder in the multimodal modeling code generation model, the text-code-image ternary dataset is randomly divided into several batches of second training samples according to a preset number. During the training phase, the training modeling requirement text and the corresponding reference modeling image in the second training samples are used as the forward input of the model, and the corresponding training BIM modeling code is used as the modeling code label.

[0023] The second training samples of each batch are sequentially input into the multimodal modeling code generation model to train the feature fusion layer and decoder layer in the multimodal modeling code generation model until the preset number of training times is reached.

[0024] Among them, the multimodal modeling code generation model segments the training modeling requirement text into words using a text segmenter when it receives a batch of second training samples, generating a text segmentation sequence.

[0025] The reference modeling image is divided into blocks using an image block divider to generate an image block sequence;

[0026] Text features are generated by encoding the segmented text sequence using a text encoder.

[0027] Image features are generated by encoding image block sequences using an image encoder.

[0028] The text and image features are fused through a feature fusion layer to generate text-image fused features.

[0029] The text-image fusion features are decoded by the decoder layer to generate the predicted BIM modeling code corresponding to the second training sample;

[0030] Based on the predicted BIM modeling code and the corresponding modeling code label, a second loss function value is calculated using the cross-entropy loss function; the optimizer is then used to update the feature fusion layer and the decoder layer based on the second loss function value.

[0031] Furthermore, obtaining the modeling requirement text and the image to be modeled includes:

[0032] Generate an interactive interface for users to upload text containing modeling requirements and images to be modeled; wherein the interactive interface includes a text upload module and an image upload module; the text upload module integrates a speech recognition engine and a text editing component; the image upload module integrates an image editing component;

[0033] When a user uploads a voice file through the text upload module, the voice file is recognized by the voice recognition engine to generate modeling requirement text.

[0034] When a user uploads text content through the text upload module, the text content is formatted using the text editing component to generate modeling requirement text.

[0035] When a user uploads an image file through the image upload module, the image file is formatted using the image editing component to generate an image to be modeled.

[0036] Furthermore, after invoking the BIM modeling software to generate a BIM 3D model based on the modified BIM modeling code, the process further includes:

[0037] The modeling requirement text is displayed in the first display area of ​​the preset multi-dimensional visualization interface;

[0038] The image to be modeled is displayed in the second display area of ​​the multi-dimensional visualization interface;

[0039] The revised BIM modeling code is displayed in the third display area of ​​the multi-dimensional visualization interface;

[0040] The BIM 3D model is displayed in the fourth display area of ​​the multi-dimensional visualization interface;

[0041] The first display area, the second display area, the third display area, and the fourth display area are independently distributed on the multi-dimensional visualization interface.

[0042] Furthermore, the regular expressions include naming convention regular expressions, parameter format regular expressions, syntax integrity regular expressions, and field mapping regular expressions;

[0043] The step of modifying the BIM modeling code according to a preset regular expression to generate modified BIM modeling code includes:

[0044] Based on the naming convention regular expression, the entity names in the BIM modeling code are standardized and corrected to generate standard entity name codes;

[0045] Based on the parameter format regular expression, the parameter value format in the standard entity name code is verified and corrected for compliance, and a parameter compliance code is generated.

[0046] Based on the grammatical integrity regular expression, for the parameter-compliant code, complete the missing syntax symbols and close the code structure to generate grammatically closed code;

[0047] Based on the field mapping regular expression, the fields in the grammatical closure code are semantically aligned with the modeling requirement text to generate the corrected BIM modeling code.

[0048] Based on the above method embodiments, the present invention provides corresponding apparatus embodiments.

[0049] One embodiment of the present invention provides a BIM modeling device based on multimodal modeling code generation, comprising: a model input data acquisition module, a modeling code generation module, a code correction module, and a three-dimensional model generation module;

[0050] The model input data acquisition module is used to acquire the modeling requirement text and the image to be modeled;

[0051] The modeling code generation module is used to input modeling requirement text and the image to be modeled into the multimodal modeling code generation model. The multimodal modeling code generation model then preprocesses the modeling requirement text and the image to be modeled through a preprocessing layer to generate text segmentation sequences and image block sequences; it encodes the text segmentation sequences and image block sequences through an encoder layer to generate text features and image features; it fuses the text features and image features through a feature fusion layer to generate text-image fusion features; and it decodes the text-image fusion features through a decoder layer to generate BIM modeling code.

[0052] The code correction module is used to correct the BIM modeling code according to a preset regular expression, and generate corrected BIM modeling code.

[0053] The 3D model generation module is used to call the BIM modeling software so that the BIM modeling software can generate a BIM 3D model based on the modified BIM modeling code.

[0054] Furthermore, the BIM modeling device based on multimodal modeling code generation model, the model input data acquisition module includes: an interactive interface generation unit, a modeling requirement text generation unit, and an image unit to be modeled.

[0055] The interactive interface generation unit is used to generate an interactive interface for users to upload modeling requirement text and images to be modeled; wherein, the interactive interface includes a text upload module and an image upload module; the text upload module integrates a speech recognition engine and a text editing component; the image upload module integrates an image editing component;

[0056] The modeling requirement text generation unit is used to generate modeling requirement text by recognizing the voice file using a speech recognition engine when a user uploads a voice file through the text upload module; and to generate modeling requirement text by formatting the text content using a text editing component when a user uploads text content through the text upload module.

[0057] The image unit to be modeled is used to format the image file according to the image editing component and generate the image to be modeled when the image file is detected to be uploaded by the user through the image upload module.

[0058] Furthermore, the BIM modeling device based on multimodal modeling code generation model also includes: a visualization display module;

[0059] The visualization display module is used to display the modeling requirement text in the first display area of ​​the preset multi-dimensional visualization interface;

[0060] The image to be modeled is displayed in the second display area of ​​the multi-dimensional visualization interface;

[0061] The revised BIM modeling code is displayed in the third display area of ​​the multi-dimensional visualization interface;

[0062] The BIM 3D model is displayed in the fourth display area of ​​the multi-dimensional visualization interface;

[0063] The first display area, the second display area, the third display area, and the fourth display area are independently distributed on the multi-dimensional visualization interface.

[0064] Based on the above method embodiments, the present invention provides corresponding electronic device embodiments.

[0065] One embodiment of the present invention provides an electronic device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the BIM modeling method based on multimodal modeling code generation model as described in any of the above-described method embodiments.

[0066] Based on the above method embodiments, the present invention provides corresponding storage medium embodiments.

[0067] One embodiment of the present invention provides a storage medium storing a computer program thereon, wherein, when the computer program is running, it controls the device where the storage medium is located to execute the BIM modeling method based on multimodal modeling code generation model as described in any of the above-described method embodiments.

[0068] Compared with the prior art, the present invention has the following beneficial effects:

[0069] This invention provides a BIM modeling method, apparatus, electronic device, and storage medium based on a multimodal modeling code generation model. The method first acquires modeling requirement text and corresponding images, and inputs them into a multimodal modeling code generation model. The model first performs word segmentation on the text and block segmentation on the images in a preprocessing layer, and then extracts text and image features through an encoder layer. Next, the two are fused in a feature fusion layer to generate fused features, and finally, a decoder generates preliminary BIM modeling code. Subsequently, the code is automatically corrected using preset regular expressions to obtain final code that conforms to specifications, and BIM software is called to construct a 3D model based on this code.

[0070] This technical solution uses a multimodal modeling code generation model to collaboratively process text and image inputs, and utilizes a feature fusion layer to achieve cross-modal semantic alignment, thus solving the problem of code deviation from requirements caused by single-modal input. At the same time, it uses regular expressions to automatically correct the generated code, significantly reducing the reliance on manual proofreading and improving correction efficiency. Finally, it calls BIM modeling software to generate standardized 3D models, meeting the needs of rapid iteration in today's construction engineering projects. Attached Figure Description

[0071] Figure 1 This is a flowchart illustrating a BIM modeling method based on multimodal modeling code generation, provided by an embodiment of the present invention.

[0072] Figure 2 This is a schematic diagram of the structure of a BIM modeling device based on a multimodal modeling code generation model provided in an embodiment of the present invention. Detailed Implementation

[0073] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0074] like Figure 1 As shown, to address the problem of code discrepancies between code and requirements caused by the use of a single modal input in existing BIM modeling technologies, an embodiment of the present invention provides a BIM modeling method based on a multimodal modeling code generation model. The multimodal modeling code generation model includes a preprocessing layer, an encoder layer, a feature fusion layer, and a decoder layer. The BIM modeling method includes at least the following steps:

[0075] Step S1: Obtain the modeling requirements text and the image to be modeled;

[0076] In a preferred embodiment, obtaining the modeling requirement text and the image to be modeled includes:

[0077] Generate an interactive interface for users to upload text containing modeling requirements and images to be modeled; wherein the interactive interface includes a text upload module and an image upload module; the text upload module integrates a speech recognition engine and a text editing component; the image upload module integrates an image editing component;

[0078] When a user uploads a voice file through the text upload module, the voice file is recognized by the voice recognition engine to generate modeling requirement text.

[0079] When a user uploads text content through the text upload module, the text content is formatted using the text editing component to generate modeling requirement text.

[0080] When a user uploads an image file through the image upload module, the image file is formatted using the image editing component to generate an image to be modeled.

[0081] Specifically, an interactive interface supporting multimodal input is constructed. Users can submit modeling requirements via the text upload module, choosing either voice or text. Voice input is automatically converted into structured text by the built-in recognition engine, while text input undergoes keyword extraction and format standardization through a text editing component. Simultaneously, the image upload module receives reference images uploaded by users and automatically adjusts resolution, coordinate system, and scale parameters through an image editing component to generate an image conforming to modeling input specifications. This interface enables one-stop uploading and standardization of voice, text, and images, reducing the operational complexity of multi-source data entry. Through this interactive interface, users can conveniently upload multimodal modeling requirements, including voice, text, and images. The system automatically completes data parsing and format standardization, significantly improving input efficiency and reducing human translation error rates.

[0082] Step S2: Input the modeling requirement text and the image to be modeled into the multimodal modeling code generation model. The multimodal modeling code generation model preprocesses the modeling requirement text and the image to be modeled through a preprocessing layer to generate text segmentation sequences and image block sequences; it then encodes the text segmentation sequences and image block sequences through an encoder layer to generate text features and image features; finally, it fuses the text features and image features through a feature fusion layer to generate text-image fusion features; and finally, it decodes the text-image fusion features through a decoder layer to generate BIM modeling code.

[0083] Specifically, the modeling requirement text and the image to be modeled are input into the multimodal modeling code generation model. The model performs structured processing on the text and image through a preprocessing layer: semantic segmentation and keyword extraction are performed on the text content to generate a text segmentation sequence with contextual relevance; region segmentation and feature block division are performed on the image to generate an image block sequence containing geometric and material information. Subsequently, the encoder layer uses a deep neural network to encode the text segmentation sequence and the image block sequence respectively, extracting the semantic features of the text and the visual features of the image. The feature fusion layer uses a cross-modal attention mechanism to dynamically assign weights and fuse the text semantic features and image visual features to generate a context-aware fused feature vector. Based on the fused feature vector, the decoder layer parses and outputs parametric code that conforms to BIM syntax specifications layer by layer through a sequence generation algorithm.

[0084] In a preferred embodiment, the preprocessing layer includes a text segmenter, a code segmenter, and an image blocker; the encoder layer includes a text encoder, a code encoder, and an image encoder.

[0085] The training of the multimodal modeling code generation model includes:

[0086] Obtain a text-code-image triplet dataset; wherein the text-code-image triplet dataset contains several triplets, each triplet containing training modeling requirement text, corresponding training BIM modeling code, and corresponding reference modeling image; wherein the training BIM modeling code is a parameterized code segment generated based on the training modeling requirement text, and calling BIM modeling software to execute the BIM modeling code can generate a BIM 3D model consistent with the reference modeling image;

[0087] The text-code-image ternary dataset is randomly divided into several batches of first training samples according to a preset number;

[0088] The first training samples of each batch are sequentially input into the multimodal modeling code generation model to train the text encoder, code encoder and image encoder in the multimodal modeling code generation model until the preset number of training times is reached.

[0089] Among them, the multimodal modeling code generation model segments the training modeling requirement text into words using a text segmenter when it receives the first training sample of each batch, and generates a text segmentation sequence.

[0090] The training BIM modeling code is segmented using a code segmenter to generate a code segmentation sequence;

[0091] The reference modeling image is divided into blocks using an image block divider to generate an image block sequence;

[0092] Text features are generated by encoding the segmented text sequence using a text encoder.

[0093] The code segmentation sequence is encoded using a code encoder to generate code features;

[0094] Image features are generated by encoding image block sequences using an image encoder.

[0095] Based on the InfoNCE loss function, the first loss function value is calculated through contrastive learning training, and the optimizer is used to update the text encoder, code encoder, and image encoder based on the first loss function value.

[0096] After training the text encoder, code encoder, and image encoder in the multimodal modeling code generation model, the text-code-image ternary dataset is randomly divided into several batches of second training samples according to a preset number. During the training phase, the training modeling requirement text and the corresponding reference modeling image in the second training samples are used as the forward input of the model, and the corresponding training BIM modeling code is used as the modeling code label.

[0097] The second training samples of each batch are sequentially input into the multimodal modeling code generation model to train the feature fusion layer and decoder layer in the multimodal modeling code generation model until the preset number of training times is reached.

[0098] Among them, the multimodal modeling code generation model segments the training modeling requirement text into words using a text segmenter when it receives a batch of second training samples, generating a text segmentation sequence.

[0099] The reference modeling image is divided into blocks using an image block divider to generate an image block sequence;

[0100] Text features are generated by encoding the segmented text sequence using a text encoder.

[0101] Image features are generated by encoding image block sequences using an image encoder.

[0102] The text and image features are fused through a feature fusion layer to generate text-image fused features.

[0103] The text-image fusion features are decoded by the decoder layer to generate the predicted BIM modeling code corresponding to the second training sample;

[0104] Based on the predicted BIM modeling code and the corresponding modeling code label, a second loss function value is calculated using the cross-entropy loss function; the optimizer is then used to update the feature fusion layer and the decoder layer based on the second loss function value.

[0105] Specifically, the training of the multimodal modeling code generation model is based on a text-code-image triplet dataset. Each triplet contains the training modeling requirement text, the corresponding parametric BIM modeling code, and a reference modeling image. The modeling code, once executed, generates a 3D model identical to the reference image. The training process consists of two stages: In the first stage, the dataset is divided into batch input models. Text, code, and image segments are structured using a text segmenter, a code segmenter, and an image block segmenter, respectively, generating text segmentation sequences, code segmentation sequences, and image block sequences. The text encoder, code encoder, and image encoder extract features from each sequence, and the encoder parameters are optimized based on the contrastive learning loss function (InfoNCE) to align features from different modalities in a shared space. In the second stage, the dataset is re-divided. The encoder trained in the previous stage extracts text and image features, and after cross-modal semantic fusion through a feature fusion layer, these features are input into the decoder to generate predicted code. The fusion layer and decoder parameters are then optimized based on the cross-entropy loss between the predicted and real codes until the model converges. Through a two-stage training strategy, the model can accurately align the cross-modal semantics of text, images, and code, generating grammatically correct BIM modeling code that is highly consistent with requirements, significantly reducing manual correction costs and improving modeling efficiency.

[0106] Step S3: Modify the BIM modeling code according to the preset regular expression to generate the modified BIM modeling code;

[0107] In a preferred embodiment, the regular expression includes naming convention regular expressions, parameter format regular expressions, syntax integrity regular expressions, and field mapping regular expressions;

[0108] The step of modifying the BIM modeling code according to a preset regular expression to generate modified BIM modeling code includes:

[0109] Based on the naming convention regular expression, the entity names in the BIM modeling code are standardized and corrected to generate standard entity name codes;

[0110] Based on the parameter format regular expression, the parameter value format in the standard entity name code is verified and corrected for compliance, and a parameter compliance code is generated.

[0111] Based on the grammatical integrity regular expression, for the parameter-compliant code, complete the missing syntax symbols and close the code structure to generate grammatically closed code;

[0112] Based on the field mapping regular expression, the fields in the grammatical closure code are semantically aligned with the modeling requirement text to generate the corrected BIM modeling code.

[0113] Specifically, the regular expressions include four categories of rules: naming conventions, parameter formatting, syntactic integrity, and field mapping. Naming convention regular expressions constrain the naming rules of entity names (e.g., correcting "WALL" to "IFCWALL"). Parameter formatting regular expressions validate and correct format errors in parameter values ​​(e.g., completing length units and filtering illegal negative values). Syntactic integrity regular expressions complete missing symbols (e.g., entity declaration symbol "#", assignment operator "=", semicolon ";") and close code block structures. Field mapping regular expressions force the alignment of semantic fields such as material and component type in the code based on a keyword library of the modeling requirements text (e.g., mapping "concrete" to CONCRETE). During correction, the above regular expressions are applied sequentially, processing layer by layer from entity naming correction, parameter format compliance, syntactic structure closure to semantic field mapping, ultimately generating BIM modeling code that conforms to the IFC standard and is consistent with user requirements.

[0114] Step S4: Call the BIM modeling software so that the BIM modeling software can generate a BIM 3D model based on the modified BIM modeling code.

[0115] Specifically, the software calls the standard API interface of the BIM modeling software, imports the modified BIM modeling code into the software kernel in IFC file format, and automatically constructs 3D model components and associates material properties by parsing entity declarations, parameter assignments, and structural relationships in the code. This ultimately generates a visual BIM 3D model that is completely consistent with the code description. The software kernel simultaneously performs geometric verification and collision detection to ensure that the model meets building codes and physical feasibility requirements.

[0116] In a preferred embodiment, after invoking the BIM modeling software to generate a BIM 3D model based on the modified BIM modeling code, the method further includes:

[0117] The modeling requirement text is displayed in the first display area of ​​the preset multi-dimensional visualization interface;

[0118] The image to be modeled is displayed in the second display area of ​​the multi-dimensional visualization interface;

[0119] The revised BIM modeling code is displayed in the third display area of ​​the multi-dimensional visualization interface;

[0120] The BIM 3D model is displayed in the fourth display area of ​​the multi-dimensional visualization interface;

[0121] The first display area, the second display area, the third display area, and the fourth display area are independently distributed on the multi-dimensional visualization interface.

[0122] Specifically, in the pre-defined multi-dimensional visualization interface, the first display area shows the user-inputted modeling requirement text in real time, and highlights key parameters (such as component type and geometric dimensions) based on semantic parsing; the second display area synchronously presents a segmented and annotated view of the image to be modeled, with the annotated areas associated with the component positions described in the text; the third display area displays the corrected BIM modeling code in syntax highlighting, and identifies fields in the code that are associated with text and images; the fourth display area embeds the 3D rendering engine of the BIM modeling software, dynamically displaying the generated BIM 3D model, supporting rotation, scaling, and viewing of component attributes. The four display areas are independently distributed in a four-quadrant split-screen layout, and cross-area interaction is achieved through an event bus (such as automatically locating the corresponding component in the model by clicking on a code field), ensuring collaborative visualization and operational consistency of multi-source data.

[0123] like Figure 2 As shown, an embodiment of the present invention provides a BIM modeling device based on multimodal modeling code generation, including: a model input data acquisition module, a modeling code generation module, a code correction module, and a three-dimensional model generation module.

[0124] The model input data acquisition module is used to acquire the modeling requirement text and the image to be modeled;

[0125] The modeling code generation module is used to input modeling requirement text and the image to be modeled into the multimodal modeling code generation model. The multimodal modeling code generation model then preprocesses the modeling requirement text and the image to be modeled through a preprocessing layer to generate text segmentation sequences and image block sequences; it encodes the text segmentation sequences and image block sequences through an encoder layer to generate text features and image features; it fuses the text features and image features through a feature fusion layer to generate text-image fusion features; and it decodes the text-image fusion features through a decoder layer to generate BIM modeling code.

[0126] The code correction module is used to correct the BIM modeling code according to a preset regular expression, and generate corrected BIM modeling code.

[0127] The 3D model generation module is used to call the BIM modeling software so that the BIM modeling software can generate a BIM 3D model based on the modified BIM modeling code.

[0128] In a preferred embodiment, the BIM modeling device based on multimodal modeling code to generate a model, the model input data acquisition module includes: an interactive interface generation unit, a modeling requirement text generation unit, and an image to be modeled unit;

[0129] The interactive interface generation unit is used to generate an interactive interface for users to upload modeling requirement text and images to be modeled; wherein, the interactive interface includes a text upload module and an image upload module; the text upload module integrates a speech recognition engine and a text editing component; the image upload module integrates an image editing component;

[0130] The modeling requirement text generation unit is used to generate modeling requirement text by recognizing the voice file using a speech recognition engine when a user uploads a voice file through the text upload module; and to generate modeling requirement text by formatting the text content using a text editing component when a user uploads text content through the text upload module.

[0131] The image unit to be modeled is used to format the image file according to the image editing component and generate the image to be modeled when the image file is detected to be uploaded by the user through the image upload module.

[0132] In a preferred embodiment, the BIM modeling device based on multimodal modeling code generation model further includes: a visualization display module;

[0133] The visualization display module is used to display the modeling requirement text in the first display area of ​​the preset multi-dimensional visualization interface;

[0134] The image to be modeled is displayed in the second display area of ​​the multi-dimensional visualization interface;

[0135] The revised BIM modeling code is displayed in the third display area of ​​the multi-dimensional visualization interface;

[0136] The BIM 3D model is displayed in the fourth display area of ​​the multi-dimensional visualization interface;

[0137] The first display area, the second display area, the third display area, and the fourth display area are independently distributed on the multi-dimensional visualization interface.

[0138] It should be noted that the embodiments of the device described above correspond to the embodiments of the present invention described above, and can realize the BIM modeling method based on multimodal modeling code generation model as described in any one of the above embodiments of the present invention. Furthermore, the embodiments of the device described above are merely illustrative. The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the accompanying drawings of the device embodiments provided by the present invention, the connection relationship between modules indicates that they have a communication connection, which can be specifically implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without creative effort.

[0139] Based on the above-described method embodiments of the present invention, a corresponding embodiment of an electronic device is provided.

[0140] An embodiment of the present invention provides an electronic device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the BIM modeling method based on multimodal modeling code generation model as described in any one of the present invention, or, when the processor executes the computer program, it implements the functions of each module in the above-described device embodiments.

[0141] For example, the computer program may be divided into one or more modules, which are stored in the memory and executed by the processor to complete the present invention. The one or more modules may be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program in the terminal device.

[0142] The terminal device may be a desktop computer, laptop, handheld computer, or cloud server, etc. The terminal device may include, but is not limited to, a processor and a memory.

[0143] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the terminal device, connecting all parts of the terminal device via various interfaces and lines.

[0144] The memory can be used to store the computer programs and / or modules. The processor implements various functions of the terminal device by running or executing the computer programs and / or modules stored in the memory and by calling data stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function, etc.; the data storage area may store data created based on the use of the mobile phone, etc. In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, RAM, plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0145] Based on the above method embodiments, the present invention provides corresponding storage medium embodiments;

[0146] Another embodiment of the present invention provides a storage medium including a stored computer program, wherein, when the computer program is running, the device where the storage medium is located executes any of the above-described BIM modeling methods based on multimodal modeling code generation models.

[0147] The aforementioned storage medium is a computer-readable storage medium. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or certain intermediate forms. The computer-readable medium can include any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0148] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0149] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A BIM modeling method based on multimodal modeling code generation model, characterized in that, The multimodal modeling code generation model includes a preprocessing layer, an encoder layer, a feature fusion layer, and a decoder layer; The BIM modeling method includes: Obtain the modeling requirements text and the image to be modeled; The modeling requirement text and the image to be modeled are input into the multimodal modeling code generation model. The multimodal modeling code generation model preprocesses the modeling requirement text and the image to be modeled through a preprocessing layer to generate text segmentation sequences and image block sequences. The encoder layer encodes the text segmentation sequences and image block sequences to generate text features and image features. The feature fusion layer fuses the text features and image features to generate text-image fusion features. The decoder layer decodes the text-image fusion features to generate BIM modeling code. The BIM modeling code is modified according to a preset regular expression to generate modified BIM modeling code; Call the BIM modeling software so that it can generate a BIM 3D model based on the modified BIM modeling code.

2. The BIM modeling method based on multimodal modeling code generation model as described in claim 1, characterized in that, The preprocessing layer includes a text segmenter, a code segmenter, and an image blocker; the encoder layer includes a text encoder, a code encoder, and an image encoder. The training of the multimodal modeling code generation model includes: Obtain a text-code-image triplet dataset; wherein the text-code-image triplet dataset contains several triplets, each triplet containing training modeling requirement text, corresponding training BIM modeling code, and corresponding reference modeling image; wherein the training BIM modeling code is a parameterized code segment generated based on the training modeling requirement text, and calling BIM modeling software to execute the BIM modeling code can generate a BIM 3D model consistent with the reference modeling image; The text-code-image ternary dataset is randomly divided into several batches of first training samples according to a preset number; The first training samples of each batch are sequentially input into the multimodal modeling code generation model to train the text encoder, code encoder and image encoder in the multimodal modeling code generation model until the preset number of training times is reached. Among them, the multimodal modeling code generation model segments the training modeling requirement text into words using a text segmenter when it receives the first training sample of each batch, and generates a text segmentation sequence. The training BIM modeling code is segmented using a code segmenter to generate a code segmentation sequence; The reference modeling image is divided into blocks using an image block divider to generate an image block sequence; Text features are generated by encoding the segmented text sequence using a text encoder. The code segmentation sequence is encoded using a code encoder to generate code features; Image features are generated by encoding image block sequences using an image encoder. Based on the InfoNCE loss function, the first loss function value is calculated through contrastive learning training, and the optimizer is used to update the text encoder, code encoder, and image encoder based on the first loss function value. After training the text encoder, code encoder, and image encoder in the multimodal modeling code generation model, the text-code-image ternary dataset is randomly divided into several batches of second training samples according to a preset number. During the training phase, the training modeling requirement text and the corresponding reference modeling image in the second training samples are used as the forward input of the model, and the corresponding training BIM modeling code is used as the modeling code label. The second training samples of each batch are sequentially input into the multimodal modeling code generation model to train the feature fusion layer and decoder layer in the multimodal modeling code generation model until the preset number of training times is reached. Among them, the multimodal modeling code generation model segments the training modeling requirement text into words using a text segmenter when it receives a batch of second training samples, generating a text segmentation sequence. The reference modeling image is divided into blocks using an image block divider to generate an image block sequence; Text features are generated by encoding the segmented text sequence using a text encoder. Image features are generated by encoding image block sequences using an image encoder. The text and image features are fused through a feature fusion layer to generate text-image fused features. The text-image fusion features are decoded by the decoder layer to generate the predicted BIM modeling code corresponding to the second training sample; Based on the predicted BIM modeling code and the corresponding modeling code label, a second loss function value is calculated using the cross-entropy loss function; the optimizer is then used to update the feature fusion layer and the decoder layer based on the second loss function value.

3. The BIM modeling method based on multimodal modeling code generation model as described in claim 2, characterized in that, The process of obtaining the modeling requirement text and the image to be modeled includes: Generate an interactive interface for users to upload text containing modeling requirements and images to be modeled; wherein the interactive interface includes a text upload module and an image upload module; the text upload module integrates a speech recognition engine and a text editing component; the image upload module integrates an image editing component; When a user uploads a voice file through the text upload module, the voice file is recognized by the voice recognition engine to generate modeling requirement text. When a user uploads text content through the text upload module, the text content is formatted using the text editing component to generate modeling requirement text. When a user uploads an image file through the image upload module, the image file is formatted using the image editing component to generate an image to be modeled.

4. The BIM modeling method based on multimodal modeling code generation model as described in claim 3, characterized in that, After invoking the BIM modeling software to generate a BIM 3D model based on the modified BIM modeling code, the process further includes: The modeling requirement text is displayed in the first display area of ​​the preset multi-dimensional visualization interface; The image to be modeled is displayed in the second display area of ​​the multi-dimensional visualization interface; The revised BIM modeling code is displayed in the third display area of ​​the multi-dimensional visualization interface; The BIM 3D model is displayed in the fourth display area of ​​the multi-dimensional visualization interface; The first display area, the second display area, the third display area, and the fourth display area are independently distributed on the multi-dimensional visualization interface.

5. The BIM modeling method based on multimodal modeling code generation model as described in claim 4, characterized in that, The regular expressions include naming convention regular expressions, parameter format regular expressions, syntax integrity regular expressions, and field mapping regular expressions; The step of modifying the BIM modeling code according to a preset regular expression to generate modified BIM modeling code includes: Based on the naming convention regular expression, the entity names in the BIM modeling code are standardized and corrected to generate standard entity name codes; Based on the parameter format regular expression, the parameter value format in the standard entity name code is verified and corrected for compliance, and a parameter compliance code is generated. Based on the grammatical integrity regular expression, for the parameter-compliant code, complete the missing syntax symbols and close the code structure to generate grammatically closed code; Based on the field mapping regular expression, the fields in the grammatical closure code are semantically aligned with the modeling requirement text to generate the corrected BIM modeling code.

6. A BIM modeling device based on multimodal modeling code generation model, characterized in that, include: The module includes a model input data acquisition module, a modeling code generation module, a code correction module, and a 3D model generation module. The model input data acquisition module is used to acquire the modeling requirement text and the image to be modeled; The modeling code generation module is used to input modeling requirement text and the image to be modeled into the multimodal modeling code generation model. The multimodal modeling code generation model then preprocesses the modeling requirement text and the image to be modeled through a preprocessing layer to generate text segmentation sequences and image block sequences; it encodes the text segmentation sequences and image block sequences through an encoder layer to generate text features and image features; it fuses the text features and image features through a feature fusion layer to generate text-image fusion features; and it decodes the text-image fusion features through a decoder layer to generate BIM modeling code. The code correction module is used to correct the BIM modeling code according to a preset regular expression, and generate corrected BIM modeling code. The 3D model generation module is used to call the BIM modeling software so that the BIM modeling software can generate a BIM 3D model based on the modified BIM modeling code.

7. The BIM modeling device based on multimodal modeling code generation model as described in claim 6, characterized in that, The model input data acquisition module includes: an interactive interface generation unit, a modeling requirement text generation unit, and an image to be modeled unit; The interactive interface generation unit is used to generate an interactive interface for users to upload modeling requirement text and images to be modeled; wherein, the interactive interface includes a text upload module and an image upload module; the text upload module integrates a speech recognition engine and a text editing component; the image upload module integrates an image editing component; The modeling requirement text generation unit is used to generate modeling requirement text by recognizing the voice file using a speech recognition engine when a user uploads a voice file through the text upload module; and to generate modeling requirement text by formatting the text content using a text editing component when a user uploads text content through the text upload module. The image unit to be modeled is used to format the image file according to the image editing component and generate the image to be modeled when the image file is detected to be uploaded by the user through the image upload module.

8. The BIM modeling device based on multimodal modeling code generation model as described in claim 7, characterized in that, Also includes: Visualization module; The visualization display module is used to display the modeling requirement text in the first display area of ​​the preset multi-dimensional visualization interface; The image to be modeled is displayed in the second display area of ​​the multi-dimensional visualization interface; The revised BIM modeling code is displayed in the third display area of ​​the multi-dimensional visualization interface; The BIM 3D model is displayed in the fourth display area of ​​the multi-dimensional visualization interface; The first display area, the second display area, the third display area, and the fourth display area are independently distributed on the multi-dimensional visualization interface.

9. An electronic device, characterized in that, The method includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to implement the BIM modeling method based on a multimodal modeling code generation model as described in any one of claims 1 to 5.

10. A storage medium, characterized in that, The storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device where the storage medium is located to execute the BIM modeling method based on multimodal modeling code generation model as described in any one of claims 1 to 5.

Images