A three-dimensional building block model generation method, device, equipment and storage medium

Abstract

The application is suitable for the field of computer graphics, and provides a three-dimensional building block model generation method, device and equipment and storage medium. The method comprises the following steps: obtaining a reference image of a building block product. Image features in the reference image are encoded to obtain an image word sequence. The image word sequence is used as a conditional guide sequence to generate building block words one by one in an autoregressive manner, and the building block words are spliced to obtain a building block whole word sequence corresponding to the reference image. Based on a preset building block total word table, the building block words in the building block whole word sequence are analyzed to generate a product engineering file of the building block product, wherein the preset building block total word table is used to represent the mapping relationship between the geometric and assembly attribute information of a single building block component and the whole type encoding. According to the product engineering file of the building block product, a three-dimensional building block model corresponding to the building block product is generated. The method significantly improves the generation accuracy and structural assembly rationality of the three-dimensional building block model.

Description

Technical Field

[0001] This application belongs to the field of computer graphics technology, and in particular relates to a method, apparatus, device and storage medium for generating three-dimensional block models. Background Technology

[0002] In recent years, customized building block products and building block IP derivatives have emerged rapidly. Users want to convert real building facades, portraits, and other scenes from photos into physical building block models that can be purchased offline and assembled according to instructions. This requires 3D building block model generation systems to not only visually reproduce the main structure, decorative details, and colors in the reference images, but also to physically ensure the assemblability of the parts (no clipping, no floating, and assembly by ordinary users according to the assembly instructions).

[0003] However, existing methods for generating 3D block models often suffer from defects such as missing multi-attribute representations of parts, contradictory position quantization, and lack of assembly geometric constraints, resulting in low accuracy in generating block models. Summary of the Invention

[0004] This application provides a method, apparatus, device, and storage medium for generating three-dimensional block models, which can solve the problem of low accuracy in generating block models due to defects in existing three-dimensional block model generation methods, such as missing multi-attribute representations of parts, contradictory position quantization, and lack of assembly geometric constraints.

[0005] In a first aspect, embodiments of this application provide a method for generating a three-dimensional block model, including: Obtain reference images of the building block products; The image features in the reference image are encoded to obtain an image word sequence, wherein the image word sequence includes multiple image words, and the image words are used to characterize the visual feature information of the reference image; Using the image lexical sequence as a conditional guide sequence, building block lexicals are generated one by one through an autoregressive method, and then spliced ​​together to obtain a building block integer lexical sequence corresponding to the reference image. The building block integer lexical sequence includes multiple building block lexicals, and the building block lexicals are used to characterize the geometric and assembly attribute information of a single building block component. Based on a preset block vocabulary, the block words in the block integer word sequence are parsed to generate the product engineering file of the block product. The preset block vocabulary is used to characterize the mapping relationship between the geometric and assembly attribute information and the integer code of the single block component. Based on the product engineering file of the building block product, generate a three-dimensional building block model corresponding to the building block product.

[0006] In one possible implementation of the first aspect, encoding the image features in the reference image to obtain an image word sequence includes: The global image features in the reference image are encoded into global image terms, wherein the global image features are used to characterize the overall outline, global tone and global structural summary information of the building block product; Multiple block image features in the reference image are encoded into multiple local image terms, wherein the block image features are used to characterize the local texture, local color and local contour details of the building block product; The global image term is used as the first element of the sequence, and the multiple local image terms are placed after the global image term to obtain the image term sequence.

[0007] In one possible implementation of the first aspect, the preset block vocabulary is constructed by the union of multiple sub-vocabulary sets, wherein the multiple sub-vocabulary sets include a part type vocabulary set, a color vocabulary set, a three-dimensional orientation vocabulary set, a coordinate position vocabulary set, and a special control character vocabulary set, wherein the special control character vocabulary set includes a sequence start character, a sequence end character, and a fill character.

[0008] In one possible implementation of the first aspect, the block-type integer lexical sequence further includes a start special lexical and a stop special lexical; The step of using the image lexical sequence as a conditional guided sequence to generate building block lexical units one by one through an autoregressive method, and concatenating them to obtain a building block integer lexical sequence corresponding to the reference image, includes: The image word sequence is used as a fixed prefix and concatenated with the starting special word to obtain the initial context sequence; The initial context sequence is input into a pre-trained causal decoding network, and a causal mask is used to constrain the decoding field of view to output the hidden features corresponding to the current decoding step. The self-attention mechanism of the decoding step is used to associate multiple image words in the image word sequence so that the information carried by the image words is aligned with the spatial region of the reference image. The hidden features are mapped to the preset block vocabulary to predict the probability distribution of the block words corresponding to the current decoding step; The current building block word sampled from the probability distribution based on a preset sampling strategy is added to the end of the initial context sequence to obtain the updated context sequence; Repeat the word-by-word prediction process to expand the context sequence until the terminating special word is sampled, and then stop to obtain the target context sequence. The image lexical sequence is removed from the target context sequence to obtain the block-type integer lexical sequence.

[0009] In one possible implementation of the first aspect, the method includes adding the current building block word sampled from the probability distribution based on a preset sampling strategy to the end of the initial context sequence to obtain an updated context sequence: After generating a preset number of building block words, the preset number of building block words are parsed into the three-dimensional network entity corresponding to the current single building block component according to the preset building block vocabulary. Based on the three-dimensional network entities corresponding to the historical single building block components that have passed verification, the three-dimensional network entities of the current single building block component are subjected to assembly conflict verification, wherein the assembly conflict verification includes at least geometric clipping verification and floating assembly verification. If an assembly conflict is detected, the non-compliant block terms within the current single block component are masked and resampled to obtain corrected block terms.

[0010] In one possible implementation of the first aspect, the step of parsing the block lexicons in the block integer lexicon sequence based on a preset block lexicon to generate the product engineering file of the block product includes: Based on the preset block vocabulary, the block words in the block integer word sequence are parsed to obtain the metadata information corresponding to each single block component; Based on the metadata information of the individual building blocks, all individual building blocks are sorted according to a preset spatial dimension priority to generate the product engineering file of the building block product.

[0011] In one possible implementation of the first aspect, the step of sorting all single building block components according to a preset spatial dimension priority based on the metadata information of the single building block components to generate the product engineering file of the building block product includes: Sort all single building block components in ascending order with Z-axis as the first priority, Y-axis as the second priority, and X-axis as the third priority to generate an assembly sequence; All individual building blocks are assembled and grouped according to the Z-axis layering to obtain the grouping number of each individual building block; According to the assembly sequence, each of the single building blocks is assigned an assembly sequence number; By integrating the metadata information of all individual building blocks, the grouping number, and the assembly sequence number, the product engineering file is obtained.

[0012] Secondly, embodiments of this application provide a three-dimensional block model generation apparatus, comprising: The acquisition module is used to acquire reference images of the building block products; An encoding module is used to encode the image features in the reference image to obtain an image word sequence, wherein the image word sequence includes multiple image words, and the image words are used to characterize the visual feature information of the reference image; The prediction module is used to generate building block words one by one through an autoregressive method using the image word sequence as a conditional guide sequence, and splice them to obtain a building block integer word sequence corresponding to the reference image. The building block integer word sequence includes multiple building block words, and the building block words are used to characterize the geometric and assembly attribute information of a single building block component. The generation module is used to parse the building block words in the building block integer word sequence based on a preset building block vocabulary and generate the product engineering file of the building block product. The preset building block vocabulary is used to characterize the mapping relationship between the geometric and assembly attribute information and the integer code of the single building block component. The building module is used to generate a three-dimensional building block model corresponding to the building block product based on the product engineering file of the building block product.

[0013] Thirdly, embodiments of this application provide a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements a three-dimensional block model generation method as described in any of the above claims.

[0014] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements a three-dimensional block model generation method as described in any of the preceding claims.

[0015] Fifthly, embodiments of this application provide a computer program product that, when run on a terminal device, causes the terminal device to execute a three-dimensional block model generation method as described in any of the first aspects above.

[0016] The beneficial effects of the embodiments of this application compared with the prior art are: This application provides a method for generating a 3D building block model, comprising: First, acquiring a reference image of a building block product. Encoding the image features in the reference image to obtain an image term sequence, wherein the image term sequence includes multiple image terms used to represent the visual feature information of the reference image. Then, using the image term sequence as a conditional guide sequence, generating building block terms one by one through an autoregressive method, and concatenating them to obtain a building block integer term sequence corresponding to the reference image, wherein the building block integer term sequence includes multiple building block terms used to represent the geometric and assembly attribute information of a single building block component. Next, based on a preset building block vocabulary, parsing the building block terms in the building block integer term sequence to generate a product engineering file for the building block product, wherein the preset building block vocabulary is used to represent the mapping relationship between the geometric and assembly attribute information of a single building block component and the integer encoding. Finally, generating a 3D building block model corresponding to the building block product based on the product engineering file of the building block product. This method guides the generation of building block word sequences through image word sequences via autoregression. It uses multi-word joint representation to characterize the complete geometry and assembly attributes of individual building block components and obtains standardized engineering files based on unified word list parsing. This effectively overcomes the shortcomings of existing 3D building block model generation methods, such as missing part attribute representation, position quantization conflicts, and lack of assembly geometric constraints, and significantly improves the generation accuracy and structural assembly rationality of 3D building block models. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating a method for generating a three-dimensional block model according to an embodiment of this application; Figure 2 This is a schematic diagram of the structure of a three-dimensional block model generation device provided in one embodiment of this application; Figure 3 This is a schematic diagram of the structure of a terminal device provided in an embodiment of this application. Detailed Implementation

[0019] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0020] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0021] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0022] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0023] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0024] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0025] In recent years, customized building block products and building block IP derivatives have emerged rapidly. Users want to convert real building facades, portraits, and other scenes from photos into physical building block models that can be purchased offline and assembled according to instructions. This requires 3D building block model generation systems to not only visually reproduce the main structure, decorative details, and colors in the reference images, but also to physically ensure the assemblability of the parts (no clipping, no floating, and assembly by ordinary users according to the assembly instructions).

[0026] Currently, existing 3D block model generation systems mainly include two technical approaches: one is based on manual modeling software, which relies on manual dragging and dropping of blocks one by one, resulting in low efficiency and high requirements for user expertise; the other is an early voxelization-based algorithm scheme, which only considers regular cuboid blocks, directly voxelizes the target 3D shape and fills it into the regular blocks, resulting in a lack of detail and weak expressiveness.

[0027] However, the aforementioned prior art has the following technical problems: (1) Insufficient ability to express parts. Existing learning-based block generation methods abstract each block into a single token or only distinguish the size of the cuboid. They cannot simultaneously carry the four attributes of "part shape type, color, three-dimensional orientation, and three-dimensional position" under a unified representation. Therefore, they can only support micro models with less than 30 blocks, monochrome, and regular cuboids. They cannot generate medium and large block models that include irregular parts such as window frames, panels, bevels, and decorative parts and involve dozens of colors.

[0028] (2) Limited input dimensions. Existing autoregressive block generation methods use text as a condition, which cannot accurately depict the layout structure (such as the row and column positions of doors and windows, roof shape, and the outline of a person's hairstyle) and color distribution in the reference image. The generated results have a significant semantic gap with the user's expected construction target based on the image.

[0029] (3) Difficulty in quantifying position. The actual size of building blocks in building block products involves non-integer multiples of millimeters (including standard bump spacing, as well as sub-bump sizes such as half bumps and thin plates). If the position is directly predicted using floating-point numbers, it will introduce quantization errors and cause parts to be unable to be connected. If the greatest common size is used for rigid quantization, it will result in an excessively large vocabulary, making it difficult to train.

[0030] (4) Assembly legality is difficult to guarantee. Existing autoregressive methods only sample the next block according to probability, without forcibly constraining the geometric assemblability. In areas with dense parts, it is very easy to cause clipping and suspension, and the generated model cannot be reproduced offline.

[0031] (5) Paired training data is extremely scarce. There are almost no paired data of "high-quality building block project files and corresponding real reference photos" on the Internet. The scale of manually labeled data (in the thousands) is far from sufficient to support the training of end-to-end image-to-building block generation models.

[0032] (6) Insufficient attribute coupling in single positive prediction. The single-token one-step prediction strategy cannot consider other attributes that will be predicted for the same part in advance when predicting the next attribute, resulting in inconsistencies between the four attributes of the part and frequent problems such as incorrect color and shape matching and conflicting orientation and position.

[0033] In summary, within acceptable latency constraints in the cloud, this application provides a method for generating 3D block models for medium to large-sized building block models containing hundreds of irregularly shaped parts, dozens of colors, and hundreds of parts. This method automatically generates models by using user-uploaded natural images as conditions, generating results that closely approximate the reference image in both geometry and color, and are naturally non-clipping, non-floating, and reproducible offline. Furthermore, it also provides a data synthesis path that can be trained on thousands of paired data sets.

[0034] The 3D block model generation method provided in this application is implemented in a cloud-based "automatic block generation" microservice. It is triggered by a reference image uploaded from the front end (such as a webpage or mini-program). The server then executes the following five modules sequentially or in parallel, ultimately sending the generated block engineering file (containing information such as the type, color, orientation, position, and assembly order of each part) back to the front end for use in downstream processes such as assembly instruction generation, bill of materials generation, and packaging / distribution. In some examples, the server may include the following five modules: image encoding module, block word segmentation module, autoregressive generation module, collision rollback module, and engineering file restoration module. Specifically, after inputting a natural reference image I (such as a photo of a building facade or a photo of a person's head) stored in RGB three channels, the image encoding module encodes the natural reference image I into an image token sequence S_I of length 257; the block segmentation module maintains a fixed vocabulary V for each block category and provides the ability to bidirectionally convert any project file into an integer token sequence; then, the autoregressive generation module uses the image token sequence S_I as a prefix condition and performs autoregressive prediction token by token starting from the starting token separator BOS until it encounters the ending token separator EOS; During the autoregressive prediction process, the collision rollback module performs an assembly validity check after each single-block component is generated (i.e., 6 consecutive tokens can represent a single-block component). If a conflict is detected, the violating token candidate is masked and resampled. Finally, the project file restoration module queries the vocabulary V according to the predicted integer token sequence to obtain the geometric and assembly attribute information of the part, such as "type / color / orientation / position". It then restores the project file with grouping and assembly order in ascending order of Z-axis, Y-axis, and X-axis and outputs it.

[0035] Please see Figure 1 , Figure 1 This is a flowchart illustrating a method for generating a 3D block model according to an embodiment of this application. As an example and not a limitation, this 3D block model generation method can be run on or applied to a terminal device, and the method includes: S11. Obtain a reference image of the building block product.

[0036] S12. Encode the image features in the reference image to obtain an image word sequence, wherein the image word sequence includes multiple image words, which are used to represent the visual feature information of the reference image.

[0037] S13. Using the image lexical sequence as the conditional guide sequence, generate building block lexicals one by one through autoregression, and splice them to obtain the building block integer lexical sequence corresponding to the reference image. The building block integer lexical sequence includes multiple building block lexicals, which are used to characterize the geometric and assembly attribute information of a single building block component.

[0038] S14. Based on the preset block vocabulary, the block words in the block integer word sequence are parsed to generate the product engineering file of the block product. The preset block vocabulary is used to represent the mapping relationship between the geometric and assembly attribute information of a single block component and the integer code.

[0039] S15. Generate a three-dimensional block model corresponding to the block product based on the product engineering file of the block product.

[0040] It should be noted that the reference image refers to a two-dimensional image that contains the appearance and shape of the target building block product. This reference image serves as the visual input for generating the three-dimensional building block model, carrying information about the product's shape, color, and spatial layout. Image features can be understood as the visual information extracted from the reference image, which may include information such as overall outline, tone, local texture, and color block boundaries.

[0041] Image tokens (or image words) refer to the basic feature units output by a visual encoder, used to represent local or global visual feature information in a reference image. Multiple image tokens arranged in an ordered manner can form an image token sequence. This image token sequence is an ordered feature sequence obtained after encoding the reference image and can serve as conditional guidance information for the subsequent generation of 3D block models.

[0042] The conditional guidance sequence can be understood as a fixed sequence of pre-constraint information in the autoregressive generation process. In this embodiment, the conditional guidance sequence is an image lexical sequence, providing a visual basis for the generation of building block lexical units.

[0043] A building block token (or block token) refers to an integer encoding unit taken from a pre-defined building block vocabulary. A single building block token represents a single-dimensional attribute (such as part type, color, orientation, or coordinates) of a building block component. The preset number is a fixed value. In this embodiment, the preset number can be 6. That is, 6 consecutive building block tokens can completely describe all the geometric and assembly attribute information of a building block (i.e., a single building block component).

[0044] A single building block component is an independent, assembleable individual block, possessing definite geometric and assembly attributes such as part type, color, 3D orientation, and spatial coordinates. It can be represented by a continuously preset number of building block lexical units. The building block integer lexical sequence is an ordered sequence generated autoregressively, composed of several building block lexical units, representing the structured encoding form of the building block component. The preset building block vocabulary is a pre-constructed mapping dictionary that establishes a bidirectional mapping relationship between various geometric and assembly attributes of the building blocks and their unique integer codes, supporting code serialization and reverse parameter parsing.

[0045] Product engineering files are structured, standardized files that can be read by 3D modeling software and modular simulation software. These files can store metadata such as the type, color, orientation, 3D coordinates, arrangement order, and assembly hierarchy of all modular components. This metadata can be directly loaded, rendered, and simulated by 3D modeling engines. A 3D modular model refers to a 3D digital model formed by assembling all individual modular components according to their spatial positions.

[0046] Specifically, firstly, a two-dimensional reference image corresponding to the block product to be generated is acquired. This two-dimensional reference image can be obtained through methods such as camera capture, image import, and drawing export. Optionally, standardization preprocessing can be performed on the two-dimensional reference image, such as size scaling, center cropping, and pixel normalization, to unify the image input specifications and meet the input requirements of the subsequent encoder. This reference image can provide appearance constraints for the target block product and serves as the visual source for cross-modal generation.

[0047] Subsequently, a visual encoder (such as the frozen unlabeled knowledge self-distillation DINOv2 network) is used to extract visual features from the reference image. The image is decomposed into global features and multiple local block features, which are then encoded into corresponding image terms (such as global image terms and individual local image terms). The global image terms carry the overall structure and overall tone summary of the building block product, while each local image term corresponds to the texture and contour details of different spatial regions of the image. All image terms are arranged in a preset order to form a complete image term sequence, which serves as the conditional input for the subsequent generation model. It should be understood that this step, by converting the pixel information of the two-dimensional image into a serialized feature representation that the model can recognize, while retaining global shape information and local detail information, can support the model to accurately match the building block components to the corresponding spatial regions of the image. This achieves the conversion from image modality to sequence feature modality, adapting to the input paradigm of Transformer-type sequence generation networks.

[0048] After obtaining the image lexical sequence, it is input into the causal decoding network as a generation constraint (i.e., a condition-guided sequence). An autoregressive model is used to sample and output building block lexicals word by word. These lexicals are taken from a pre-defined building block vocabulary and are used to represent a single geometric or assembly attribute of the building block. Each consecutive pre-defined number of building block lexicals are combined to completely represent a single building block component, corresponding to geometric and assembly attributes such as part type, color, 3D orientation, and three-axis coordinates. Then, the generation process continues iteratively until a termination lexical is output, ultimately forming a complete building block integer lexical sequence. It should be understood that this step uses the image lexical sequence as a continuous generation condition. By employing an autoregressive generation mechanism, a structured sequence of building blocks is constructed step by step, ensuring that the overall shape and spatial layout of the generated building blocks remain consistent with the reference image. By using multiple image lexicals to jointly represent a single building block component, multi-dimensional parameters such as part type, orientation, and position can be uniformly represented, ensuring complete information expression. The output standardized integer lexical sequence facilitates subsequent table lookup and parsing, enabling an end-to-end automated process.

[0049] Then, the sequence of integer lexical units in the building blocks is traversed, grouped according to a preset number of consecutive building block lexical units, and a preset building block lexical table is queried (this preset building block lexical table establishes a mapping relationship between building block attributes and integer codes). By looking up the table, the grouped building block lexical units (i.e., each group of integer codes) are decoded in reverse to restore the metadata (such as part model, color, rotation posture, and 3D coordinates) corresponding to each single building block component. All building block components are sorted according to preset spatial sorting rules, all component metadata is integrated, and a standardized product engineering file containing all assembly information is generated. It should be understood that this step completes the conversion of integer codes into building block parameters that can be used in engineering, and opens up the data link from the AI-generated sequence to 3D modeling. Bidirectional encoding and decoding are achieved based on a unified lexical table, resulting in strong compatibility. Moreover, the engineering file saves structured assembly information and can still be used independently after being separated from the generated model.

[0050] Finally, the product engineering file is read by the 3D modeling engine, and the parameters of each individual building block are loaded sequentially. Based on the metadata of each individual building block, the corresponding 3D basic model is retrieved, and the blocks are placed in space according to the parsed coordinates, orientation, and other parameters, thus instantiating each individual building block. All building blocks are then assembled and rendered to generate a complete 3D digital model. This 3D building block model also supports subsequent operations such as viewing, perspective switching, exporting, and assembly simulation. It should be understood that this step can directly output a usable 3D solid model of the building blocks without the need for manual repositioning. The component parameters are complete, and the model possesses physical assembly meaning. It can not only be previewed visually but also used for building block simulation, rendering of effect diagrams, and generation of assembly drawings.

[0051] It is understood that this application provides a method for generating a 3D building block model. The method includes: First, acquiring a reference image of the building block product. Then, encoding the image features in the reference image to obtain an image word sequence, wherein the image word sequence includes multiple image words used to represent the visual feature information of the reference image. Next, using the image word sequence as a conditional guide sequence, generating building block words one by one through an autoregressive method, and concatenating them to obtain a building block integer word sequence corresponding to the reference image, wherein the building block integer word sequence includes multiple building block words used to represent the geometric and assembly attribute information of a single building block component. Then, based on a preset building block vocabulary, parsing the building block words in the building block integer word sequence to generate a product engineering file for the building block product, wherein the preset building block vocabulary is used to represent the mapping relationship between the geometric and assembly attribute information of a single building block component and the integer encoding. Finally, generating a 3D building block model corresponding to the building block product based on the product engineering file of the building block product. This method guides the generation of building block word sequences through image word sequences via autoregression. It uses multi-word joint representation to characterize the complete geometry and assembly attributes of individual building block components and obtains standardized engineering files based on unified word list parsing. This effectively overcomes the shortcomings of existing 3D building block model generation methods, such as missing part attribute representation, position quantization conflicts, and lack of assembly geometric constraints, and significantly improves the generation accuracy and structural assembly rationality of 3D building block models.

[0052] In one possible implementation, image features in the reference image are encoded to obtain an image word sequence, including: The global image features in the reference image are encoded into global image terms, where the global image features are used to characterize the overall outline, global tone, and global structural summary information of the building block product.

[0053] Multiple block image features in the reference image are encoded into multiple local image terms, where the block image features are used to characterize the local texture, local color, and local contour details of the building block product.

[0054] By placing the global image term at the beginning of the sequence and then placing multiple local image terms after the global image term, an image term sequence is obtained.

[0055] It should be noted that global image features refer to the overall feature information extracted from the entire reference image, not limited to local areas of the image, summarizing the overall shape, color distribution, and overall layout of the building block product (i.e., the overall outline, global tone, and global structural summary information of the building block product). Global image terms refer to the feature terms obtained by mapping global image features through an image encoding module. For example, this global image term can correspond to the [CLS] term in an image encoding module (such as a visual encoder), serving as a summary carrier of the overall image information.

[0056] Patch-based image features refer to the local visual features (such as local texture, local color, or local contour details) extracted from each patch of a reference image, which is uniformly divided into several patch blocks. Local image terms refer to the image terms obtained by mapping the patch features of each image block through an image encoding module. Each local image term corresponds to a spatial region in the reference image.

[0057] Image lexical sequences can be understood as ordered feature sequences formed by global image lexical sequences and multiple local image lexical sequences arranged in a fixed order, which serve as conditional prefix inputs for subsequent causal decoding networks.

[0058] In some examples, specifically, after the reference image undergoes normalization preprocessing (such as scaling, center cropping, and pixel normalization), it is fed into an image encoding module (such as a visual encoder). The image encoding module performs global feature aggregation on the entire reference image, extracts global image features, and maps them to generate independent global image terms. These global image terms can condense the overall outline shape, main color tone, and overall structural layout summary of the building block product, recording global design constraints and preventing significant deviations between the generated overall building block structure and the reference image. It should be understood that global image terms can carry overall design constraints, guiding the model to control the overall shape and proportions of the building block product during subsequent autoregressive generation, preventing structural imbalances and design deviations caused by relying solely on local features, and ensuring that the macroscopic shape of the building block model remains consistent with the reference image.

[0059] Simultaneously, the image encoding module equally divides the reference image into non-overlapping image patches, extracts the patch features corresponding to each region, and independently encodes each patch feature to generate corresponding local image terms. Each local image term is bound to a fixed spatial position of the image patch, recording information such as contour lines, color blocks, and local texture details within the corresponding region. In this embodiment, the reference image size can be 224×224, and the patch size can be 16×16, resulting in 256 16×16 image patches, thus generating 256 corresponding local image terms. It should be understood that local image terms can provide refined local visual information, supporting the model in aligning individual building block components to their corresponding spatial positions in the image; each local image term is naturally bound to the two-dimensional spatial position information of the reference image, facilitating the decoder to achieve region matching through self-attention and improving the accuracy of local part reconstruction.

[0060] 3) After obtaining the global image terms and multiple local image terms, an ordered sequence can be constructed according to a fixed arrangement rule. The global image term is placed at the first position of the sequence, and all local image terms are arranged sequentially after the global image term according to a preset grid order, assembling to form a complete image term sequence. This image term sequence can then be directly used as a fixed prefix to be concatenated into the decoder input, participating in the self-attention calculation of each step of autoregressive decoding. It should be understood that the global image term is located at the beginning of the sequence, facilitating the network's priority reading of overall structural information while considering both global constraints and local details. Furthermore, this encoding method simultaneously preserves both global image shape constraints and local spatial details; the ordered term sequence input into the subsequent causal decoding network as a fixed prefix allows the decoder to simultaneously access global visual information and information from various local image regions through self-attention during each step of autoregressive prediction, achieving precise alignment between the generated positions of the building blocks and the spatial regions of the reference image.

[0061] In one possible implementation, the preset block vocabulary is constructed by the union of multiple sub-vocabulary sets, which include a part type vocabulary set, a color vocabulary set, a three-dimensional orientation vocabulary set, a coordinate position vocabulary set, and a special control character vocabulary set. The special control character vocabulary set includes a sequence start character, a sequence end character, and a fill character.

[0062] The pre-built block vocabulary is a discrete encoding dictionary that establishes a one-to-one mapping between integer index codes and various attributes and sequence control symbols of the blocks. This is used to realize the serialization encoding and reverse parsing decoding of block component information. The sub-vocabulary set union construction can be understood as the total vocabulary being composed of the union of multiple independent word sets. The internal encoding intervals of different word sets are isolated from each other, and there are no encoding conflicts.

[0063] Multiple sub-vocabulary sets can include part type vocabulary sets, color vocabulary sets, 3D orientation vocabulary sets, coordinate position vocabulary sets, and special control character vocabulary sets. Among them, the part type vocabulary set (e.g., V_type) can include all available standard block part models, with each part type assigned an independent integer code to represent the part type of the block component. The color vocabulary set (e.g., V_color) can include the standard solid colors and semi-transparent colors supported by the blocks, with each color corresponding to an independent code to represent the block's appearance color. The 3D orientation vocabulary set (e.g., V_orient) can include the codes for the legal 3D mounting postures of the blocks, representing the rotational orientation of the blocks in 3D space. The coordinate position vocabulary set (e.g., V_pos) can discretize the set of spatial displacement indices; after index conversion, the three-axis coordinates of the block relative to the model's bounding box reference point are obtained, representing the block's spatial position.

[0064] The special control character set does not correspond to actual block attributes; it is used for sequence boundary marking and training batch length padding. For example, the special control character set may include sequence start characters, sequence end characters, and padding characters. The sequence start character (BOS) is the start marker generated by decoding, the sequence end character (EOS) indicates the end of block sequence generation, and the padding character (PAD) is used to pad unequal-length sequences during batch training.

[0065] In some examples, a predefined block glossary is used. It can be done according to ( The process involves several steps. First, a senior building block designer selects available building block parts from all standard building block parts based on the target application category (e.g., facade / figure), constructing a part type terminology V. For example, the facade category contains 103 types of irregularly shaped parts. A color vocabulary set (V) is created by collecting commonly used building block colors under this target application category. Color, for example, the facade category is 37 colors, including several semi-transparent colors used for windows. Select all feasible 3D axis-aligned installation postures for the project to form a 3D orientation vocabulary set V. orient, for example, the three-dimensional orientation word set V The `orient` keyword retains only 24 3D axis alignment orientations, covering all common installation postures of "convex points up, down, left, right, front, and back". Based on the spatial discrete granularity (e.g., 0.8mm step size) and the maximum model size, the coordinate value range is determined, constructing a coordinate position vocabulary V{pos}. For example, multiplying an integer index by a 0.8mm step size yields the displacement of the smallest corner point of the single block component relative to the overall bounding box of the model. 0.8mm is taken from the greatest common factor of the dimensions of all candidate block components and their connections. The displacement range for the facade category is 0 to 439 (i.e., the maximum overall size is 35.2cm = 440mm × 0.8mm), therefore V_pos is 440. Finally, sequence start symbols, sequence end symbols, and fill symbols are introduced to form a special control symbol vocabulary. Therefore, the preset total vocabulary size for the facade category can be 103 + 37 + 24 + 440 + 3 = 607 words. Each sub-vocabulary is assigned a non-overlapping coding interval, which is then merged to form the complete preset total vocabulary for the blocks. When serializing the project file, the encoded words corresponding to the type, color, orientation, and X / Y / Z position of each individual building block are selected in sequence. After the model inferences and outputs the integer word sequence, the various attributes of the building block are restored by querying the preset building block vocabulary.

[0066] It should be understood that in this embodiment, when adding new parts, colors, and size ranges, only the corresponding sub-word sets need to be expanded, without reconstructing the entire vocabulary, to adapt to different categories (facade, portrait, and other block modeling tasks). Simultaneously, by splitting the type, color, posture, and position of the blocks into independent word sets, the model can more easily learn the constraints between different attributes, which is beneficial for the accompanying MTP multi-token joint prediction mechanism to suppress internal attribute conflicts within the blocks and reduce the learning difficulty of the model. The encoding intervals of each sub-word set are isolated, with clear encoding boundaries, allowing for rapid determination of the attribute category of the tokens during the parsing stage, reducing the logical complexity of sequence grouping parsing and facilitating encoding and decoding implementation. Furthermore, the special control character word set unifies the start and end identifiers of the sequence, solving the problem of inconsistent sample lengths in batch training and ensuring compatibility with the data input specifications of autoregressive networks.

[0067] In one possible implementation, the block-type integer lexical sequence also includes a start special lexical and a stop special lexical.

[0068] Using the image word sequence as a conditional guided sequence, building block words are generated one by one through an autoregressive approach, and then concatenated to obtain a building block integer word sequence corresponding to the reference image, including: The image word sequence is used as a fixed prefix and concatenated with the starting special word to obtain the initial context sequence.

[0069] The initial context sequence is input into a pre-trained causal decoding network, and a causal mask is used to constrain the decoding field of view, outputting the hidden features corresponding to the current decoding step. The self-attention mechanism in the decoding step is used to associate multiple image words in the image word sequence, so that the information carried by the image words is aligned with the spatial regions of the reference image.

[0070] The hidden features are mapped to a pre-defined block vocabulary to predict the probability distribution of the block words corresponding to the current decoding step.

[0071] The current building block words sampled from the probability distribution based on a preset sampling strategy are added to the end of the initial context sequence to obtain the updated context sequence.

[0072] Repeat the word-by-word prediction process to expand the context sequence until the terminating special word is sampled, and then stop to obtain the target context sequence.

[0073] The image word sequence in the target context sequence is removed to obtain the block-type integer word sequence.

[0074] It should be noted that the Start Special Word (BOS) is a sequence start marker, serving as the initial trigger symbol for the autoregressive decoding process. It indicates the start of block word generation and belongs to the special control symbol set. The End Special Word (EOS) is a sequence end marker. When the network samples and outputs this End Special Word, it can be determined that the block component word generation is complete, and the inference process terminates.

[0075] The block integer lexicon sequence is the final output, which is an ordered integer encoding sequence consisting only of block lexicons (representing block attributes), start special lexicons, and end special lexicons, after removing the leading image lexicon prefix. It can be used to query the preset block vocabulary to parse the block assembly parameters.

[0076] In this embodiment, the conditional guidance sequence is an image lexical sequence, which serves as a fixed prefix for the decoder input throughout, continuously providing two-dimensional visual constraints for the generation of building block lexical units. The initial context sequence can be understood as the initial decoding input sequence composed of the image lexical sequence (fixed prefix) and the starting special lexical unit during the inference initiation phase.

[0077] The pre-trained causal decoding network employs a causal Transformer decoder architecture and is trained on a block-generated dataset, supporting conditional autoregressive sequence generation. Causal masking, an attention masking mechanism, constrains each position during decoding to only access the sequence content preceding the current position, preventing access to future word positions, thus eliminating information leakage and ensuring the compliance of the autoregressive generation logic.

[0078] The decoding step refers to a single iteration of the autoregressive generation process, where each decoding step only predicts and samples a single building block word. Hidden features refer to the hidden layer embedding vectors output by the Transformer decoder in the pre-trained causal decoding network, carrying feature information fused with the current context.

[0079] The self-attention mechanism can be understood as a feature interaction module inside a pre-trained causal decoding network. In the causal Transformer decoder architecture provided in this embodiment, the image word sequence is directly concatenated to the prefix of the input sequence. Self-attention can establish the association between the building block word to be predicted and all image words and historical building block words.

[0080] The preset sampling strategy is a pre-defined sampling strategy, which can include greedy sampling, Top-K sampling, etc., and can select the final output building block words based on the word probability distribution. The updated context sequence can be understood as the new context sequence formed by appending to the end of the sequence after sampling new building block words in each round, which serves as the input for the next decoding step. The target context sequence refers to the complete long sequence formed after sampling the terminating special word and stopping generation. The structure of the target context sequence can be: image word sequence + starting special word + sequentially generated building block words + terminating special word.

[0081] In this embodiment, the image word sequence is used as a permanent prefix and concatenated into the decoder input sequence. Self-attention is used to achieve image visual information reuse. Building block words are generated one token at a time. After generation, the image prefix is ​​stripped to obtain a pure building block integer word sequence.

[0082] Specifically, firstly, the complete image word sequence obtained by encoding the reference image is placed at the beginning of the sequence as a fixed prefix. Then, a special initial word is concatenated after the image word sequence to form the initial context sequence. This initial context sequence is used as the input to the first round of the decoding network. Throughout all subsequent decoding iterations, the image word sequence is always retained as a prefix and is never removed. Image visual features participate in every step of the decoding calculation, continuously constraining the generated content to fit the shape of the reference image.

[0083] After obtaining the initial context sequence, it is fed into a pre-trained causal decoding network. This network uses causal masking to ensure that any decoding position can only access all words preceding that position. Through a self-attention mechanism, the network enables the currently predicted building block word to be associated with all image words in its prefix, reading both global and local image words. The multi-layer Transformer operations in this network fuse contextual information, outputting the hidden features corresponding to the current decoding step. Relying on the attention association between image words and building block words, the generated building block components match the corresponding two-dimensional spatial positions of the reference image. It should be understood that the causal masking strictly follows the autoregressive generation paradigm to prevent future information leakage and match the constraints during model training. No additional cross-attention module is needed; cross-modal constraints can be achieved using native self-attention, simplifying the network structure. By aligning the building block components with the spatial regions of the reference image, the overall layout misalignment and part position offset problems of the building block model are effectively reduced.

[0084] Then, the fixed-dimensional hidden features are mapped to the preset total vocabulary dimension through the output projection layer. The generation probability of each candidate word in the preset total vocabulary is calculated, and the probability distribution corresponding to all word elements at the current position is output. A preset sampling strategy (such as greedy sampling or Top-K sampling) is used to select a single word element from the probability distribution. The sampled word element is appended to the end of the current initial context sequence to form an updated context sequence, which is then used as the input for the next decoding step.

[0085] The process iteratively executes feature extraction, probability prediction, lexical sampling, and sequence update, continuously adding building block lexical units to the end of the context sequence. Iterative reasoning stops immediately once the sampling output reaches a terminating special lexical unit, resulting in the complete target context sequence. The target context sequence begins with a fixed-prefix image lexical sequence, followed by a starting special lexical unit, building block lexical units, and a terminating special lexical unit. After obtaining the target context sequence, all the preceding image lexical units are stripped away, leaving the remaining ordered sequence, which is the building block integer lexical sequence, directly supplied to subsequent parsing processes.

[0086] It should be understood that in this embodiment, the image word sequence is retained as the input prefix of the decoder throughout. Each decoding step can make full use of the global and local visual information of the reference image. It relies on self-attention to complete the cross-modal association between visual features and building block sequences. No additional cross-attention module is needed. The building block encoding sequence is stably constructed through word-by-word serial generation. At the same time, the conditional sequence and the target generation sequence are distinguished, which facilitates the subsequent parsing and restoration of building block assembly parameters.

[0087] In some examples, the pre-trained causal decoding network in this embodiment may consist of a backbone network and six layers of dedicated prediction heads stacked at the output of the backbone network.

[0088] The backbone network (F_backbone) can be a 24-layer causal Transformer decoder in the OPT-1.3B architecture style, with a hidden dimension of 2048. This backbone network does not load pre-trained weights from the natural language domain, adopting a zero-initialization training method, adapting to a modular integer vocabulary, and avoiding representational interference from natural language semantics. The network internally configures causal masks to strictly ensure autoregressive generation constraints. During decoding, it can only access the current and historical context sequences, and cannot observe future word positions. Simultaneously, a self-attention mechanism establishes a correlation between the context sequence and the image word sequence as a fixed prefix, achieving alignment between image word features and corresponding spatial regions of the reference image.

[0089] Each of the six dedicated prediction heads (F_0, F_1, ..., F_5) is a single-layer Transformer DecoderBlock. All six prediction heads share the same output head, which maps the hidden feature states to a predefined total vocabulary dimension. Specifically, the i-th prediction head (F_i) predicts the probability distribution of the block term at position t+i based on the hidden features at decoding position t.

[0090] In some examples, the pre-trained causal decoding network in this embodiment fully utilizes all six layers of dedicated prediction heads during the network training phase, and uses multi-token joint prediction loss to complete network parameter optimization. That is, for each real decoding position t in the training sequence, the network predicts the probability distribution of block words at six positions: t, t+1, t+2, t+3, t+4, and t+5 in parallel. Cross-entropy loss is calculated for all 6×T (T is the effective length of the training sequence) prediction results, and the average value of all losses (average training loss) is used as the overall training loss for backpropagation to update the network parameters.

[0091] Among them, the average training loss for: ; t represents the decoding position, i=0 corresponds to predicting bit t itself, i=1 predicts bit t+1, i=5 predicts bit t+5. Represents a block lexical sequence, Represents an image word sequence. Represents the probability distribution of lexical units. This represents all historical building block words that have been generated or are known before the decoding position t.

[0092] The purpose of setting up parallel prediction of 6 subsequent terms is that a single building block component is represented by exactly 6 consecutive building block terms, corresponding to part type, color, orientation, X-axis position, Y-axis position, and Z-axis position, respectively. When predicting the first attribute term of a single building block component, this pre-trained causal decoding network can simultaneously and comprehensively plan the remaining five attributes of the same single building block component, realizing joint collaborative optimization of multiple attributes of building block parts, and effectively suppressing defects such as mismatch of internal attributes and contradictory position parameters of a single building block component.

[0093] After pre-training, the pre-trained causal decoding network enters the inference phase and executes a building block word autoregressive generation process. During inference, only the dedicated prediction head F_0 is used; dedicated prediction heads F_1-F_5 are disabled and do not participate in the computation. First, the image word sequence is concatenated with the starting special word BOS as a fixed prefix to construct the initial context sequence input to the causal decoding network. The backbone network applies a causal mask to constrain the decoding field of view, relies on self-attention to associate the image word sequence, and outputs the hidden features of the current decoding step. Then, the hidden features are fed into the dedicated prediction head F_0 and the shared output head H to output the probability distribution of the building block word at the current position t. The current building block word is selected through a greedy sampling or Top-k sampling strategy and appended to the end of the context sequence to form an updated context sequence. This process iterates word-by-word generation until the terminating special word EOS is sampled to obtain the target context sequence. Then, the image word sequence at the front of the target context sequence is removed, finally obtaining the building block integer word sequence.

[0094] During the inference phase, only a dedicated prediction head F_0 is retained for serial sampling per token. This retains the flexibility of autoregressive sequence generation, supports mid-process sampling intervention and conflict detection rollback, and avoids the computational overhead of multiple prediction heads. During the training phase, the model is endowed with global planning capabilities by relying on MTP multi-term joint prediction, enabling training to have the ability to coordinate planning of the entire component, while inference maintains the autoregressive serial generation architecture.

[0095] It should be noted that the training samples for this pre-trained causal decoding network are paired datasets. Each training sample may contain: a reference image and a fully annotated standard block integer lexical truth sequence. The input samples are two-dimensional reference images corresponding to various block shapes, encoded by a DINOv2 encoder to obtain an image lexical sequence, which serves as the conditional prefix for each sample. The label-side truth sequence consists of a start special word (BOS), consecutive block lexical groups, and a stop special word (EOS). Every six consecutive block lexical groups represent the complete attribute encoding of a single block component, including part type, color, pose, and 3D spatial coordinate information.

[0096] The dataset can be derived from manually built block model rendering images and exported project files with automatically converted lexical units, and real block assembly data with actual block images. It does not require a large amount of manual image annotation, but only relies on the automatic serialization of block project files to generate truth sequences.

[0097] In one possible implementation, the method includes adding the current building block lexical sampled from the probability distribution based on a preset sampling strategy to the end of the initial context sequence to obtain an updated context sequence. After generating a preset number of building block words, the preset number of building block words are parsed into the three-dimensional network entity corresponding to the current single building block component according to the preset total building block vocabulary.

[0098] Based on the 3D network entities corresponding to the historical single-block components that have passed verification, the 3D network entities of the current single-block component are subjected to assembly conflict verification. The assembly conflict verification includes at least geometric clipping verification and floating assembly verification.

[0099] If an assembly conflict is detected, the non-compliant block terms within the current single block component are masked and resampled to obtain the corrected block terms.

[0100] The preset quantity is the number of lexical groups corresponding to a single building block component. In this embodiment, the preset quantity is preferably 6, that is, 6 consecutive building block lexical groups jointly represent a complete single building block component, corresponding to the part type, color, 3D orientation, X-axis position, Y-axis position, and Z-axis position attributes of the single building block component. The current single building block component refers to the new building block unit to be verified obtained after the latest cumulative generation of the preset quantity of building block lexical groups during the autoregressive generation process, which is the target building block component to be assembled.

[0101] A 3D network entity can be understood as a 3D geometric entity model of building blocks, obtained by parsing a lookup table of building block morphemes. It possesses real 3D dimensions, spatial coordinates, and rotational orientation, and has digital entity attributes that can be collided and spatially verified. Historical single building block components refer to all building block components that have been generated, verified, and assembled in compliance with regulations and without conflicts before the current building block morpheme generation step. They are the confirmed and legal assembly bases.

[0102] Assembly conflict verification refers to a mechanism for verifying the spatial rationality of newly added building block components and existing assembly bases. It is used to screen for erroneous structures that do not conform to the physical assembly rules of building blocks. Assembly conflict verification can include geometric clipping verification and suspended assembly verification. Geometric clipping verification, also known as spatial geometric conflict verification, is used to detect whether the three-dimensional entity of the current building block component and the entity of historically valid building block components have spatial overlap, volume interpenetration, surface overlap, or interference issues. Suspended assembly verification, also known as assembly support rationality verification, is used to detect whether the bottom of the current building block component lacks legal support and is suspended in mid-air, thus conforming to the physical assembly rules of building blocks from bottom to top.

[0103] Violation block words refer to the corresponding attribute block words that cause assembly conflicts or unreasonable spaces in the current block components. They are mostly abnormal codes such as position words and orientation words.

[0104] Masking can be understood as a model inference correction mechanism. It masks a set of building block words corresponding to a single building block component that causes a conflict, eliminating the influence of erroneous historical sequences and prohibiting violating words from participating in subsequent decoding inference. Resampling involves re-predicting and sampling word probability distributions based on the image conditional context and legal historical sequences at the positions of the violating building blocks after masking, generating compliant corrected building block words.

[0105] Specifically, during the continuous process of autoregressive word-by-word sampling and generation, and iterative updating of the context sequence, the cumulative number of generated building block words is counted in real time. Whenever the cumulative number of generated building block words reaches a preset number (e.g., 6), the parsing logic of a single building block component is immediately triggered. Based on the preset total building block vocabulary, the preset number of consecutive building block words are decoded word by word to restore the complete metadata of the current building block component, such as part type, color, three-dimensional orientation, and three-axis spatial coordinates. Based on the metadata, a three-dimensional network entity with real three-dimensional spatial attributes for the current single building block component is instantiated and generated.

[0106] Then, retrieve the 3D network entities corresponding to all historical single-block components that have completed verification and been judged to be compliant with assembly standards, and construct a stable and legal block assembly base. Using this block assembly base as a reference, conduct comprehensive assembly conflict verification on the 3D network entities of the newly generated single-block components. Assembly conflict verification can include at least two core dimensions: first, geometric clipping verification, which detects whether the spatial bounding box and volume of the current block entity overlap, intersect, or interfere with the geometric conflicts of historical block entities; second, floating assembly verification, which detects whether there are effective supporting blocks at the bottom of the current block entity, judges whether there are unsupported, suspended, or floating assembly violations, and strictly matches the assembly physical rules of the solid blocks.

[0107] After completing the assembly conflict verification, the result is judged. If the current single block component has no assembly conflict, it is determined to be a legal component and directly included in the historical legal assembly base to participate in the subsequent block word generation constraints. If geometric clipping or floating assembly violations are detected, all violating block words corresponding to the current single block component that caused the conflict are accurately located. Masking is performed on this group of violating block words to block the feature influence of erroneous words on subsequent decoding inference. Subsequently, based on the image conditional guidance sequence and the legal and conflict-free historical block word sequence, the word probability distribution prediction and sampling operation is re-executed to complete the resampling of the violation position, generate corrected block words that conform to the assembly rules, and replace the original violating words with the corrected block words to ensure the spatial rationality of the generated block integer word sequence.

[0108] It should be understood that in this embodiment, a component-by-component verification and correction mechanism is embedded midway through the autoregressive generation of each word and byte. This differs from the traditional approach of uniformly correcting errors after all sequences are generated. Instead, it allows for immediate screening and correction of conflicts after each individual block component is generated, preventing the continuous accumulation of numerous violating block words and eliminating structural defects such as clipping and floating from the generation source. This enables real-time dynamic error correction during the generation process. Simultaneously, targeted dual core verification rules for geometric clipping and floating assembly are set, perfectly aligning with the assembly logic of real blocks—"cannot be interlocked, must be supported"—solving the industry pain point of pure AI vision generation that only fits the appearance and does not adhere to physical assembly rules. This ensures that the generated 3D block model not only matches the reference image in appearance but also possesses realistic, assembleable engineering attributes. Furthermore, when assembly conflicts occur, only the local words corresponding to the violating single block component are masked and resampled, without clearing or restarting the entire sequence generation process. This ensures error correction accuracy while preserving the maximum amount of legitimate historical generated content, effectively reducing the computational load of model inference and balancing generation quality and inference efficiency.

[0109] This embodiment relies on legitimate historical building block components as the verification benchmark to realize a closed-loop iterative mechanism of "generation-verification-correction-regeneration". This ensures that the generation of subsequent building block terms is always based on a compliant assembly base, gradually accumulating legitimate assembly structures, continuously constraining the subsequent generated content, and greatly improving the structural stability and rationality of the overall building block model.

[0110] In one possible implementation, based on a pre-defined block vocabulary, the block lexical units in the block integer lexical sequence are parsed to generate a product engineering file for the block product, including: Based on the pre-defined block vocabulary, the block words in the block integer word sequence are parsed to obtain the metadata information corresponding to each single block component.

[0111] Based on the metadata information of each individual building block, all individual building blocks are sorted according to the preset spatial dimension priority to generate the product engineering file of the building block product.

[0112] It should be noted that a single building block component is an independent assembly unit, jointly represented by a continuously preset number of building block terms. Metadata information refers to the complete structured parameters corresponding to a single building block component, which may include parameter information that can be used for 3D modeling and assembly, such as part type, color, 3D orientation, and X / Y / Z spatial position. Preset spatial dimension priority refers to the pre-defined building block sorting rules, which control the building block assembly order based on the priority of spatial coordinate axes.

[0113] Specifically, firstly, the complete sequence of integer building block terms is traversed and grouped according to a fixed preset number of terms. Each group of consecutive building block terms corresponds to an independent single building block component. For each group of terms, a preset building block vocabulary is queried, and the integer part type terms, color terms, 3D orientation terms, and 3D position terms are decoded one by one in reverse order to restore the complete metadata information of each single building block component, such as part model, color parameters, 3D rotation posture, and 3D spatial coordinates. This completes the structured conversion from the digital term sequence to the actual physical parameters of the building blocks.

[0114] Then, after acquiring the metadata information of all individual building block components, the unordered output order generated by the model's autoregression is discarded. All building block components are globally reordered according to a preset spatial dimension priority rule, resulting in a building block assembly sequence that conforms to the actual building logic. All information regarding the part parameters, colors, postures, spatial coordinates, and assembly order of all sorted building block components is integrated and packaged into standardized building block product engineering files, providing complete and standardized data support for subsequent 3D model reconstruction, assembly simulation, and model rendering.

[0115] It should be understood that in this embodiment, reverse parsing of terms is completed through a preset block vocabulary, accurately converting the discrete integer sequences generated by autoregression into block assembly metadata with physical meaning. This solves the problem that the sequence generation results cannot be directly modeled, establishes a data link between autoregressive generation and engineering applications, and achieves accurate restoration of the encoded sequence to engineering parameters. Since the generation order of each term in the model does not have spatial assembly logic, spatial dimension priority reordering is used to unify and standardize the block arrangement order, avoiding modeling errors and layering chaos caused by disordered generation, and significantly improving the overall aesthetics and structural rationality of the 3D block model. Based on spatial dimension priority sorting, matching the physical assembly habit of building blocks from bottom to top and layered construction, the generated engineering files can not only be visualized and rendered, but also directly used for block assembly simulation, disassembly and assembly demonstration, and assembly drawing export.

[0116] In one possible implementation, based on the metadata information of each individual building block, all individual building blocks are sorted according to a preset spatial dimension priority to generate a product engineering file for the building block product, including: All single building block components are sorted in ascending order with Z-axis as the first priority, Y-axis as the second priority, and X-axis as the third priority to generate an assembly sequence.

[0117] All individual building blocks are assembled and grouped according to the Z-axis layering, resulting in a group number for each individual building block.

[0118] Assign an assembly sequence number to each individual building block according to the assembly sequence.

[0119] By integrating the metadata, grouping numbers, and assembly sequence numbers of all individual building blocks, the product engineering documents are obtained.

[0120] In this embodiment, the preset spatial dimension priority can be: first priority Z-axis > second priority Y-axis > third priority X-axis. The coordinate axis values ​​are arranged from smallest to largest, with a smaller Z-value representing a lower spatial position, consistent with the logic of building blocks being assembled from the bottom up. The assembly sequence refers to the ordered list of building block components formed after sorting by the three-axis priority. Z-axis layering refers to dividing the horizontal assembly layers according to the Z-coordinate range of a single building block component, with building blocks within the same height range grouped into the same assembly layer. The group number refers to the number used to identify the assembly layer to which the building block belongs; components in the same layer have the same group number. The assembly sequence number refers to the sequence number assigned according to the assembly sequence, representing the recommended assembly order of the building blocks.

[0121] Specifically, the system reads the 3D coordinates from the metadata of all individual building blocks and prioritizes arranging them in ascending order of their Z-axis coordinates. When Z-axis coordinates are equal, they are arranged in ascending order of their Y-axis coordinates; when both Z-axis and Y-axis coordinates are the same, they are arranged in ascending order of their X-axis coordinates, ultimately forming a complete assembly sequence. The sorting logic implements a bottom-up, front-back, and left-to-right arrangement order, matching the building habits of real building blocks.

[0122] Then, based on the Z-axis coordinates of each building block, the system is divided into intervals. Building blocks with Z-axis coordinates within the same height interval are grouped into the same assembly layer. Each assembly layer is assigned a unique group number, and the group number is associated with the corresponding individual building block. The same group number indicates that the individual building block is in the same horizontal assembly layer, which facilitates layered display and layer-by-layer assembly simulation.

[0123] Iterate through the generated assembly sequences and assign a unique assembly sequence number to each individual building block according to the sequence order. The assembly sequence number represents the recommended assembly order; the smaller the assembly sequence number, the higher the priority for assembly.

[0124] Finally, the original geometric assembly metadata, the grouping number of the layer to which each individual building block component belongs, and the assembly sequence number are bound and summarized, and uniformly packaged into structured data to generate the product engineering file of the building block product.

[0125] It should be understood that in this embodiment, the Z-axis coordinates of individual building blocks are prioritized in ascending order to ensure that the bottom-level building blocks are placed first, conforming to the bottom-up building block assembly rule and avoiding simulation logic errors caused by assembling the upper-level building blocks first. By using Z-axis coordinates to mark the layer to which individual building blocks belong, it is possible to display a single layer of building blocks and load the assembly effect layer by layer, facilitating the generation of assembly instructions and disassembly demonstration videos. Furthermore, by including geometric parameters, layer information, and assembly order in the product engineering file, compared to ordinary model files that only store coordinates, it can not only render static 3D models but also directly drive assembly simulation programs to simulate the entire building block assembly process, supporting various downstream services such as modeling, simulation, drawing generation, and assembly guidance.

[0126] It should be noted that this application also provides a training data synthesis pipeline (applicable to facade categories), which is an automated, pipeline-style data generation chain that automatically generates paired training samples (i.e., reference images and corresponding block ground truth word sequences) required for model training without requiring extensive manual annotation. This process involves raw material input, simulation rendering, structured export, serialization encoding, cleaning and filtering, until a standard dataset suitable for training causal decoding networks is formed. In some examples, this training data synthesis pipeline may include a procedural generator and a block-to-real-image translator.

[0127] First, a procedural generator is used to generate procedurally generated building block engineering files. This involves using 2,000 sets of facade engineering files manually built by professional designers as a base, extracting design elements with parametric capabilities, including the number of floors, the arrangement of window rows and columns, the location of door openings, wall textures, component colors, and decorative shapes. Then, 38,000 sets of brand-new facade engineering files are synthesized in batches through random parameter perturbation, ensuring that the style distribution of the synthesized samples remains consistent with the original manual samples.

[0128] This block-to-realistic image translator outputs photorealistic images. It employs a dual-diffusion transformer (DiT) architecture (reference DiT and generative DiT). Both DiT modules are initialized with weights based on the Stable Diffusion 3.5 model. The reference DiT takes the block rendering image as input and extracts structural features layer by layer. In each denoising step, the generative DiT concatenates the reference features at the same level with noise words and feeds them into the next layer, achieving precise injection of block geometric features into the denoising process. The final output is a photorealistic image that is strictly aligned with the layout of the block facade, doors, windows, and walls. To further enhance image realism, a generative pre-trained transformer can be introduced to generate translation samples. 1000 samples are collected, and 200 high-quality samples are selected to participate in the supervised training of the translator along with the designer's original data.

[0129] Then, 38,000 procedurally generated block project files were paired with photo-realistic images output by the translator, and overlaid with 2,000 manually constructed samples, resulting in a total of 40,000 training samples. The test set was derived from the 2,000 manually constructed samples. Using a unified facial and body structure as a shared basic template, 10 designers freely adjusted appearance elements such as hairstyles, accessories, and clothing colors, ultimately obtaining 3,300 training samples. This data synthesis pipeline is versatile and applicable to other block categories that can be parametrically and template-basedly built.

[0130] It should be noted that in some examples, for the pre-trained causal decoding network for facade categories, the training hardware can use a high-performance graphics processor, the optimizer is Adam, the initial learning rate is set to 1e-4, the batch size is 64, the iteration is 60 rounds, and the complete training cycle is approximately 2 days. For the pre-trained causal decoding network for portrait categories, the training hardware can use a high-performance graphics processor, the optimizer is Adam, the initial learning rate is set to 1e-4, the batch size is 12, the iteration is 150 rounds, and the complete training cycle is approximately 3 hours. For the block-to-realistic image translator, the training hardware uses a general-purpose high-performance graphics processor, the optimizer is Adam, the initial learning rate is set to 1e-5, the batch size is 32, and the iteration is 100 rounds. Inference deployment can adopt a single-card inference architecture, where the average generation time for facade shapes is 55 seconds per model (a single model contains approximately 300 block parts), and the average generation time for portrait shapes is 6 seconds per model (a single model contains 40-63 block parts), with an overall generation speed of up to 300 parts / minute. The model capabilities can be encapsulated as a cloud-based high-concurrency generation API to provide services externally.

[0131] It should be noted that all computational modules in this application are deployed on cloud server GPU nodes for execution; the front end (web page or mini-program) only undertakes the functions of uploading reference images, previewing 3D effects, and displaying and downloading the block product engineering files. Each module is executed in a fixed sequence: image encoding processing, autoregressive generation of block terms (assembly conflict verification and conflict rollback correction are performed simultaneously during the generation process), restoration of the block product engineering files, and return of the engineering files to the front end.

[0132] In summary, this application proposes a compact block segmentation scheme using part attributes as the smallest semantic unit. Each single block component is decomposed into six integer tokens: type, color, orientation, X-axis position, Y-axis position, and Z-axis position. The position quantization step size is 0.8 mm, the greatest common factor of all part dimensions and connection dimensions in the model. This allows for the accurate representation of hundreds of irregularly shaped parts, dozens of colors, and overall machine dimensions in the 400 mm range within a finite vocabulary (e.g., 607 items for the facade category). Furthermore, an autoregressive block generation network structure is proposed, using DINOv2 embedding of reference images as a prefix condition, an OPT-type decoder as the backbone, and a six-layer cascaded Transformer head with a shared output head. This is combined with a multi-token joint prediction (MTP) training strategy, which simultaneously predicts six consecutive tokens for the same part and even the next part during each training step. This allows the model to pre-consider subsequent attributes when predicting the next attribute, improving the coordination between the four attributes of a part. A part-level collision detection-rollback online inference mechanism is also proposed. This mechanism performs assembly semantic-aware collision detection on each generated part and all previously generated parts. If a conflict is found, the candidate token is masked and resampled to ensure that the final output is collision-free. A reverse data synthesis pipeline is also proposed. A procedural generator is used to batch-produce block facade models, and a block-to-real-image translator based on a dual DiT architecture (reference DiT and generative DiT) is used to reverse-translate the block renderings into photorealistic reference images. Several high-quality samples distilled by a generative pre-trained transformer are used for supervision, thereby expanding the manually paired data from two thousand to forty thousand.

[0133] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0134] Corresponding to the three-dimensional block model generation method in the above embodiment, Figure 2The diagram shows a structural schematic of a three-dimensional block model generation device according to an embodiment of this application. For ease of explanation, only the parts related to the embodiment of this application are shown.

[0135] Reference Figure 2 The three-dimensional block model generation device 2 in this embodiment includes: The acquisition module 21 is used to acquire reference images of the building block products.

[0136] The encoding module 22 is used to encode the image features in the reference image to obtain an image word sequence, wherein the image word sequence includes multiple image words, and the image words are used to represent the visual feature information of the reference image.

[0137] The prediction module 23 is used to generate building block words one by one through an autoregressive method using the image word sequence as the conditional guide sequence, and splice them to obtain the building block integer word sequence corresponding to the reference image. The building block integer word sequence includes multiple building block words, which are used to characterize the geometric and assembly attribute information of a single building block component.

[0138] The generation module 24 is used to parse the building block words in the integer word sequence based on the preset building block vocabulary and generate the product engineering file of the building block product. The preset building block vocabulary is used to represent the mapping relationship between the geometric and assembly attribute information of a single building block component and the integer code.

[0139] Module 25 is used to generate a 3D block model corresponding to the block product based on the product engineering file of the block product.

[0140] Furthermore, the encoding module 22 includes: The first encoding submodule is used to encode global image features in the reference image into global image terms, wherein the global image features are used to characterize the overall outline, global tone and global structural summary information of the building block product.

[0141] The second encoding submodule is used to encode multiple block image features in the reference image into multiple local image terms, wherein the block image features are used to characterize the local texture, local color and local contour details of the building block product.

[0142] The third encoding submodule is used to take the global image term as the first word of the sequence and place multiple local image terms after the global image term to obtain the image term sequence.

[0143] Furthermore, the preset block vocabulary is constructed by the union of multiple sub-vocabulary sets. These sub-vocabulary sets include a part type vocabulary set, a color vocabulary set, a three-dimensional orientation vocabulary set, a coordinate position vocabulary set, and a special control character vocabulary set. The special control character vocabulary set includes a sequence start character, a sequence end character, and a fill character.

[0144] Furthermore, the block-type integer lexical sequence also includes a starting special lexical and a ending special lexical.

[0145] Prediction module 23 includes: The initial splicing submodule is used to splice the image word sequence as a fixed prefix with the starting special word to obtain the initial context sequence.

[0146] The lexical decoding submodule is used to input the initial context sequence into the pre-trained causal decoding network, constrain the decoding field of view using causal masks, and output the hidden features corresponding to the current decoding step. The self-attention mechanism in the decoding step is used to associate multiple image lexical units in the image lexical sequence, so that the information carried by the image lexical units is aligned with the spatial regions of the reference image.

[0147] The probability prediction submodule is used to map hidden features to a preset block vocabulary and predict the probability distribution of the block words corresponding to the current decoding step.

[0148] The sequence update submodule is used to add the current building block word sampled from the probability distribution based on a preset sampling strategy to the end of the initial context sequence to obtain the updated context sequence.

[0149] The repeated execution submodule is used to repeatedly execute the word-by-word prediction process, expand the context sequence, and stop when the terminating special word is sampled, thus obtaining the target context sequence.

[0150] The lexical removal submodule is used to remove the image lexical sequence from the target context sequence to obtain a block-shaped integer lexical sequence.

[0151] Furthermore, the three-dimensional block model generation device 2 includes: The entity parsing module is used to parse the preset number of building block words into the three-dimensional network entity corresponding to the current single building block component after accumulating and generating a preset number of building block words, based on the preset total building block vocabulary.

[0152] The conflict verification module is used to perform assembly conflict verification on the 3D network entity of the current single block component based on the 3D network entity corresponding to the historical single block component that has passed the verification. The assembly conflict verification includes at least geometric clipping verification and floating assembly verification.

[0153] The verification processing module is used to mask and resample the non-compliant block words in the current single block component if an assembly conflict is detected, so as to obtain the corrected block words.

[0154] Furthermore, the generation module 24 includes: The information parsing submodule is used to parse the block words in the block integer word sequence based on the preset block vocabulary to obtain the metadata information corresponding to each single block component.

[0155] The file generation submodule is used to sort all single building block components according to preset spatial dimension priority based on the metadata information of the single building block components, and generate product engineering files for the building block products.

[0156] Furthermore, the file generation submodule includes: The sorting unit is used to sort all single building block components in ascending order with Z-axis as the first priority, Y-axis as the second priority, and X-axis as the third priority, to generate an assembly sequence.

[0157] The grouping determination unit is used to assemble and group all single building blocks according to the Z-axis layer to obtain the grouping number of each single building block.

[0158] The numbering determination unit is used to assign an assembly sequence number to each individual building block component according to the assembly sequence.

[0159] The information integration unit is used to integrate the metadata information, grouping numbers and assembly sequence numbers of all individual building blocks to obtain product engineering documents.

[0160] It should be noted that the information interaction and execution process between the modules in the above-mentioned three-dimensional block model generation device 2 are based on the same concept as the method embodiment of this application. For details on their specific functions and technical effects, please refer to the method embodiment section, which will not be repeated here.

[0161] This application also provides a terminal device, such as... Figure 3 As shown, Figure 3 This is a schematic diagram of the structure of a terminal device provided in an embodiment of this application. (Refer to...) Figure 3 The terminal device 3 in this embodiment includes a memory 31, a processor 32, and a computer program stored in the memory 31 and executable on the processor 32. When the processor 32 executes the computer program, it implements the steps in the three-dimensional block model generation method embodiment described above.

[0162] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can implement the steps in the above-described method embodiments.

[0163] This application provides a computer program product that, when run on a mobile terminal, enables the mobile terminal to implement the steps described in the various method embodiments.

[0164] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying the computer program code to a photographic device / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks.

[0165] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0166] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0167] In the embodiments provided in this application, it should be understood that the disclosed apparatus / network devices and methods can be implemented in other ways. For example, the apparatus / network device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0168] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0169] The technical features of the various embodiments described above in this application can be combined arbitrarily without conflict. For the sake of brevity, this specification does not describe all possible combinations, but as long as these combinations do not violate the technical spirit of this application, they should all be considered within the scope of this application. Based on the content disclosed in this application, those skilled in the art can reasonably combine, delete, or replace the technical features of the above embodiments according to actual needs, and such modifications and variations all fall within the protection scope of this application.

[0170] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for generating a three-dimensional block model, characterized in that, include: Obtain reference images of the building block products; The image features in the reference image are encoded to obtain an image word sequence, wherein the image word sequence includes multiple image words, and the image words are used to characterize the visual feature information of the reference image; Using the image lexical sequence as a conditional guide sequence, building block lexicals are generated one by one through an autoregressive method, and then spliced ​​together to obtain a building block integer lexical sequence corresponding to the reference image. The building block integer lexical sequence includes multiple building block lexicals, and the building block lexicals are used to characterize the geometric and assembly attribute information of a single building block component. Based on a preset block vocabulary, the block words in the block integer word sequence are parsed to generate the product engineering file of the block product. The preset block vocabulary is used to characterize the mapping relationship between the geometric and assembly attribute information and the integer code of the single block component. Based on the product engineering file of the building block product, generate a three-dimensional building block model corresponding to the building block product.

2. The method for generating a three-dimensional block model as described in claim 1, characterized in that, The process of encoding the image features in the reference image to obtain an image word sequence includes: The global image features in the reference image are encoded into global image terms, wherein the global image features are used to characterize the overall outline, global tone and global structural summary information of the building block product; Multiple block image features in the reference image are encoded into multiple local image terms, wherein the block image features are used to characterize the local texture, local color and local contour details of the building block product; The global image term is used as the first element of the sequence, and the multiple local image terms are placed after the global image term to obtain the image term sequence.

3. The method for generating a three-dimensional block model as described in claim 2, characterized in that, The preset block vocabulary is constructed by the union of multiple sub-vocabulary sets, wherein the multiple sub-vocabulary sets include a part type vocabulary set, a color vocabulary set, a three-dimensional orientation vocabulary set, a coordinate position vocabulary set, and a special control character vocabulary set. The special control character vocabulary set includes a sequence start character, a sequence end character, and a fill character.

4. The method for generating a three-dimensional block model as described in claim 3, characterized in that, The block-type integer word sequence also includes a start special word and a stop special word; The step of using the image lexical sequence as a conditional guided sequence to generate building block lexical units one by one through an autoregressive method, and concatenating them to obtain a building block integer lexical sequence corresponding to the reference image, includes: The image word sequence is used as a fixed prefix and concatenated with the starting special word to obtain the initial context sequence; The initial context sequence is input into a pre-trained causal decoding network, and a causal mask is used to constrain the decoding field of view to output the hidden features corresponding to the current decoding step. The self-attention mechanism of the decoding step is used to associate multiple image words in the image word sequence so that the information carried by the image words is aligned with the spatial region of the reference image. The hidden features are mapped to the preset block vocabulary to predict the probability distribution of the block words corresponding to the current decoding step; The current building block word sampled from the probability distribution based on a preset sampling strategy is added to the end of the initial context sequence to obtain the updated context sequence; Repeat the word-by-word prediction process to expand the context sequence until the terminating special word is sampled, and then stop to obtain the target context sequence. The image lexical sequence is removed from the target context sequence to obtain the block-type integer lexical sequence.

5. The method for generating a three-dimensional block model as described in claim 4, characterized in that, In the process of adding the current building block word sampled from the probability distribution based on a preset sampling strategy to the end of the initial context sequence to obtain an updated context sequence, the method includes: After generating a preset number of building block words, the preset number of building block words are parsed into the three-dimensional network entity corresponding to the current single building block component according to the preset building block vocabulary. Based on the three-dimensional network entities corresponding to the historical single building block components that have passed verification, the three-dimensional network entities of the current single building block component are subjected to assembly conflict verification, wherein the assembly conflict verification includes at least geometric clipping verification and floating assembly verification. If an assembly conflict is detected, the non-compliant block terms within the current single block component are masked and resampled to obtain corrected block terms.

6. The method for generating a three-dimensional block model as described in claim 5, characterized in that, The process of parsing the block words in the block integer word sequence based on a preset block vocabulary to generate the product engineering file for the block product includes: Based on the preset block vocabulary, the block words in the block integer word sequence are parsed to obtain the metadata information corresponding to each single block component; Based on the metadata information of the individual building blocks, all individual building blocks are sorted according to a preset spatial dimension priority to generate the product engineering file of the building block product.

7. The method for generating a three-dimensional block model as described in claim 6, characterized in that, The step of sorting all single building block components according to a preset spatial dimension priority based on their metadata information to generate the product engineering file for the building block product includes: Sort all single building block components in ascending order with Z-axis as the first priority, Y-axis as the second priority, and X-axis as the third priority to generate an assembly sequence; All individual building blocks are assembled and grouped according to the Z-axis layering to obtain the grouping number of each individual building block; According to the assembly sequence, each of the single building blocks is assigned an assembly sequence number; By integrating the metadata information of all individual building blocks, the grouping number, and the assembly sequence number, the product engineering file is obtained.

8. A three-dimensional block model generation device, characterized in that, include: The acquisition module is used to acquire reference images of the building block products; An encoding module is used to encode the image features in the reference image to obtain an image word sequence, wherein the image word sequence includes multiple image words, and the image words are used to characterize the visual feature information of the reference image; The prediction module is used to generate building block words one by one through an autoregressive method using the image word sequence as a conditional guide sequence, and splice them to obtain a building block integer word sequence corresponding to the reference image. The building block integer word sequence includes multiple building block words, and the building block words are used to characterize the geometric and assembly attribute information of a single building block component. The generation module is used to parse the building block words in the building block integer word sequence based on a preset building block vocabulary and generate the product engineering file of the building block product. The preset building block vocabulary is used to characterize the mapping relationship between the geometric and assembly attribute information and the integer code of the single building block component. The building module is used to generate a three-dimensional building block model corresponding to the building block product based on the product engineering file of the building block product.

9. A terminal device, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the method as claimed in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method as described in any one of claims 1 to 7.

Images