An indoor intelligent generation system based on multi-modal user intention understanding

Abstract

The application relates to the field of semantic processing and database structure, and discloses an indoor intelligent generation system based on multi-modal user intention understanding. A database comprises a region layer, a function group layer and a single component layer, and each node is bound with a corresponding granularity semantic label vector and a physical bounding box parameter. A semantic processing engine analyzes multi-modal instructions to generate a hierarchical semantic label sequence from coarse to fine. A step-by-step drilling search engine matches step by step from top to bottom in the database according to the sequence. A constraint filtering engine performs a Boolean intersection operation on the bounding box parameters of the candidate nodes and the remaining space capacity of the parent node in each layer of search to eliminate out-of-bound nodes, and maps the filtered bottom layer nodes to a coordinate system to complete assembly. The scheme blocks the mutual interference of different granularity semantic features, guarantees the logical coordination of the scene in global space planning and local detail morphology, and eliminates the space interference defects between components in the search stage.

Description

Technical Field

[0001] This invention relates to the fields of semantic processing and database structure, and discloses an indoor intelligent generation system based on multimodal user intent understanding. Background Technology

[0002] Current intelligent interior design systems generally employ a flat database storage structure to manage interior components. The system stores all interior components, including regional blocks, functional combinations, and independent entities, in a flat list format. Each component node is accompanied by a single-granularity text label and geometric dimension parameters. When the system receives multimodal design instructions containing images and text, it uses a feature extraction network to uniformly transform the multimodal instructions into fixed-dimensional feature vectors and performs global similarity matching in the flat database. This matching process does not consider the hierarchical relationship of components within the interior space; it places macroscopic regional style features and microscopic component material features in the same vector space dimension for distance calculation. After a successful match, the system directly imports the matched component model into a 3D coordinate system mesh and completes scene assembly based on simple edge alignment rules.

[0003] The aforementioned flattened retrieval method cannot distinguish semantic levels of different granularities when processing multimodal instructions that include both macro-level spatial planning and micro-level detailed descriptions. Since all components are compared in the same dimension, macro-level style features and micro-level detail features interfere with each other during vector calculation, resulting in a logical mismatch between the global spatial planning and local detailed forms of the retrieved component set. Consequently, the generated interior scene loses its harmony between the overall layout and the local arrangement. Summary of the Invention

[0004] The purpose of this invention is to provide an indoor intelligent generation system based on multimodal user intent understanding, which can effectively solve the problems in the background art mentioned above.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows: An intelligent indoor design system based on multimodal user intent understanding includes: a hierarchical indoor component database, a semantic processing engine, a drill-down retrieval engine, and a constraint filtering engine. The hierarchical indoor component database comprises a region layer, a functional group layer, and a single component layer. Each node in the region layer, functional group layer, and single component layer is bound to a semantic tag vector and physical bounding box parameters of corresponding granularity. The semantic processing engine parses multimodal indoor design instructions and extracts a hierarchical semantic tag sequence from coarse to fine. The drill-down retrieval engine... According to the hierarchical semantic tag sequence, a top-down hierarchical matching process is performed in the hierarchical interior component database, from the region layer to the functional group layer and then to the individual component layer. The constraint filtering engine is coupled to each level of the drill-down retrieval engine and is used to perform Boolean intersection operation between the physical bounding box parameters of the current layer candidate node and the remaining space allocated to the parent node during each level of retrieval, eliminate candidate nodes with space out-of-bounds, and instantiate the filtered final bottom-level individual component nodes and map them to the three-dimensional coordinate system according to the hierarchical relationship to complete the scene assembly system.

[0006] Preferably, the hierarchical interior component database manages nodes by constructing a multidimensional vector space index tree. The nodes of the region layer correspond to the overall interior space blocks, the nodes of the functional group layer correspond to the functional combinations under the region layer nodes, and the nodes of the individual component layer correspond to the independent entities under the functional group layer nodes. During the initialization phase, the hierarchical interior component database performs a vectorization embedding operation on each node, fusing the text description and three-dimensional geometric features in the node into a fixed-dimensional semantic label vector through a multi-head attention mechanism. At the same time, it extracts the extreme point coordinates of the three-dimensional model corresponding to the node in the orthogonal projection direction to construct the physical bounding box parameters, and establishes a bidirectional pointer link between the parent node and the child node in the multidimensional vector space index tree, so that the vector cluster center of the semantic label vector of the child node is limited to the preset offset threshold range of the semantic label vector of the parent node.

[0007] Preferably, the semantic processing engine includes a modal feature extractor and a semantic hierarchy partitioner. The modal feature extractor encodes the text instructions and image instructions in the multimodal interior design instructions, converting the text instructions into a first feature sequence and the image instructions into a second feature sequence through a convolutional network. The semantic hierarchy partitioner performs cross-modal feature alignment between the first feature sequence and the second feature sequence, and divides the fused features into global response features, local response features, and detail response features based on the receptive field of the feature activation response. The semantic hierarchy partitioner maps the global response features to the macro style labels of the corresponding region layer in the hierarchical semantic label sequence, maps the local response features to the circulation layout labels of the corresponding functional group layer, and maps the detail response features to the material and shape labels of the corresponding individual component layer, forming the hierarchical semantic label sequence with strict sequential dependencies.

[0008] Preferably, the drill-down retrieval engine employs a dynamic candidate set pruning strategy when performing top-down step-by-step matching. In the region layer retrieval stage, the drill-down retrieval engine calculates the cosine similarity between the macro style tags in the hierarchical semantic tag sequence and the semantic tag vectors of all nodes in the region layer, and selects several region layer nodes with the highest similarity as the primary candidate set. When entering the functional group layer retrieval stage, the drill-down retrieval engine locks the set of child nodes under each region layer node in the primary candidate set, and performs a secondary similarity calculation on the child node set based on the movement layout tags in the hierarchical semantic tag sequence. At the same time, a parent node confidence decay factor is introduced, and the similarity score generated in the region layer retrieval stage is used as a weight to multiply the similarity score in the functional group layer retrieval stage. Nodes with a product lower than a preset dynamic threshold are eliminated to form a secondary candidate set.

[0009] Preferably, the constraint filtering engine performs the Boolean intersection operation by constructing a spatial octree model. In the retrieval path from the region layer to the functional group layer, when a parent node is selected, the constraint filtering engine initializes and generates a root node spatial octree based on the physical bounding box parameters of the parent node. As the retrieval progresses to the next level, the constraint filtering engine performs node segmentation and status marking on the root node spatial octree based on the physical bounding box parameters of the selected sibling or parent node. The octree leaf nodes corresponding to occupied space are marked as unavailable, and the remaining unmarked octree leaf nodes are merged into a continuous remaining space capacity. When a candidate node is input at the next level, the constraint filtering engine parses the coordinate axis aligned bounding box of the candidate node's physical bounding box parameters, calculates the three-dimensional intersection volume between the coordinate axis aligned bounding box and the octree leaf node corresponding to the remaining space capacity. If the three-dimensional intersection volume is less than the volume of the candidate node's physical bounding box parameters, it is determined that there is a space out-of-bounds condition and a culling operation is performed.

[0010] Preferably, the mapping process for completing scene assembly includes local coordinate system alignment and global coordinate system transformation. In the local coordinate system alignment stage, the system reads the local origin coordinates and normal direction stored in the final bottom-level individual component node after filtering, translates the local origin coordinates to the preset anchor point position within the remaining space capacity allocated by the parent node, and rotates the normal direction to make it parallel to the bottom normal direction of the parent node. In the global coordinate system transformation stage, the system extracts the complete link from the region layer to the individual component layer in the hierarchical indoor component database, performs an accumulation matrix multiplication operation on the vertex coordinates of the individual component nodes that have completed local coordinate system alignment based on the absolute offset of the physical bounding box parameters of each level parent node in the link in the global three-dimensional coordinate system, and writes the three-dimensional mesh data of all individual component nodes mapped to the global three-dimensional coordinate system into the video memory to complete the final rendering assembly.

[0011] Preferably, the hierarchical interior component database has a dynamic topology reorganization mechanism. When new component data is imported into the hierarchical interior component database, the system extracts the semantic tag vector and physical bounding box parameters of the new component, calculates the distance metric between the semantic tag vector of the new component and the semantic tag vectors of existing nodes at each level in the multidimensional vector space index tree, and if the distance metric indicates that the new component simultaneously satisfies the proximity condition with the semantic tag vectors of multiple parent nodes in the functional group layer, and the volume span of the physical bounding box parameters of the new component exceeds the remaining space capacity of any of the multiple parent nodes, the system triggers a node splitting operation, creates a new transitional functional group layer node between the functional group layer and the regional layer, rebinds the new component and some child nodes under the multiple parent nodes to the new transitional functional group layer node, and updates the bidirectional pointer link and the index structure of the multidimensional vector space index tree.

[0012] Preferably, the semantic hierarchy partitioner incorporates modal conflict resolution logic when performing cross-modal feature alignment. When there is an angular difference between the spatial orientation represented by the first feature sequence of the text instruction conversion and the perspective vanishing point direction represented by the second feature sequence of the image instruction conversion, the semantic hierarchy partitioner extracts environmental context features from the multimodal interior design instruction. The environmental context features include the location of door and window openings and the distribution features of load-bearing walls. The system calculates the fit scores of the spatial orientation and the perspective vanishing point direction with the environmental context features, respectively. The modal features with higher fit scores are used as dominant features, and the modal features with lower fit scores are used as auxiliary features. The weight distribution of the auxiliary features is corrected based on the dominant features, and the global response features, the local response features, and the detailed response features are regenerated based on the corrected weight distribution to prevent modal conflicts from causing logical mutual exclusion in the hierarchical semantic label sequences.

[0013] Preferably, the constraint filtering engine introduces a topology reachability verification mechanism in the retrieval path from the functional group layer to the individual component layer. Before calculating the three-dimensional intersection volume of the coordinate axis aligned bounding box of the candidate node and the octagonal leaf node corresponding to the remaining space capacity, the system generates a virtual navigation mesh in the root node spatial octagon based on the movement layout label of the functional group layer node. The virtual navigation mesh is composed of connected octagonal leaf nodes with passage attributes. When the candidate node belongs to the obstruction type component, the constraint filtering engine simulates adding the physical bounding box parameters of the candidate node to the virtual navigation mesh and runs a pathfinding algorithm to detect the connectivity from the starting entry node to the ending exit node in the virtual navigation mesh. If the connectivity is blocked, it is directly determined that the candidate node has spatial out-of-bounds and a culling operation is performed, and the calculation of the three-dimensional intersection volume is no longer performed.

[0014] Preferably, after the global coordinate system transformation stage, a gap elimination and topology closure post-processing stage is introduced. The system traverses the 3D mesh data of all individual component nodes mapped to the global 3D coordinate system, extracts the contact boundary lines between adjacent individual component nodes, calculates the set of shortest distances between mesh vertices on the contact boundary lines, and when there is a gap distance in the set of shortest distances that is greater than a preset tolerance threshold, the system performs a mesh patch extension operation based on the normal direction of the mesh vertices on both sides of the gap distance. After the extension operation, the system performs Laplacian smoothing on the mesh vertices near the contact boundary lines to eliminate non-manifold edges and singular triangles generated by the extension operation, ensuring that adjacent individual component nodes form a watertight closed structure at the 3D mesh topology level, and replaces the initial 3D mesh data in the video memory with the post-processed 3D mesh data to complete the final rendering assembly.

[0015] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This invention establishes a hierarchical interior component database comprising regional, functional group, and individual component layers. A semantic processing engine extracts hierarchical semantic tag sequences from coarse to fine, enabling a drill-down retrieval engine to match macro-style tags, local circulation layout tags, and micro-material shape tags to different levels of the database. This prevents interference between semantic features of different granularities within the same retrieval dimension. The constraint filtering engine, in each retrieval path, performs a Boolean intersection operation between the physical bounding box parameters of candidate nodes and the remaining space allocated to the parent node. This process eliminates nodes with spatial out-of-bounds errors during the retrieval phase, ensuring that the final scene assembly result mapped to the 3D coordinate system simultaneously satisfies the logical coordination requirements of global spatial planning and local detail morphology.

[0016] 2. The hierarchical interior component database establishes bidirectional pointer links between parent and child nodes through a multi-dimensional vector space index tree. This ensures that the vector cluster center of the child node's semantic label vector is constrained by a preset offset threshold of the parent node's semantic label vector, reducing the traversal range of candidate nodes during the drill-down retrieval process. The drill-down retrieval engine introduces a parent node confidence decay factor during top-down matching, multiplying the upper-level retrieval score by the lower-level retrieval score to eliminate nodes below a preset dynamic threshold, preventing invalid nodes from being passed to the next level. A topology reachability verification mechanism is introduced in the retrieval path from the functional group layer to the individual component layer. By running a pathfinding algorithm in the virtual navigation mesh generated by the spatial octree, connectivity is detected, blocking physical obstructions to the interior circulation by obstructive components. Combined with mesh patch extension after global coordinate system transformation and Laplacian smoothing, physical gaps between adjacent individual components are eliminated, maintaining a watertight closed structure of the three-dimensional mesh topology. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating the overall execution process of the indoor intelligent generation system of the present invention. Figure 2 This is a flowchart illustrating the construction process of the hierarchical interior component database of this invention; Figure 3 This is a flowchart of the multimodal instruction parsing and semantic hierarchy division of the present invention; Figure 4 This is a flowchart of the step-by-step drill-down retrieval and dynamic candidate set pruning process of the present invention; Figure 5 This is a flowchart of the spatial interference determination and topology verification process for the constraint filtering engine of this invention. Figure 6 This is a flowchart illustrating the local alignment, global transformation, and post-processing assembly process of this invention. Detailed Implementation

[0018] Please refer to the attached document. Figure 1 This embodiment provides an indoor intelligent generation system based on multimodal user intent understanding, comprising a hierarchical indoor component database, a semantic processing engine, a drill-down retrieval engine, and a constraint filtering engine. The hierarchical indoor component database is configured with a region layer, a functional group layer, and a single component layer. Each node in the region layer, functional group layer, and single component layer is bound to a semantic tag vector and a physical bounding box parameter of corresponding granularity.

[0019] The semantic processing engine receives multimodal interior design instructions, parses the input instructions, and extracts a hierarchical semantic tag sequence from coarse to fine. The drill-down retrieval engine is coupled with the hierarchical interior component database and the semantic processing engine. Based on the hierarchical semantic tag sequence output by the semantic processing engine, it performs a top-down matching operation in the hierarchical interior component database, from the region layer to the functional group layer and then to the individual component layer.

[0020] The constraint filtering engine is coupled to each level of the drill-down search engine in each search path. During each level of search, the constraint filtering engine obtains the physical bounding box parameters of the candidate nodes in the current level, performs a Boolean intersection operation with the remaining space allocated to the corresponding parent node, removes candidate nodes that have exceeded the space limit based on the operation result, performs instantiation operation on the final bottom-level single component nodes after filtering, and maps them to the three-dimensional coordinate system according to the hierarchical relationship between nodes to complete the scene assembly.

[0021] The regional layer nodes of the hierarchical interior component database correspond to the overall interior space blocks, including but not limited to complete interior functional spaces such as living room space blocks, bedroom space blocks, kitchen space blocks, and bathroom space blocks. The semantic tag vector bound to each regional layer node is a macro-granular vector that represents the overall style, spatial scale, lighting attributes, orientation attributes, and other macro-semantic information of the corresponding space block. The bound physical bounding box parameters represent the complete three-dimensional spatial boundary of the corresponding space block, including the extreme value range of the space in the three orthogonal coordinate axis directions.

[0022] The nodes of the functional group layer correspond to the functional combinations under the regional layer nodes, including but not limited to the reception function combination, movie-watching function combination, and storage function combination under the living room space block; the rest function combination, dressing function combination, and dressing function combination under the bedroom space block; and the cooking function combination, washing function combination, and storage function combination under the kitchen space block. The semantic tag vector bound to each functional group layer node is a meso-granular vector, representing the meso-level semantic information such as the circulation layout, functional adaptation attributes, and collaborative relationships of the components within the combination. The bound physical bounding box parameter represents the occupancy range of the corresponding functional combination within the spatial boundary of the regional layer node.

[0023] Each node in the individual component layer corresponds to an independent entity under the functional group layer node, including but not limited to sofas, coffee tables, side tables, and floor lamps under the reception function group; beds, bedside tables, mattresses, and wardrobes under the rest function group; and cabinets, stoves, range hoods, and sinks under the cooking function group. The semantic tag vector bound to each individual component layer node is a micro-granular vector that represents the micro-semantic information of the corresponding independent entity, such as material properties, shape features, size specifications, color attributes, and craftsmanship features. The bound physical bounding box parameters represent the three-dimensional geometric boundary range of the corresponding independent entity.

[0024] The semantic processing engine integrates multimodal interior design commands, including text commands and image commands. Text commands are natural language descriptions input by the user, containing information on the macro style, functional layout, and component details of the interior space. Image commands are reference images of interior scenes input by the user, containing visual information on the style, layout, component style, and material representation of the interior space. The semantic processing engine performs feature extraction operations on the input text and image commands separately, performs fusion processing on the extracted multimodal features, and divides them into semantic tags corresponding to different levels based on the semantic granularity of the fused features. These tags are arranged in order from coarse to fine, forming a hierarchical semantic tag sequence with hierarchical dependencies.

[0025] The hierarchical semantic tag sequence strictly corresponds to the hierarchical order of the hierarchical interior component database. The first tag in the sequence is the macroscopic semantic tag corresponding to the region layer, the middle tags are the mesoscopic semantic tags corresponding to the functional group layer, and the last tag is the microscopic semantic tag corresponding to the individual component layer. Preceding tags in the sequence provide semantic constraints for subsequent tags, ensuring that the semantic scope of a subsequent tag does not exceed the semantic coverage of its predecessor. For example, when the macroscopic semantic tag at the beginning of the sequence is a modern minimalist style living room space, the mesoscopic semantic tags in the middle can only correspond to functional combinations within the living room space, and the microscopic semantic tags at the end can only correspond to the physical components used in a modern minimalist style living room, thus avoiding semantic overreach.

[0026] After the drill-down retrieval engine obtains the hierarchical semantic tag sequence output by the semantic processing engine, it first locks the regional layer of the hierarchical interior component database. It then performs a matching operation between the macro semantic tags of the corresponding regional layer in the hierarchical semantic tag sequence and the semantic tag vectors of all nodes in the regional layer to obtain the matching degree results of each node in the regional layer. Based on the matching degree results, it selects candidate nodes for the regional layer.

[0027] After completing the matching and filtering at the regional level, the drill-down retrieval engine proceeds down to the functional group level, locking onto the set of child nodes corresponding to the candidate nodes at the regional level. It then performs a matching operation between the meso-level semantic tags of the corresponding functional group level in the hierarchical semantic tag sequence and the semantic tag vectors of all nodes within that child node set, obtaining the matching degree results for each node within the functional group level. Based on these matching degree results, candidate nodes for the functional group level are selected. After completing the matching and filtering at the functional group level, the drill-down retrieval engine continues down to the individual component level, locking onto the set of child nodes corresponding to the candidate nodes at the functional group level. It then performs a matching operation between the micro-level semantic tags of the corresponding individual component level in the hierarchical semantic tag sequence and the semantic tag vectors of all nodes within that child node set, obtaining the matching degree results for each node within the individual component level. Based on these matching degree results, candidate nodes for the individual component level are selected.

[0028] The constraint filtering engine performs filtering operations synchronously during each level of the drill-down search engine. In the regional layer search stage, the constraint filtering engine obtains the physical bounding box parameters of the candidate nodes in the regional layer, performs Boolean intersection operations with the target space boundary corresponding to the input interior design instructions, and eliminates candidate nodes in the regional layer whose physical bounding box parameters and target space boundaries exceed the boundary, thus completing the regional layer filtering operation.

[0029] During the functional group layer retrieval phase, the constraint filtering engine obtains the physical bounding box parameters of the candidate nodes in the functional group layer and performs a Boolean intersection operation with the remaining space capacity allocated to the parent node in the corresponding region layer. The remaining space capacity is the spatial range corresponding to the physical bounding box parameters of the parent node in the region layer, after deducting the occupied space corresponding to the physical bounding box parameters of the selected nodes in the same level, and the remaining continuous spatial range. In the calculation process, the physical bounding box of the candidate node is first transformed to the local coordinate system of the parent node, and then the intersection calculation is performed with the remaining space capacity. If the intersection volume of the physical bounding box parameters of the candidate node in the functional group layer and the remaining space capacity is less than the total volume of the physical bounding box of the candidate node, it is determined that the candidate node has a space out-of-bounds condition, and a rejection operation is performed to complete the filtering operation of the functional group layer.

[0030] During the individual component layer retrieval phase, the constraint filtering engine obtains the physical bounding box parameters of the candidate nodes in the individual component layer and performs a Boolean intersection operation with the remaining space capacity allocated to the parent node of the corresponding functional group layer. The remaining space capacity is the spatial range corresponding to the physical bounding box parameters of the parent node of the functional group layer, and the continuous spatial range remaining after deducting the occupied space corresponding to the physical bounding box parameters of the selected nodes at the same level. If the intersection volume of the physical bounding box parameters of the candidate node in the individual component layer and the remaining space capacity is less than the total volume of the physical bounding box of the candidate node, it is determined that the candidate node has space out of bounds, and a rejection operation is performed to complete the filtering operation of the individual component layer.

[0031] After the constraint filtering engine completes filtering operations at all levels, it outputs the final set of bottom-level individual component nodes. The system instantiates all individual component nodes within this set, reads the 3D model data bound to the nodes, and maps the 3D models of the individual component nodes to the 3D coordinate system based on the hierarchical relationships between nodes. During the mapping process, the system first determines the anchor point position of the individual component node within the parent node's spatial range based on the parent node's spatial allocation information, aligns the 3D model of the individual component node to the anchor point position, and then transforms the aligned individual component node 3D model to the global 3D coordinate system based on the complete link from the region layer to the individual component layer in the hierarchical interior component database. After completing the mapping of all individual component nodes, the system integrates the mesh data of all 3D models to complete the scene assembly.

[0032] Table 1. Definition of Core Attributes of Each Level Node in the Hierarchical Interior Component Database

[0033] This table clarifies the node attribute boundaries of the three core levels in the hierarchical interior component database, providing a standardized definition basis for the binding operation of semantic label vectors and physical bounding box parameters of nodes at each level. This ensures that the semantic granularity and spatial coverage of nodes at each level have a strict hierarchical correspondence, avoiding overlap between the semantic and spatial constraints of nodes at different levels.

[0034] In this embodiment, a hierarchical database structure is used to divide interior components into three levels based on semantic granularity and spatial hierarchy. This, combined with semantic tag sequences extracted by the semantic processing engine for each level, allows the retrieval process to proceed from top to bottom, limiting the matching of semantic features of different granularities to their corresponding levels and avoiding interference between features of different granularities in the same retrieval dimension. The constraint filtering engine simultaneously performs spatial boundary checks in each level's retrieval path, eliminating nodes that do not meet spatial constraints during the retrieval phase. This ensures that the final scene assembly result simultaneously meets both semantic matching and spatial constraint requirements, guaranteeing the logical consistency between the global layout and local details of the generated scene.

[0035] In another embodiment, please refer to the appendix. Figure 2 The hierarchical interior component database manages the index of all nodes by constructing a multi-dimensional vector space index tree. Nodes at the region level correspond to the overall interior space blocks, nodes at the functional group level correspond to the functional combinations under the region level nodes, and nodes at the individual component level correspond to the independent entities under the functional group level nodes. During the initialization phase, the hierarchical interior component database performs a vectorization embedding operation on each node, fusing the text description and 3D geometric features in the node into a fixed-dimensional semantic label vector through a multi-head attention mechanism. At the same time, it extracts the extreme point coordinates of the 3D model corresponding to the node in the orthogonal projection direction to construct physical bounding box parameters, and establishes bidirectional pointer links between parent and child nodes in the multi-dimensional vector space index tree, so that the vector cluster center of the semantic label vector of the child node is limited to the preset offset threshold range of the semantic label vector of the parent node.

[0036] Before the vectorized embedding operation, the system performs preprocessing operations on the node's text description, including word segmentation, stop word removal, synonym replacement, and entity linking, converting the text description into a standardized text sequence to ensure semantic consistency. The node's text description includes information such as the name, functional attributes, style features, applicable scenarios, and process types of the corresponding entity. After preprocessing, the system inputs the standardized text sequence into a pre-trained language coding model, performs multi-layer self-attention operations, and extracts a text feature sequence. Each feature vector in the text feature sequence corresponds to the semantic features of the word segmentation unit in the text sequence, with a fixed feature dimension. Simultaneously, the system performs preprocessing operations on the 3D model data bound to the node, including mesh resampling, normal normalization, and scale normalization, scaling the 3D model to a unit cube space to eliminate the impact of model scale differences on geometric feature extraction. A uniformly distributed set of mesh vertices is extracted from the preprocessed 3D model, and the coordinates, normal vectors, Gaussian curvature, and mean curvature features of each mesh vertex are extracted. The extracted features are arranged in vertex order to obtain a geometric feature sequence, with the feature dimension of the geometric feature sequence consistent with that of the text feature sequence.

[0037] The system inputs text feature sequences and geometric feature sequences into the multi-head attention fusion module, performs cross-feature attention fusion operations, enabling deep interaction between text semantic features and 3D geometric features, and outputs a fused global feature vector. This global feature vector is the fixed-dimensional semantic label vector bound to the node. The operation process of the multi-head attention fusion module is defined by the following formula:

[0038] in, The attention query matrix is ​​obtained by linear transformation of the input text feature sequence; The attention key matrix is ​​obtained by linear transformation of the input geometric feature sequence; The attention value matrix is ​​obtained by linear transformation of the input geometric feature sequence; The feature dimension of the key matrix; This is a normalized exponential function used to map attention weights to a range of 0 to 1, with the sum of all weights being 1.

[0039]

[0040] in, For the number of heads; For the first The output of each attention head, ; , , These are the linear transformation weight matrices for the query matrix, key matrix, and value matrix corresponding to the i-th attention head, respectively. The linear transformation weight matrix for the multi-head attention output; This is a concatenation operation used to concatenate the output features of multiple attention heads into a single feature sequence.

[0041] After the system generates the semantic tag vector, it performs the physical bounding box parameter construction operation. It reads the coordinates of all mesh vertices of the 3D model corresponding to the node, projects all mesh vertex coordinates onto three orthogonal coordinate axes, extracts the minimum and maximum extreme point coordinates in each coordinate axis direction, and combines them to obtain the coordinate axis aligned physical bounding box parameters of the node. The specific definition is implemented through the following formula:

[0042] in, , , These are the minimum coordinate values ​​of all mesh vertices in the 3D model along the X, Y, and Z axes, respectively. , , These represent the maximum coordinate values ​​of all mesh vertices in the 3D model along the X, Y, and Z axes, respectively. The physical bounding box parameter is aligned to the coordinate axes and represents the minimum bounding box boundary of the 3D model.

[0043] The system constructs a multidimensional vector space index tree based on the generated semantic label vectors and physical bounding box parameters. The multidimensional vector space index tree is constructed based on the vector space of semantic label vectors. The depth of the tree corresponds to the number of levels in the hierarchical interior component database. The root node of the tree corresponds to the set of regional layer nodes, the middle nodes of the tree correspond to the set of functional group layer nodes, and the leaf nodes of the tree correspond to the set of individual component layer nodes.

[0044] During the index tree construction process, the system establishes bidirectional pointer links between each parent node and its corresponding child nodes. The parent node's pointer points to all its subordinate child nodes, and the child node's pointer points to its unique parent node. This bidirectional pointer link enables rapid retrieval and traversal of hierarchical relationships between nodes, quickly locating the target node's parent and child nodes without performing a full database traversal. Simultaneously, the system performs cluster analysis on the semantic tag vectors of all child nodes under each parent node, through... The mean clustering algorithm calculates the cluster center of the semantic label vectors of child nodes. A preset offset threshold is set to ensure that the vector distance between the cluster center and the semantic label vector of the parent node does not exceed the preset offset threshold range. The specific constraint relationship is defined by the following formula:

[0045] in, It serves as the cluster center for the semantic label vectors of all child nodes under the same parent node; This is the semantic label vector of the corresponding parent node; It is the L2 norm, used to calculate the Euclidean distance between two vectors; This is a preset offset threshold used to limit the maximum allowable distance between the cluster center of child nodes and the vector of the parent node.

[0046] The hierarchical interior component database has a dynamic topology reorganization mechanism. When new component data is imported into the hierarchical interior component database, the system performs preprocessing operations on the new component data, extracts the semantic label vector and physical bounding box parameters of the new component, and calculates the distance metric between the semantic label vector of the new component and the semantic label vectors of existing nodes at each level in the multidimensional vector space index tree. The distance metric is calculated using the following formula:

[0047] in, This is the semantic tag vector for the newly added component; For the multidimensional vector space index tree, the first The semantic label vector of each existing node; The Euclidean distance between two vectors is used as a distance metric.

[0048] The system determines the proximity relationship between the newly added component and existing nodes based on the calculated distance metric. If the distance metric is less than a preset proximity threshold, the newly added component and the corresponding existing node are considered to meet the proximity condition. If the distance metric indicates that the newly added component simultaneously meets the proximity condition with the semantic label vectors of multiple parent nodes in the functional group layer, and the volume span of the physical bounding box parameter of the newly added component exceeds the remaining space capacity of any of the multiple parent nodes, the system triggers a node splitting operation. A new transitional functional group layer node is created between the functional group layer and the region layer. The newly added component and some child nodes under the multiple parent nodes are rebound to the new transitional functional group layer node, and the bidirectional pointer links and the index structure of the multidimensional vector space index tree are updated.

[0049] After the index structure is updated, the system recalculates the cluster center of the semantic label vectors of all child nodes under the new transition function group layer node, and verifies whether the distance between the cluster center and the semantic label vector of the new transition function group layer node meets the constraint of the preset offset threshold. If it does not meet the constraint, the binding adjustment of the child nodes is re-executed until the constraint condition is met.

[0050] Table 2. Multidimensional Vector Space Index Tree Hierarchical Index Configuration Parameters

[0051] This table clarifies the configuration parameters and mapping rules for each level of the multidimensional vector space index tree, providing a standardized configuration basis for the construction, updating, and dynamic topology reorganization of the index tree. It ensures that the hierarchical structure of the index tree strictly corresponds to the hierarchical structure of the hierarchical interior component database, while also providing clear parameter constraint boundaries for the topology adjustment of newly added nodes.

[0052] In this embodiment, a multi-head attention mechanism is used to fuse text descriptions and 3D geometric features to generate semantic label vectors, ensuring that the semantic label vectors simultaneously contain both textual semantic information and geometric feature information, thus improving the accuracy of semantic matching. A multi-dimensional vector space index tree and bidirectional pointer links, along with offset threshold constraints on child node cluster centers, reduce the candidate node traversal range during the step-by-step drill-down retrieval process, improving retrieval efficiency. A dynamic topology reorganization mechanism enables dynamic adjustment of the database structure, adapting to the import requirements of newly added component data and ensuring the stability of the database hierarchy and the consistency of semantic constraints.

[0053] In one alternative embodiment, please refer to the appendix. Figure 3 The semantic processing engine includes a modal feature extractor and a semantic hierarchy segmenter. The modal feature extractor encodes the text instructions and image instructions in the multimodal interior design instructions, converting the text instructions into a first feature sequence and the image instructions into a second feature sequence through a convolutional network.

[0054] The semantic hierarchy partitioner performs cross-modal feature alignment between the first feature sequence and the second feature sequence. Based on the receptive field of the feature activation response, the fused features are divided into global response features, local response features, and detailed response features. The semantic hierarchy partitioner maps the global response features to the macro style labels of the corresponding region layer in the hierarchical semantic label sequence, maps the local response features to the movement layout labels of the corresponding functional group layer, and maps the detailed response features to the material and shape labels of the corresponding individual component layer, forming a hierarchical semantic label sequence with strict sequential dependencies.

[0055] The modal feature extractor has built-in text encoding and image encoding branches. The text encoding branch receives text instructions and performs preprocessing operations on the text instructions, including word segmentation, punctuation filtering, special character removal, and sequence length padding. The preprocessed text sequence is then input into the Transformer encoder to perform multi-layer self-attention operations and output a first feature sequence with fixed dimensions. Each feature vector in the first feature sequence corresponds to the semantic features of the word segmentation unit in the text instruction. The feature dimension is consistent with the semantic label vector dimension of the nodes in the hierarchical interior component database.

[0056] The image encoding branch receives image instructions and performs preprocessing operations on them, including size normalization, pixel value normalization, and color space conversion. The preprocessed image is then input into the convolutional neural network backbone, where multi-scale convolution and pooling operations are performed to extract feature maps of different depths. The convolutional neural network backbone uses a ResNet structure for feature map extraction. The feature maps are divided into five stages, each corresponding to a different receptive field. The receptive field of the feature map is largest at each stage. The feature map at the stage has the smallest receptive field. The system performs flattening and linear transformation on the feature maps at different depths and outputs a second feature sequence with fixed dimensions. The feature dimension of the second feature sequence is consistent with the feature dimension of the first feature sequence. Each feature vector in the second feature sequence corresponds to the visual features of a local region in the image instruction.

[0057] After the semantic hierarchy partitioner obtains the first and second feature sequences, it performs a linear transformation on the two feature sequences to map them to the same feature space. Then, it performs a cross-modal feature alignment operation to obtain the fused aligned feature sequence. The cross-modal feature alignment process is defined by the following formula:

[0058] in, This is the first characteristic sequence; This is the second characteristic sequence; This is the linear transformation weight matrix corresponding to the first feature sequence; is the linear transformation weight matrix corresponding to the second feature sequence; b is the bias term of the linear transformation; This is the fused feature sequence after cross-modal alignment.

[0059] The semantic hierarchical partitioner performs feature activation response analysis on the aligned fused feature sequence, calculating the receptive field range corresponding to each feature vector in the fused feature sequence. The receptive field range represents the coverage area of ​​the input information corresponding to the feature vector. The semantic hierarchical partitioner sets a global receptive field threshold and a detail receptive field threshold. The global receptive field threshold corresponds to the minimum receptive field range covering the entire input image, and the detail receptive field threshold corresponds to the maximum receptive field range covering the local detail regions of the input image. Feature vectors with a receptive field range greater than the global receptive field threshold are classified as global response features, feature vectors with a receptive field range between the global and detail receptive field thresholds are classified as local response features, and feature vectors with a receptive field range less than the detail receptive field threshold are classified as detail response features.

[0060] The semantic hierarchy partitioner performs global average pooling on the global response features to obtain a global semantic feature vector. This global semantic feature vector is then mapped to a macro-style label for the corresponding region layer in a hierarchical semantic label sequence. The macro-style label represents macro-semantic information such as the overall style, space type, global scale, and orientation attributes of the interior space. The semantic hierarchy partitioner also performs regional average pooling on the local response features, dividing them into multiple spatial regions based on their spatial location. Average pooling is then performed on the feature vector within each spatial region to obtain multiple local semantic feature vectors. These local semantic feature vectors are then mapped to circulation layout labels for the corresponding functional group layer in a hierarchical semantic label sequence. The circulation layout labels represent meso-level semantic information such as functional zoning, circulation paths, positional relationships of functional combinations, and entrance / exit distribution within the interior space.

[0061] The semantic hierarchy partitioner performs local max pooling on the detailed response features, extracting the feature vector with the highest activation response within each local region to obtain multiple detailed semantic feature vectors. These detailed semantic feature vectors are then mapped to material and shape labels for the corresponding individual component layers in the hierarchical semantic label sequence. The material and shape labels represent the micro-level semantic information of the interior components, such as material type, shape features, color attributes, size specifications, and craftsmanship details. The semantic hierarchy partitioner arranges the macro-style labels, circulation layout labels, and material and shape labels in a coarse-to-fine order, forming a hierarchical semantic label sequence with strict sequential dependencies. The semantic scope of a preceding label in the sequence provides semantic constraints for subsequent labels, ensuring that the semantic information of a subsequent label does not exceed the semantic coverage of its preceding label.

[0062] The semantic hierarchical partitioner incorporates modal conflict resolution logic when performing cross-modal feature alignment. When there is an angular difference between the spatial orientation represented by the first feature sequence of text instruction conversion and the perspective vanishing point direction represented by the second feature sequence of image instruction conversion, the semantic hierarchical partitioner extracts environmental context features from the multimodal interior design instructions. The environmental context features include the location of door and window openings and the distribution features of load-bearing walls. The environmental context features are jointly extracted from the spatial description information of the text instructions and the scene structure information of the image instructions.

[0063] The system calculates the fit scores between the spatial orientation and perspective vanishing point direction and the environmental context features. Modal features with higher fit scores are designated as dominant features, while those with lower scores are designated as auxiliary features. The weight distribution of the auxiliary features is adjusted based on the dominant features, and global, local, and detail response features are regenerated based on this adjusted weight distribution. This prevents logical mutual exclusion in hierarchical semantic label sequences caused by modal conflicts. The fit score calculation process is defined by the following formula:

[0064] in, For the first The fit score of each modality; N is the number of key feature points in the environmental context features, including the corners of door and window openings and the endpoints of load-bearing walls; For the first The direction vector of the mode and the _th mode The angle between the normal directions of the key feature points; The cosine of the angle is used to characterize the degree of matching between the direction vector and the normal direction of the key feature point.

[0065] After calculating the fitness scores of two modalities, the system compares the numerical values ​​of the two scores and sets the feature sequence corresponding to the modality with the higher score as the dominant feature sequence and the feature sequence corresponding to the modality with the lower score as the auxiliary feature sequence. Based on the feature distribution of the dominant feature sequence, the system performs a correction operation on the weight matrix of the auxiliary feature sequence, reducing the weight of feature vectors in the auxiliary feature sequence that conflict with the dominant feature sequence and increasing the weight of feature vectors in the auxiliary feature sequence that are consistent with the dominant feature sequence. The system then re-performs cross-modal feature alignment with the dominant feature sequence to obtain a corrected fused feature sequence. Based on the corrected fused feature sequence, it re-performs feature hierarchy partitioning to generate new global response features, local response features, and detail response features, ultimately obtaining a hierarchical semantic label sequence after eliminating modal conflicts.

[0066] Table 3. Mapping Relationship of Multimodal Feature Levels

[0067] This table clarifies the hierarchical division rules and mapping relationships of multimodal fusion features, providing a standardized basis for feature classification and label mapping operations of the semantic hierarchy divider. It ensures that features with different perceptual field ranges can be accurately mapped to the corresponding levels in the hierarchical interior component database, guaranteeing a strict correspondence between the semantic granularity of the hierarchical semantic label sequence and the database hierarchy.

[0068] In this embodiment, a dual-branch modal feature extractor encodes text and image commands separately. The fused features are hierarchically divided based on the receptive field of the feature activation response, allowing semantic features of different granularities to correspond to different levels in the database. This ensures the matching of the semantic granularity of the hierarchical semantic tag sequence with the database's hierarchical structure. The built-in modal conflict resolution logic effectively resolves semantic conflicts between text and image commands, preventing logical mutual exclusion in the hierarchical semantic tag sequence and improving the accuracy and stability of semantic parsing.

[0069] In yet another alternative embodiment, please refer to the appendix. Figure 4The drill-down retrieval engine employs a dynamic candidate set pruning strategy when performing top-down hierarchical matching. In the region-level retrieval stage, the engine calculates the cosine similarity between the macro-style labels in the hierarchical semantic label sequence and the semantic label vectors of all nodes in the region layer, selecting the top-ranking region-level nodes as the primary candidate set. Upon entering the functional group-level retrieval stage, the engine locks the child node sets under each region-level node in the primary candidate set and performs a secondary similarity calculation on the child node sets based on the movement layout labels in the hierarchical semantic label sequence. Simultaneously, a parent node confidence decay factor is introduced, multiplying the similarity score generated in the region-level retrieval stage by the similarity score in the functional group-level retrieval stage, and eliminating nodes whose product is below a preset dynamic threshold to form the secondary candidate set.

[0070] In the region-level retrieval stage, the retrieval engine drills down level by level to obtain the query vector corresponding to the macro style label in the hierarchical semantic label sequence. The cosine similarity between the query vector and the semantic label vector of all nodes in the region layer is calculated to obtain the similarity score of each node in the region layer. The calculation process of cosine similarity is defined by the following formula:

[0071] in, This is the query tag vector corresponding to the level in the hierarchical semantic tag sequence; This is the semantic tag vector of the corresponding hierarchical node in the hierarchical interior component database; Cosine similarity is the similarity between two vectors, ranging from -1 to 1. The closer the value is to 1, the higher the semantic matching degree between the two vectors.

[0072] The system sorts the regional layer nodes in descending order of similarity score, selects a preset number of the top-ranked regional layer nodes to form a primary candidate set. The number of nodes in the primary candidate set is dynamically adjusted according to the average similarity score of the regional layer nodes. The higher the average similarity score, the fewer the nodes in the primary candidate set, and the lower the average similarity score, the more the nodes in the primary candidate set, ensuring a balance between the coverage and matching accuracy of the candidate set.

[0073] After constructing the initial candidate set, the drill-down retrieval engine proceeds to the functional group layer retrieval stage. For each region layer node in the initial candidate set, all functional group layer child nodes under that region layer node are locked through bidirectional pointer links, forming a child node set. The drill-down retrieval engine obtains the query vector corresponding to the movement layout label in the hierarchical semantic label sequence, calculates the cosine similarity between the query vector and the semantic label vector of each functional group layer node in the child node set, and obtains the similarity score of the functional group layer node. The drill-down retrieval engine introduces a parent node confidence decay factor, using the similarity score obtained by the corresponding region layer parent node in the region layer retrieval stage as a weight, and performs a weighted operation with the similarity score of the functional group layer node to obtain the comprehensive score of the functional group layer node. The calculation process of the comprehensive score is defined by the following formula:

[0074] in, The overall score for the sub-nodes of the functional group layer; The cosine similarity between the functional group layer child nodes and the corresponding query tag vectors; The cosine similarity score is given to the parent node of the corresponding region layer. This is the confidence decay factor for the parent node, with a value ranging from 0 to 1. It is used to adjust the degree of influence of the similarity of the parent node on the score of the lower-level node.

[0075] The drill-down search engine sets a preset dynamic threshold, which is a preset proportion of the average comprehensive score of the secondary candidate set. This dynamic threshold is dynamically adjusted during the search process. Nodes in the functional group layer whose comprehensive score is lower than the preset dynamic threshold are removed, while nodes with a comprehensive score equal to or higher than the preset dynamic threshold are retained to form the secondary candidate set. Upon entering the individual component layer search stage, the drill-down search engine locks the set of child nodes under each functional group layer node in the secondary candidate set. Based on the material and shape labels in the hierarchical semantic label sequence, it performs three similarity calculations on the child node set. Simultaneously, it introduces a parent node confidence decay factor, multiplies the comprehensive score generated in the functional group layer search stage by the similarity score in the individual component layer search stage, and removes nodes whose product is lower than the preset dynamic threshold, forming the final candidate set.

[0076] Please refer to the attached document. Figure 5The constraint filtering engine performs Boolean intersection operations by constructing a spatial octree model. In the retrieval path from the region layer to the functional group layer, once a parent node is selected, the constraint filtering engine initializes and generates a root node spatial octree based on the physical bounding box parameters of the parent node. The boundary of the root node spatial octree completely coincides with the boundary of the parent node's physical bounding box. As the retrieval progresses to the next level, the constraint filtering engine performs node segmentation and status marking on the root node spatial octree based on the physical bounding box parameters of the selected sibling or parent nodes. It marks the leaf nodes of the octree corresponding to occupied space as unavailable and merges the remaining unmarked leaf nodes into a continuous remaining space capacity.

[0077] When a candidate node is input at the next level, the constraint filtering engine parses the coordinate axis-aligned bounding box of the candidate node's physical bounding box parameters, transforms the coordinate axis-aligned bounding box to the parent node's local coordinate system, and calculates the 3D intersection volume between the coordinate axis-aligned bounding box and the octagonal leaf node corresponding to the remaining space capacity. If the 3D intersection volume is less than the volume of the candidate node's physical bounding box parameters, it is determined that there is a space out-of-bounds error and a culling operation is performed. The calculation process of the 3D intersection volume is defined by the following formula:

[0078] in, The 3D intersection volume of the bounding box of the candidate node and the remaining space capacity is defined by the coordinate axes of the node being aligned with the coordinate axes of the bounding box. X, Y, and Z are the coordinate ranges of the X-axis, Y-axis, and Z-axis corresponding to the physical bounding box of the parent node, respectively. This is an indicator function for aligning the coordinate axes of the candidate nodes with the bounding box. It takes a value of 1 within the spatial range corresponding to the bounding box, and a value of 0 otherwise. This is an indicator function for the remaining space capacity corresponding to the octagonal leaf nodes. It takes a value of 1 within the available space range, and a value of 0 otherwise.

[0079] The constraint filtering engine introduces a topology reachability verification mechanism in the retrieval path from the functional group layer to the individual component layer. Before calculating the three-dimensional intersection volume of the coordinate axis aligned bounding box of the candidate node and the octagonal leaf node corresponding to the remaining space capacity, the system generates a virtual navigation grid in the root node spatial octagon based on the movement layout label of the functional group layer node. The virtual navigation grid is composed of connected octagonal leaf nodes with accessibility attributes. There are no fixed obstacles in the space corresponding to the octagonal leaf nodes with accessibility attributes, which meets the spatial requirements for personnel passage.

[0080] When a candidate node is an obstruction-type component, the constraint filtering engine simulates adding the physical bounding box parameters of the candidate node to the virtual navigation mesh. Leaf nodes in the virtual navigation mesh that intersect with the physical bounding box of the candidate node are marked as obstructions. A pathfinding algorithm is then run to detect the connectivity from the starting entrance node to the ending exit node in the virtual navigation mesh. If connectivity is blocked, the candidate node is directly determined to have exceeded its spatial boundaries and is removed; the calculation of the 3D intersection volume is not performed. Obstruction-type components include, but are not limited to, walls, partitions, cabinets, and other components that obstruct spatial passage. The starting entrance node and ending exit node are determined based on the entrance / exit positions corresponding to the circulation layout labels of the functional group layer nodes. The pathfinding algorithm detects whether a continuous path exists between the starting entrance node and the ending exit node; if not, connectivity is considered blocked.

[0081] Please refer to the attached document. Figure 6 The mapping process for completing scene assembly includes local coordinate system alignment and global coordinate system transformation. In the local coordinate system alignment stage, the system reads the local origin coordinates and normal direction stored in the final bottom-level single component node after filtering, translates the local origin coordinates to the preset anchor point position within the remaining space allocated by the parent node, and rotates the normal direction to make it parallel to the bottom normal direction of the parent node. The transformation process of local coordinate system alignment is defined by the following formula:

[0082] in, These are the original local vertex coordinates of the 3D model of a single component node; This is a rotation matrix used to rotate the normal direction of a single component node so that it is parallel to the opposite direction of the normal direction of the parent node's bottom surface. This is a translation vector used to translate the local origin coordinates of a single component node to the preset anchor point position assigned by the parent node; The vertex coordinates after local coordinate system alignment.

[0083] During the global coordinate system transformation phase, the system extracts the complete link from the region layer to the individual component layer from the hierarchical interior component database. Based on the absolute offset of the physical bounding box parameters of each parent node in the link in the global 3D coordinate system, it performs an accumulation matrix multiplication operation on the vertex coordinates of the individual component nodes that have completed local coordinate system alignment. The 3D mesh data of all individual component nodes mapped to the global 3D coordinate system are then written to the video memory to complete the final rendering assembly. The accumulation matrix multiplication operation for the global coordinate system transformation is defined by the following formula:

[0084] in, These are the vertex coordinates in the global 3D coordinate system. to For each level in the hierarchical link from the region layer to the parent node of the single component layer, the transformation matrix contains the absolute offset and rotation of the corresponding level node in the global three-dimensional coordinate system. The vertex coordinates after local coordinate system alignment.

[0085] Following the global coordinate system transformation stage, a post-processing stage for gap elimination and topology closure is introduced. The system traverses the 3D mesh data of all individual component nodes mapped to the global 3D coordinate system, extracts the contact boundary lines between adjacent individual component nodes, which are the intersection boundaries of the 3D meshes of two adjacent individual component nodes, and calculates the set of shortest distances between mesh vertices on the contact boundary lines. When there is a gap distance in the set of shortest distances that is greater than a preset tolerance threshold, the system performs a mesh patch extension operation based on the normal direction of the mesh vertices on both sides of the gap distance. During the extension process, the mesh vertices on both sides of the gap distance are taken as the starting point, and the extension is carried outward along the normal direction of the vertex. The extension distance is equal to the gap distance at the corresponding position, so that the extended mesh vertices can contact the mesh vertices of adjacent individual component nodes, thus eliminating the gap.

[0086] After the stretching operation, the system performs Laplacian smoothing on the mesh vertices near the contact boundary line to eliminate non-manifold edges and singular triangles generated by the stretching operation. This ensures that adjacent individual component nodes form a watertight closed structure at the 3D mesh topology level. The post-processed 3D mesh data then replaces the initial 3D mesh data in the video memory to complete the final rendering and assembly. The Laplacian smoothing process is defined by the following formula:

[0087] in, The original coordinates of the grid vertices to be smoothed near the contact boundary line; These are the coordinates of the smoothed mesh vertices; The smoothing coefficient is used to adjust the intensity of the smoothing process. As vertex The number of adjacent vertices; As vertex The set of adjacent vertices.

[0088] After completing the Laplace smoothing process, the system performs a topology check on the 3D mesh to identify and correct topological defects such as non-manifold edges, repeated vertices, and degenerate triangles, ensuring that all mesh patches are manifold structures and that the contact boundaries of adjacent individual component nodes are completely closed, forming a watertight 3D mesh topology.

[0089] Table 4. Rules for Marking and Processing Node States in Spatial Octree

[0090] This table clarifies the definition of the state marking of spatial octree nodes and the corresponding processing rules, providing a standardized execution basis for the spatial octree construction, node state marking, Boolean intersection operation and topology reachability verification operations of the constraint filtering engine, ensuring the logical consistency and execution accuracy of spatial constraint verification and mobility connectivity verification.

[0091] In this embodiment, a dynamic candidate set pruning strategy and a parent node confidence decay factor are used to gradually eliminate invalid nodes with low matching degree during the drill-down retrieval process. This prevents invalid nodes from being passed to the next level, reduces the number of candidate nodes in the retrieval process, and improves retrieval efficiency. By performing Boolean intersection operations using a spatial octree model, nodes that do not meet spatial constraints are eliminated during the retrieval stage, ensuring the rationality of the spatial layout of the generated scene.

[0092] The topology accessibility verification mechanism can identify and remove obstructive components that block indoor circulation during the retrieval stage, avoiding circulation blockage issues in the generated scene. Through local coordinate system alignment and global coordinate system transformation, combined with gap elimination and Laplacian smoothing in the post-processing stage, physical gaps between adjacent individual components are eliminated, maintaining the watertight closed structure of the 3D mesh topology and improving the model quality of the final generated scene.

Claims

1. An indoor intelligent generation system based on multimodal user intent understanding, characterized in that, include: The system comprises a hierarchical interior component database, a semantic processing engine, a drill-down retrieval engine, and a constraint filtering engine. The hierarchical interior component database includes a region layer, a functional group layer, and a single component layer. Each node in each of these layers is bound to a semantic tag vector and a physical bounding box parameter of corresponding granularity. The semantic processing engine parses multimodal interior design instructions, extracting a hierarchical semantic tag sequence from coarse to fine. The drill-down retrieval engine performs a top-down, step-by-step matching process in the hierarchical interior component database based on the hierarchical semantic tag sequence, moving from the region layer to the functional group layer and then to the single component layer. The constraint filtering engine, coupled to each level of the drill-down retrieval engine, performs a Boolean intersection operation between the physical bounding box parameter of the current layer's candidate node and the remaining space allocated to the parent node during each level's retrieval process. Candidate nodes with spatial out-of-bounds errors are eliminated, and the filtered final bottom-level single component nodes are instantiated and mapped to a 3D coordinate system according to the hierarchical relationship to complete the scene assembly system.

2. The indoor intelligent generation system based on multimodal user intent understanding according to claim 1, characterized in that, The hierarchical interior component database manages nodes by constructing a multidimensional vector space index tree. Nodes in the region layer correspond to the overall interior space blocks, nodes in the functional group layer correspond to the functional combinations under the region layer nodes, and nodes in the individual component layer correspond to the independent entities under the functional group layer nodes. During the initialization phase, the hierarchical interior component database performs a vectorization embedding operation on each node, fusing the text description and three-dimensional geometric features in the node into a fixed-dimensional semantic label vector through a multi-head attention mechanism. At the same time, it extracts the extreme point coordinates of the three-dimensional model corresponding to the node in the orthogonal projection direction to construct the physical bounding box parameters, and establishes bidirectional pointer links between parent and child nodes in the multidimensional vector space index tree, so that the vector cluster center of the semantic label vector of the child node is limited to the preset offset threshold range of the semantic label vector of the parent node.

3. The indoor intelligent generation system based on multimodal user intent understanding according to claim 1, characterized in that, The semantic processing engine includes a modal feature extractor and a semantic hierarchy partitioner. The modal feature extractor encodes the text instructions and image instructions in the multimodal interior design instructions, converting the text instructions into a first feature sequence and the image instructions into a second feature sequence through a convolutional network. The semantic hierarchy partitioner performs cross-modal feature alignment between the first feature sequence and the second feature sequence, and divides the fused features into global response features, local response features, and detail response features based on the receptive field of the feature activation response. The semantic hierarchy partitioner maps the global response features to the macro style labels of the corresponding region layer in the hierarchical semantic label sequence, maps the local response features to the circulation layout labels of the corresponding functional group layer, and maps the detail response features to the material and shape labels of the corresponding individual component layer, forming the hierarchical semantic label sequence with strict sequential dependencies.

4. The indoor intelligent generation system based on multimodal user intent understanding according to claim 1, characterized in that, The drill-down retrieval engine employs a dynamic candidate set pruning strategy when performing top-down hierarchical matching. In the region layer retrieval stage, the engine calculates the cosine similarity between the macro style tags in the hierarchical semantic tag sequence and the semantic tag vectors of all nodes in the region layer. It selects several region layer nodes with the highest similarity as the primary candidate set. When entering the functional group layer retrieval stage, the engine locks the child node set under each region layer node in the primary candidate set and performs a secondary similarity calculation on the child node set based on the movement layout tags in the hierarchical semantic tag sequence. At the same time, it introduces a parent node confidence decay factor, multiplies the similarity score generated in the region layer retrieval stage by the similarity score in the functional group layer retrieval stage, and removes nodes whose product is lower than a preset dynamic threshold to form a secondary candidate set.

5. The indoor intelligent generation system based on multimodal user intent understanding according to claim 1, characterized in that, The constraint filtering engine performs the Boolean intersection operation by constructing a spatial octree model. In the retrieval path from the region layer to the functional group layer, when a parent node is selected, the constraint filtering engine initializes and generates a root node spatial octree based on the physical bounding box parameters of the parent node. As the retrieval progresses to the next level, the constraint filtering engine performs node segmentation and status marking on the root node spatial octree based on the physical bounding box parameters of the selected sibling or parent node. The octree leaf nodes corresponding to occupied space are marked as unavailable, and the remaining unmarked octree leaf nodes are merged into a continuous remaining space capacity. When a candidate node is input at the next level, the constraint filtering engine parses the coordinate axis aligned bounding box of the candidate node's physical bounding box parameters, calculates the three-dimensional intersection volume between the coordinate axis aligned bounding box and the octree leaf node corresponding to the remaining space capacity. If the three-dimensional intersection volume is less than the volume of the candidate node's physical bounding box parameters, it is determined that there is a space out-of-bounds condition and a culling operation is performed.

6. The indoor intelligent generation system based on multimodal user intent understanding according to claim 1, characterized in that, The mapping process for completing scene assembly includes local coordinate system alignment and global coordinate system transformation. In the local coordinate system alignment stage, the system reads the local origin coordinates and normal direction stored in the final bottom-level individual component nodes after filtering, translates the local origin coordinates to the preset anchor point position within the remaining space capacity allocated by the parent node, and rotates the normal direction to make it parallel to the bottom normal direction of the parent node. In the global coordinate system transformation stage, the system extracts the complete link from the region layer to the individual component layer in the hierarchical interior component database, and performs an accumulation matrix multiplication operation on the vertex coordinates of the individual component nodes that have completed local coordinate system alignment based on the absolute offset of the physical bounding box parameters of each level parent node in the global three-dimensional coordinate system. The system writes the three-dimensional mesh data of all individual component nodes mapped to the global three-dimensional coordinate system into the video memory to complete the final rendering assembly.

7. The indoor intelligent generation system based on multimodal user intent understanding according to claim 2, characterized in that, The hierarchical interior component database has a dynamic topology reorganization mechanism. When new component data is imported into the hierarchical interior component database, the system extracts the semantic tag vector and physical bounding box parameters of the new component, calculates the distance metric between the semantic tag vector of the new component and the semantic tag vectors of existing nodes at each level in the multidimensional vector space index tree. If the distance metric indicates that the new component simultaneously satisfies the proximity condition with the semantic tag vectors of multiple parent nodes in the functional group layer, and the volume span of the physical bounding box parameter of the new component exceeds the remaining space capacity of any of the multiple parent nodes, the system triggers a node splitting operation, creates a new transitional functional group layer node between the functional group layer and the regional layer, rebinds the new component and some child nodes under the multiple parent nodes to the new transitional functional group layer node, and updates the bidirectional pointer link and the index structure of the multidimensional vector space index tree.

8. The indoor intelligent generation system based on multimodal user intent understanding according to claim 3, characterized in that, The semantic hierarchy partitioner incorporates modal conflict resolution logic when performing cross-modal feature alignment. When there is an angular difference between the spatial orientation represented by the first feature sequence of the text instruction conversion and the perspective vanishing point direction represented by the second feature sequence of the image instruction conversion, the semantic hierarchy partitioner extracts environmental context features from the multimodal interior design instruction. The environmental context features include the location of door and window openings and the distribution features of load-bearing walls. The system calculates the fit scores of the spatial orientation and the perspective vanishing point direction with the environmental context features, respectively. The modal features with higher fit scores are used as dominant features, and the modal features with lower fit scores are used as auxiliary features. The weight distribution of the auxiliary features is corrected based on the dominant features, and the global response features, the local response features, and the detailed response features are regenerated based on the corrected weight distribution.

9. The indoor intelligent generation system based on multimodal user intent understanding according to claim 5, characterized in that, The constraint filtering engine introduces a topology reachability verification mechanism in the retrieval path from the functional group layer to the individual component layer. Before calculating the 3D intersection volume of the coordinate axis aligned bounding box of the candidate node and the octagonal leaf node corresponding to the remaining space capacity, the system generates a virtual navigation mesh in the root node spatial octagon based on the movement layout label of the functional group layer node. The virtual navigation mesh is composed of connected octagonal leaf nodes with passage attributes. When the candidate node is an obstruction type component, the constraint filtering engine simulates adding the physical bounding box parameters of the candidate node to the virtual navigation mesh and runs a pathfinding algorithm to detect the connectivity from the starting entry node to the ending exit node in the virtual navigation mesh. If the connectivity is blocked, the candidate node is directly determined to have spatial out-of-bounds and a culling operation is performed, and the calculation of the 3D intersection volume is no longer performed.

10. The indoor intelligent generation system based on multimodal user intent understanding according to claim 6, characterized in that, Following the global coordinate system transformation stage, a post-processing stage for gap elimination and topology closure is introduced. The system traverses the 3D mesh data of all individual component nodes mapped to the global 3D coordinate system, extracts the contact boundary lines between adjacent individual component nodes, and calculates the set of shortest distances between mesh vertices on the contact boundary lines. When there is a gap distance in the set of shortest distances that is greater than a preset tolerance threshold, the system performs a mesh patch extension operation based on the normal direction of the mesh vertices on both sides of the gap distance. After the extension operation, the system performs Laplacian smoothing on the mesh vertices near the contact boundary lines to eliminate non-manifold edges and singular triangles generated by the extension operation, ensuring that adjacent individual component nodes form a watertight closed structure at the 3D mesh topology level. The post-processed 3D mesh data replaces the initial 3D mesh data in the video memory to complete the final rendering assembly.

Images