A Partitioned 3D Scene Generation Method and System for Irregular Boundary Constraints

By employing functional area planning and local perspective adaptation, the problems of occlusion and semantic consistency in the generation of irregular boundary 3D scenes were solved, achieving high-quality 3D scene generation.

CN123118929APending Publication Date: 2026-11-17SHANDONG UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611593086.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-10-15
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing 3D scene generation methods struggle to adapt to irregular boundaries, resulting in insufficient stability in terms of semantic consistency, boundary adaptability, physical rationality, and visual quality. In particular, they are prone to issues such as content overstepping boundaries, missing regions, and occlusion in complex scenes.

Method used

By functional zoning, irregular rooms are decomposed into multiple semantically clear functional areas, local observation perspectives are adaptively determined, local scene images are generated using image completion, and local 3D object sets are generated through object recognition and segmentation and pose optimization. Finally, these are fused together in the global coordinate system to form a 3D scene model.

Benefits of technology

It improves the ability to adapt to irregular boundaries, reduces the impact of occlusion, enhances the semantic consistency and visual quality of the generated results, and ensures physical rationality and layout coordination.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123118929A_ABST
    Figure CN123118929A_ABST
Patent Text Reader

Abstract

This invention belongs to the fields of 3D computer vision and digital twin technology, and particularly relates to a method and system for generating partitioned 3D scenes with irregular boundary constraints. The method includes: acquiring an indoor space planar structure diagram and user text requirements; dividing an irregular room into multiple functional areas with semantic descriptions and spatial constraints; adaptively selecting the optimal local viewing angle for each area; performing local image completion to generate local scene images from the selected angle; performing object recognition and segmentation on the local scene images; extracting spatial structure data of each object for pose optimization; obtaining a set of local 3D objects through relational constraints; and finally mapping the optimized sets of local 3D objects from each area to a global coordinate system for fusion, outputting a complete 3D indoor scene. This invention effectively adapts to irregular boundaries through partitioning planning, adaptive viewing angles, and relational constraints, ensuring layout rationality and physical legitimacy.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the field of 3D computer vision and digital twin technology, and particularly relates to a method and system for generating partitioned 3D scenes with irregular boundary constraints. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] Automatic generation of 3D scenes is an important research direction in computer graphics and computer vision, with wide applications in game development, film and television production, virtual reality, digital twins, and interior design. High-quality 3D scenes need to be semantically consistent with the user in terms of object types and quantities, and the spatial layout must conform to physical common sense, usage habits, and geometric constraints such as boundaries, doors, and windows. However, traditional 3D scene modeling processes rely heavily on human experience, resulting in long production cycles and high costs, making it difficult to meet the needs of rapid iteration and large-scale generation.

[0004] Among existing methods for automatic generation of 3D scenes, deep learning-based feedforward generation methods directly output 3D scenes by learning the joint distribution of scene layouts. Their advantage lies in end-to-end efficiency, but they are highly dependent on training data and have insufficient generalization ability for irregular boundaries and custom requirements. In addition, the lack of explicit geometric constraints makes it difficult to guarantee physical legality such as no boundary crossings and no collisions.

[0005] Secondly, inference-based generation methods based on pre-trained large models parse text into layout descriptions and then iteratively optimize it. While highly flexible, these methods have limited ability to model precise spatial scales, complex boundaries, and visual aesthetics. When there are many functional areas and complex relationships, they are prone to problems such as unreasonable layouts and insufficient fine-grained control. Furthermore, generation methods using 2D images as intermediate representations, which generate scene images using text-generated graph models and then reconstruct the 3D structure, can improve layout consistency and appearance quality to some extent. However, a single global image cannot consistently satisfy complex spatial constraints, leading to frequent content overflows and missing regions. Additionally, the visibility of distant and corner areas is poor, severely impacting the accuracy of subsequent segmentation and pose estimation.

[0006] Therefore, there is an urgent need for a 3D scene generation technology that can take into account both the visual priors of the image generation model and the geometric constraints of 3D space, so that it can adapt to irregular room boundaries, reduce the impact of local occlusion in complex scenes, and improve the stability of the generated results in terms of semantic consistency, boundary adaptability, physical rationality and visual quality. Summary of the Invention

[0007] To overcome the shortcomings of the existing technologies, this invention provides a partitioned 3D scene generation method and system for irregular boundary constraints. The aim is to decompose an irregular room into multiple semantically defined functional areas through functional area planning, and adaptively select the optimal local viewpoint for each area to reduce occlusion effects. After generating the scene using local image completion, spatial relationship constraints between objects are introduced to optimize pose, ensuring physical validity and layout coordination, ultimately merging to form a complete 3D indoor scene.

[0008] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions: Firstly, a partitioned 3D scene generation method for irregular boundary constraints is disclosed, including: Obtain the floor plan of the interior space and the user's textual requirements; based on the floor plan and the user's textual requirements, perform functional area planning to obtain multiple functional areas with semantic descriptions and spatial constraints; For each functional area, the local observation angle of each functional area is adaptively determined based on the relative relationship between its area range and the boundary structure in the planar structure diagram. Based on the semantic description and spatial constraints, image completion is performed on the corresponding functional area from the local observation perspective to generate a local scene image; The local scene image is subjected to object recognition and segmentation, the spatial structure data of each object is extracted for pose optimization, and a local three-dimensional object set is obtained through relation constraints. The optimized local 3D object sets of each functional area are uniformly mapped to the global coordinate system for scene fusion, generating a 3D scene model diagram.

[0009] Furthermore, the functional area includes the area range, functional semantic description, object list, and description of the spatial relationships between objects or between objects and building structures.

[0010] Furthermore, the adaptive determination of the local observation angle of each functional area includes: setting candidate viewing angles in the four diagonal directions of each functional area; calculating the degree of overlap between each side of the functional area and the exterior wall of the room to obtain the wall coverage ratio; Candidate viewing angles are evaluated from three dimensions: near-end openness, far-end wall stability, and regional visibility. Scores are obtained through a scoring function, and the candidate viewing angle with the highest score is selected as the optimal local observation angle.

[0011] Furthermore, the scoring function is:

[0012] in, Candidate directions; For proximal openness score; Score the stability of the far-end wall; The visibility score is assigned to the region. , , These are the corresponding weighting coefficients.

[0013] Furthermore, image completion of the corresponding functional area under the local observation view specifically includes: rendering a blank local scene image with functional area color labels under the selected view; generating completion prompts based on the semantic description of the functional area, the object list, the spatial relationship description, and the target area color labels; inputting the blank local scene image and the completion prompts into the image completion model to complete the corresponding functional area.

[0014] Furthermore, the extraction of spatial structure data of each object for pose optimization specifically includes: using a visual language model to identify objects and segment them into masks, combining monocular depth estimation and camera parameter back projection to obtain visible point clouds; estimating the initial pose through re-rendering matching, and using object transformation parameters as variables to construct a joint optimization objective to obtain a local 3D object set.

[0015] Furthermore, the joint optimization objective includes point cloud alignment constraints, spatial relationship constraints, interleaving constraints, boundary constraints, and regularization constraints.

[0016] Secondly, a publicly available partitioned 3D scene generation system for irregular boundary constraints includes: The partitioned scene design module is used to obtain the floor plan of the interior space and user text requirements, and to plan functional areas based on the floor plan and user text requirements to obtain multiple functional areas with semantic descriptions and spatial constraints. The perspective determination module is used to adaptively determine the local observation perspective of each functional area based on the relative relationship between its area range and the boundary structure in the planar structure diagram. The image completion module is used to complete the corresponding functional area based on the semantic description and spatial constraints under the local observation perspective, and generate a local scene image. The relation-constrained 3D estimation module is used to perform object recognition and segmentation on the local scene image, extract the spatial structure data of each object for pose optimization, and obtain a local 3D object set through relation constraints. The scene fusion module is used to uniformly map the optimized local 3D object sets of each functional area to the global coordinate system for scene fusion, generating a 3D scene model diagram.

[0017] Thirdly, a computer device is disclosed, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to perform the steps of the method described above.

[0018] Fourthly, a computer-readable storage medium is disclosed having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described above.

[0019] The above one or more technical solutions have the following beneficial effects: This invention utilizes boundary-aware functional area planning and partitioned local scene generation. It can divide irregular boundary spaces into multiple functional areas with clear boundaries and semantics, avoiding problems such as content overstepping boundaries, missing areas, and structural inconsistencies that easily occur when complex scenes are generated as a whole at once. This improves the adaptability to irregular room boundaries such as L-shaped, T-shaped, U-shaped, concave and convex boundaries, and those with corridor connections.

[0020] This invention utilizes adaptive local viewpoint generation for functional areas and region-constrained image completion. It can select a local observation viewpoint with low occlusion and complete field of view for each functional area and generate objects only within the target functional area. This reduces the errors in instance segmentation, depth estimation, and 3D reconstruction caused by insufficient visible information of distant areas, corner areas, and occluded objects in a single global image, thereby improving the stability of the complex scene generation process.

[0021] This invention fully utilizes the visual prior of the image completion model gpt-image-1, while introducing functional area descriptions, a list of main objects, and spatial relationship constraints, so that the generated results can better meet the user's text needs and have better overall performance in terms of object category, quantity, functional area organization, overall style, and visual harmony.

[0022] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0023] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0024] Figure 1 This is a flowchart of a partitioned 3D scene generation method for irregular boundary constraints according to Embodiment 1 of the present invention. Figure 2 This is a comparison chart of the generation results of the partitioned 3D scene generation method for irregular boundary constraints according to Embodiment 1 of the present invention with those of existing 3D scene generation methods on the SceneEval dataset. Figure 2 (a) in the diagram represents the result generated by LayoutGPT; Figure 2 (b) in the table represents the result generated by HOLODECK; Figure 2 (c) in the diagram represents the generated LayoutVLM; Figure 2 (d) in the diagram represents the result generated by SceneWeaver; Figure 2 (e) in the text represents the result generated by ArtiScene; Figure 2 (f) in this embodiment represents the result generated by the method. Figure 3 This is a comparison of the generated images and synthesized 3D scene results of the scene generation method based on a single global image in the partitioned 3D scene generation method for irregular boundary constraints according to Embodiment 1 of the present invention, under rectangular boundary and irregular boundary conditions. Figure 3 (a) shows the generated images and synthesized 3D scenes of ArtiScene's scene generation method based on a single global image under rectangular and irregular boundary conditions; Figure 3 (b) in this embodiment shows the generated images and the synthesized 3D scene generated by the scene generation method based on a single global image under rectangular and irregular boundary conditions; Figure 4 This is a diagram showing the ablation experiment results in the partitioned 3D scene generation method for irregular boundary constraints according to Embodiment 1 of the present invention; Figure 4 (a) in the figure is a 3D estimation display diagram of a scene design without partitions and without relational constraints; Figure 4 (b) is a 3D estimation diagram with relational constraints for a scene design without partitions; Figure 4 (c) in the figure is a 3D estimation display diagram with partitioned scene design and no relational constraints; Figure 4 (d) in the figure is a 3D estimation diagram with partitioned scene design and relational constraints. Detailed Implementation

[0025] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0026] It should be noted that the terminology used herein is for the purpose of describing particular implementations only and is not intended to limit the exemplary implementations of the present invention.

[0027] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0028] Example 1 like Figure 1 As shown, this embodiment discloses a partitioned 3D scene generation method for irregular boundary constraints, including: Step 1: Obtain the floor plan of the interior space and the user's requirements text; based on the floor plan and the user's requirements text, perform functional area planning to obtain multiple functional areas with semantic descriptions and spatial constraints.

[0029] In this embodiment, the input data includes the room's two-dimensional boundary, a set of doors, a set of windows, and user requirement text. The room's two-dimensional boundary is composed of sequentially arranged boundary vertices, with adjacent vertices connected to form the room's outer wall boundary. Doors and windows can both be represented as line segments located on the room's outer boundary. This input format can describe rectangular boundaries, L-shaped boundaries, T-shaped boundaries, U-shaped boundaries, concave and convex boundaries, and complex irregular boundaries connected by corridors.

[0030] Organize room boundaries, door sets, window sets, and user requirement text into a structured description: ,in Represents the two-dimensional boundary of the room. Represents a set of gates. Represents a set of windows. A textual description representing user needs. This includes room boundaries. A sequence of vertices arranged in clockwise order Composition, that is Doors and windows are modeled as line segments on the outer boundary.

[0031] Based on room boundaries, door and window information, and user needs, a partitioned scene design module divides the complex interior space into several semantically clear and well-defined functional areas, serving as the basis for subsequent local image generation and 3D structure reconstruction. Specifically: The structured description is input into the pre-trained language model. Preferably, in this embodiment, the GPT-5.4 large language model is used for functional area planning. The GPT-5.4 large language model first analyzes the functional goals, main object categories, regional relationships, and usage habits included in the user's needs, and then outputs K functional areas. Each functional area Including the area Functional semantic description List of main objects And descriptions of spatial relationships between objects or between objects and building structures. .

[0032] Among them, the area range A two-dimensional rectangular area is used to define the location and size of the functional area in the room plan; Functional semantic description This is used to describe the purpose of the area, such as a reception area, dining area, rest area, storage area, performance area, etc.; a list of main objects. Used to define the core object categories that should appear in this area; spatial relationship description Used to constrain the way objects are placed, such as placing them against a wall, arranging them near a window, preserving passageways, not obstructing doors and windows, or arranging them around a central object.

[0033] During the functional zoning planning process, the following spatial constraints must be met: each functional area should be located within the legal room boundaries; significant area overlap between different functional areas should be avoided; each functional area should cover the main interior space as much as possible; entrance transition spaces, main passageways, window-side lighting areas, and main functional areas should be kept reasonable. A small number of areas not covered by any functional area can be retained as blank transition spaces or public passageways, and will not participate in subsequent area-level object completion. If excessive overlap of functional areas is detected, the functional zoning planning process can be restarted until the overlap area between functional areas is below a preset threshold.

[0034] Through the above steps, this embodiment divides complex and irregular rooms into multiple locally regular and functionally defined generation sub-problems, thereby reducing the difficulty of generating the overall scene at once and improving the controllability of irregular boundaries and functional requirements in the subsequent generation process.

[0035] Step 2: For each functional area, based on the relative relationship between its area range and the boundary structure in the planar structure diagram, adaptively determine the local observation angle of the functional area.

[0036] Given an initial empty scene containing room boundaries, walls, doors, windows, and functional area divisions. In this embodiment, each rectangular functional area Candidate viewpoints are set in the four diagonal directions. This maintains a consistent isometric perspective style.

[0037] To assess the occlusion of the target functional area under different candidate viewpoints, this embodiment quantifies the potential structural occlusion by calculating the degree of overlap between each side of the functional area and the exterior wall of the room.

[0038] set up For the set of exterior wall segments of the room, for the edges of functional areas Define the wall coverage ratio as ,in, .when The larger the value, the more likely the side is to be adjacent to an exterior wall of the room, and the more easily it will be obstructed by the wall when viewed from that side; when... The smaller the value, the more likely the side is to be adjacent to other indoor areas or to be on an open side, making it more suitable as the direction for the camera.

[0039] For each candidate direction Determine the set of near edges based on the corner point where the camera is located. And determine the set of far edges based on the camera orientation. For example, when the candidate direction is At this time, the camera is located on the lower left side of the function area, with its near edge representing the left and front sides, and its far edge representing the right and rear sides. The near edge describes whether the direction in which the camera is located is open, and the far edge describes whether there is a stable wall background in the depth of the image.

[0040] This embodiment evaluates candidate viewpoints from three aspects: near-end openness, far-end wall stability, and regional visibility.

[0041] Proximal side openness This is used to measure whether the side where the camera is located is close to an external wall or obstructing structure. When the near-end wall has less coverage, this direction is more suitable as the camera's viewing direction, which helps to reduce the wall's obstruction of the target functional area. Its specific definition is:

[0042] in, For indexing functional areas; The index of the functional area edge; Candidate directions; Indicates the first The first functional area Edge; Candidate observation directions The corresponding set of near edges; This represents the number of edges contained in the near-end edge set; Indicates functional area edge The wall coverage ratio with the exterior walls of the room; Indicate candidate direction The corresponding proximal openness score.

[0043] Far-end wall stability This is used to measure whether the camera, when facing away from the room, has a clearly defined room boundary or wall background. When the far-end boundary is adjacent to or coincides with an exterior wall, the spatial boundary in the local image is more clearly defined, which helps prevent the image completion content from spreading outwards from the room. Specifically, it is defined as:

[0044] in, Indicates candidate observation direction The corresponding set of far edges; This indicates the number of edges contained in the far edge set; Indicates candidate observation direction The corresponding stability score of the far-end wall.

[0045] Area visibility Used to measure the current candidate perspective The visibility of the target functional area. A higher visibility indicates that more target areas can be observed from that viewpoint, which is beneficial for subsequent image completion, instance segmentation, depth estimation, and 3D reconstruction. It is defined as:

[0046] in, Indicates the first Each functional area; Indicates the direction of observation from the candidate Functional areas during observation The visible part; Indicates the area of ​​the region; Indicates candidate observation direction The corresponding area visibility rate.

[0047] Taking into account the above factors, each candidate perspective is scored using the following scoring function: ,in, , , The candidate viewpoint with the highest score is selected as the optimal local observation viewpoint for that functional area.

[0048] This embodiment calculates the overlap between the functional area edges and the exterior walls, quantifies occlusion in each candidate direction, and adaptively selects the optimal equidistant viewing angle by combining a comprehensive score of near-end openness, far-end wall stability, and regional visibility. This mechanism prioritizes open directions, effectively avoiding wall occlusion and ensuring the target area is fully observed; at the same time, it ensures clear far-end boundaries, preventing image completion from spreading outdoors. By decomposing the viewing angle decision of a complex room into independent local viewing angles for each functional area, the difficulty of global occlusion processing is reduced, providing reliable observation conditions with low occlusion and clear boundaries for high-quality image completion and 3D reconstruction of each area independently.

[0049] Step 3: Based on the semantic description and spatial constraints, perform image completion on the corresponding functional area from the local observation perspective to generate a local scene image.

[0050] After determining the camera orientation of the optimal local observation angle, the orthogonal projection range is adaptively set according to the size of the functional area, so that the target functional area is fully entered into the local image field of view, and sufficient space is reserved for the generation of subsequent objects.

[0051] To further alleviate wall occlusion, when rendering a partial empty scene image, the near-end walls, doors, windows, or structural elements that obscure the target functional area can be temporarily hidden. This hiding operation only changes the visibility in the current view and does not change the true geometry of the global scene, thus obtaining an unobstructed empty scene of the target area without disrupting the overall spatial relationships of the room.

[0052] Subsequently, a blank local scene image with functional area color markers is rendered from the selected local viewpoint. Before rendering, assign easily distinguishable floor identification colors to different functional areas based on the functional area planning results. It also records the mapping relationship between functional area identifiers and corresponding colors. The target area color identifier is automatically generated by the functional area planning results, without the need for additional user input.

[0053] Based on the blank partial scene image of the current functional area, the functional semantic description, the list of main objects, the spatial relationship description, and the color markers of the target area, generate a region-level image completion prompt. , denoted as:

[0054] in, This indicates the process of generating region-level image completion prompts; Subsequently, the blank local scene image and the region-level image completion prompt are input into the image completion model. In this embodiment, the image completion model adopts the gpt-image-1 model, which determines the functional region to be completed based on the target region color identifier, and performs image completion on the blank local scene image to generate the corresponding local scene image, denoted as:

[0055] During image completion, the following constraints must be met: only edit the current target functional area; keep walls, doors, windows, boundary lines, and adjacent areas unchanged; newly added objects must be completely located within the current functional area; do not block doorways, obstruct windows, or block main passageways; minimize severe occlusion between objects within the area; and maintain consistency in scale and style between the local scene and the global room structure.

[0056] The above constraints are described using region-level image completion prompts, which are then input into the image completion model as textual conditions. Specifically, the prompts clearly define the color identifiers of the target functional areas and require the model to add objects only within these areas, while keeping walls, doors, windows, boundary lines, adjacent areas, and camera viewpoints unchanged. Based on the functional semantic description, the list of main objects, and the spatial relationship description, constraints are imposed on the category, quantity, and layout of the generated objects. Newly added objects are required not to extend beyond the target functional area, block doorways, obstruct windows, or block main passageways, and severe occlusion between objects should be minimized.

[0057] The region-level image completion prompts are generated according to the aforementioned formula and input into the image completion model along with the blank local scene image to complete the local scene image generation. It should be noted that the aforementioned constraints are not numerically optimized through an explicit loss function during the image completion stage; instead, they are encoded as region-level image completion prompts and input as conditional constraints into the image completion model. Explicitly computable physical constraints such as object boundary crossings and intersecting are further constrained through corresponding loss functions in the subsequent 3D pose optimization stage.

[0058] Through the above steps, this embodiment can utilize the visual prior of the image completion model gpt-image-1 to generate local scene images with natural layout, harmonious style and low occlusion in local areas, and avoid problems such as missing distant areas, invisible corner areas and severe object occlusion that are prone to occur in complex boundary scenes by a single global image.

[0059] Step 4: Perform object recognition and segmentation on the local scene image, extract the spatial structure data of each object for pose optimization, and obtain a local 3D object set through relation constraints.

[0060] Based on each completed local image The structure of the three-dimensional objects within the functional area is restored, and the position, orientation, and scale of the objects are optimized through relational constraints.

[0061] First, for the completed local image, this embodiment uses the Qwen3-VL-30B visual language model to identify the set of objects in the image. This yields the category names of each object. Based on the identified object categories, the SAM3 model is used to perform instance segmentation on the image, obtaining the instance mask for each object. The Depth Anything V2 monocular depth estimation model was used to obtain the depth map corresponding to the local image. Combined with the camera parameters of this local image The pixels within each object's mask region are back-projected into a spatial point cloud, thus obtaining the visible point cloud of the object in the local scene coordinate system. Simultaneously, using SAM3D-Objects to generate 3D models, a corresponding standard spatial 3D model is generated for each identified object. .

[0062] For each object, this embodiment needs to estimate the transformation parameters from the normalized model space to the local scene space. This includes object rotation, scale, and translation. Considering that most furniture and objects in indoor scenes rotate primarily around a vertical axis, this embodiment simplifies object rotation to an orientation angle around the vertical axis. ,Right now This reduces the complexity of pose estimation.

[0063] To obtain a stable initial pose, this embodiment employs a re-rendering matching strategy to estimate the object's orientation and scale. Specifically, for each object's 3D model, multiple candidate angles for rotation around the vertical axis are constructed, and the candidate results are rendered in the corresponding local camera view.

[0064] Then, MASt3R is used to compare the candidate rendered image with the object regions in the original local image. Perform feature point matching and select the candidate angle with the highest matching score as the initial orientation of the object: .

[0065] After determining the initial orientation, the object scaling factor is estimated based on the projection scale relationship of the matching points. For heavily occluded objects, a priori category size can be incorporated to avoid underestimating the object scale based solely on the locally visible region. The initial center position of an object can be determined by the center of the bounding box of its visible point cloud. This yields the initial transformation for each object. .

[0066] Since the initial pose may be affected by image occlusion, depth error, and model geometric differences, problems such as positional shift, inaccurate scale, incorrect support relationships, or object interpenetration may still occur. Therefore, this embodiment further classifies the furniture to be optimized into non-severely occluded furniture and severely occluded furniture according to the degree of occlusion of objects in the local scene image, and adopts different joint optimization constraints for different degrees of occlusion.

[0067] Under relational constraints, this embodiment uses the transformation parameters of the furniture object to be optimized in the local scene as the optimization variables. For the first... The orientation of the furniture object has been determined through the aforementioned MASt3R re-rendering matching, so the main adjustment in the joint optimization phase is to adjust the horizontal position and scale of the object.

[0068] Before optimization, furniture was categorized into non-severely occluded furniture and severely occluded furniture based on their visibility in the local scene image. Utilizing the prior knowledge of the visual large language model GPT-5.4, the ground truth mask and the original image were input into GPT-5.4 to determine whether the mask was severely occluded by other objects in the image. For non-severely occluded furniture, since the visible area of ​​the object was relatively complete, the point cloud obtained by backprojecting the depth map could accurately reflect the object's position and size. Therefore, constraints were applied using MASt3R feature matching, point cloud center, and point cloud size. For severely occluded furniture, since its visible point cloud only corresponds to a part of the object, estimating the complete object's center and size using the residual point cloud could easily produce large errors. Therefore, the constraints on point cloud center and size were removed, the weight of the MASt3R feature matching constraint was increased, and further constraints were applied using visible instance masks and occlusion depth order.

[0069] Constructing a joint loss function By finding The final optimization result is obtained by minimizing the object's position and scale. The joint loss function is:

[0070] in, This indicates point cloud alignment constraints. Indicates spatial relationship constraints. Indicates interleaving constraints. Indicates boundary constraints. This represents a regularization constraint.

[0071] Point cloud alignment constraints are used to ensure that the optimized 3D object is consistent with the spatial observation results obtained from the local scene image. This embodiment utilizes both MASt3R feature point matching results and the visible point cloud obtained from monocular depth backprojection to constrain the object's position and scale, as expressed below:

[0072] in, The positional error between the matching points on the 3D model, after being reprojected onto the local scene image, and the corresponding matching points obtained by MASt3R, can be expressed as:

[0073] in, Indicates the first The first object Matching points for MASt3R images This indicates the position of the corresponding 3D model point reprojected into the image after the current transformation. This indicates the confidence level of the matching point.

[0074] set up and They represent the optimized i-th The center and dimensions of the bounding box of an object. and Let represent the center and size of the reference bounding box obtained from the visible point cloud, respectively. Then, the constraints of the bounding box center and size are expressed as follows:

[0075]

[0076] in, and These represent the weights of the center position constraint and the size constraint, respectively. For objects that are not severely occluded, and The value is 0.9. For severely occluded objects, since the point cloud can only reflect the locally visible part of the object, the center position and size constraints are removed, and the MASt3R matching constraint weights are increased. The point cloud alignment constraint is expressed as follows:

[0077] Spatial relationship constraints are mainly used to constrain the visible area of ​​severely occluded objects and their positional relationship with other occluded objects in front of them. These constraints include visible instance mask constraints and occlusion depth order constraints.

[0078] in, This represents the visible instance mask error. Let the visible mask of the re-rendered object be... The object instance mask in the original local scene image is First, calculate the crossover-union ratio of the two:

[0079] The instance mask constraint is represented as follows:

[0080] in, This represents the difference in the outlines of two masks. This indicates the difference in the center positions of the two masks. This represents the area difference between the two masks.

[0081] For two objects that are occluded, let the depth of the occluded object be . The depth of the object obstructing the view in front is Then the occlusion depth order constraint is expressed as:

[0082] This constraint ensures that the occluded object is positioned behind the occluding object, while maintaining a certain front-to-back distance.

[0083] Intersection constraints are used to penalize unreasonable 3D intersections and collisions between different objects; let the 3D bounding boxes of the two objects be respectively... and First, calculate the relative overlap ratio between the two. The interleaving constraint is expressed as:

[0084] in This represents the relative overlap ratio between the two objects, calculated by dividing the overlap volume of the two bounding boxes by the volume of the smaller bounding box. A value of 0.01 indicates that a small amount of slight overlap is allowed between the objects. No penalty is incurred when the overlap ratio does not exceed 1%. Beyond 1%, the more severe the overlap, the greater the penalty.

[0085] Boundary constraints are used to ensure that the object's ground projection does not exceed the boundaries of the current functional area or room; for the first For each of the following objects, calculate the distances it extends beyond the allowed area to the left, right, forward, and backward, and record the total out-of-bounds distance as . :

[0086] Boundary constraints are expressed as:

[0087] When the object is completely within the allowed space. =0; the greater the distance the object extends beyond the allowed area, the greater the corresponding boundary loss.

[0088] Regularization constraints are used to prevent unreasonable large deviations in the optimization result relative to the initial pose. Let the initial scale of the object before optimization be... The optimized scale is The scaling error is expressed as:

[0089] For objects with minor occlusion, the regularization weight is set to 0.15; for objects with severe occlusion, the regularization weight is set to 0.20. Therefore:

[0090] in, This represents a set of objects that are not severely occluded. This indicates a set of objects that are severely obscured.

[0091] By minimizing the joint loss function L, the horizontal position and scale of each object in the local scene are iteratively adjusted. For non-severely occluded objects, optimization is mainly achieved using MASt3R feature matching and the position and size information of the depth point cloud; for severely occluded objects, optimization is mainly achieved using MASt3R feature matching, visible instance masking, and occlusion depth order; at the same time, interleaving constraints, boundary constraints, and regularization constraints are used to reduce object collisions, out-of-bounds errors, and abnormal scale changes, thereby obtaining an optimized set of local 3D objects.

[0092] Step 5: Map the optimized local 3D object sets of each functional area to the global coordinate system for scene fusion and generate a 3D scene model.

[0093] The recovered local 3D object sets from each functional area are mapped back to the global room coordinate system to generate a complete 3D interior scene. Specifically: After completing the local 3D reconstruction and relational constraint optimization of each functional area, this embodiment transforms the objects in each local coordinate system into a unified global coordinate system based on the functional area planning results and local camera parameters. For each object, its category label, 3D model, spatial location, orientation, and scale are retained, and its final position in the global room is determined according to its functional area.

[0094] The final output is a complete 3D indoor scene. This 3D indoor scene includes several 3D objects, each with a corresponding category label, 3D model, spatial location, pose orientation, and scale parameters. Through these objects and their spatial relationships, a complete 3D scene that meets user needs can be formed, and can be further used in applications such as virtual reality, digital twins, interior design, game development, and embodied intelligent simulation.

[0095] Through the above steps, this embodiment unifies functional area planning, local image generation, object-level 3D reconstruction, relational constraint pose optimization, and global fusion in complex boundary space into the same framework, realizing the automatic generation of editable, constrained, and structurally complete 3D scenes.

[0096] Specific application examples: To verify the effectiveness of the method in this embodiment, experiments were conducted using both conventional rectangular boundary data and irregular boundary data.

[0097] In the experiments with standard rectangular boundaries, the SceneEval dataset was used as the testing basis. Since the original format of this dataset is primarily geared towards text-driven 3D scene evaluation and lacks explicit room boundary inputs, the examples supplement the test scenes with rectangular room boundaries and basic door and window information to evaluate the 3D scene generation quality of different methods under standard room conditions.

[0098] In the irregular boundary experiment, a test set of irregular boundary scenarios with various boundary shapes was constructed. This test set includes multiple test examples with boundary constraints, and the boundary types include rectangular boundaries as well as random irregular boundaries such as L-shaped, T-shaped, and Z-shaped boundaries, which are used to verify the adaptability of the present invention under complex boundary conditions.

[0099] In practical implementation, functional area planning and regional image generation can be accomplished using pre-trained language models and image completion models; instance segmentation, depth estimation, and object-level 3D model generation can be accomplished using instance segmentation models, monocular depth estimation models, and object-level image-to-3D model generation models, respectively.

[0100] The evaluation metrics include three aspects: semantic consistency, physical feasibility, and scene realism. Semantic consistency metrics include object quantity satisfaction rate and object relationship satisfaction rate, used to evaluate whether the generated result meets the input text requirements and the spatial relationships between objects or between objects and building structures. Physical feasibility metrics include object boundary crossing rate and object collision rate, used to evaluate whether generated objects are located within legal areas and whether unreasonable collisions occur between objects. Scene realism metrics include layout rationality score and functional integrity score, used to evaluate the overall visual appeal, spatial harmony, accessibility, and functional completeness of the generated scene.

[0101] Comparison experiment on generating 3D scenes with rectangular boundaries: like Figure 2 As shown, in the rectangular boundary experiment, the method in this embodiment is compared with existing 3D scene generation methods. The comparison methods include the layout generation method based on a large language model (LayoutGPT), the 3D scene generation method based on an optimizer (HOLODECK), the layout reasoning method based on a visual language model (LayoutVLM), the reflective agent scene generation method (SceneWeaver), and the 3D scene generation method based on intermediate representation of 2D images (Artiscene).

[0102] In the rectangular boundary experiment, the following input was given to each model: "A living room with a TV, sofa, and bookshelves. There is no coffee table in the room."

[0103] "This is a bedroom. A bunk bed is placed next to the window, decorated with plush toys. There are two desks against the wall, each with a chair. More than a dozen plush toys are scattered on the desks, adding a touch of playfulness and warmth to the space."

[0104] "The dim lighting in the room creates a somber atmosphere in this warm and beautifully furnished living room. A dining table sits in the center of the room, with four wooden chairs neatly arranged around it… Despite the disturbing news broadcast on the television, the room seems to reflect the occupant's taste for a cozy and pleasant environment. The juxtaposition of the tranquil living room and the chaotic scenes and screams on the television screen creates a tense and unsettling atmosphere."

[0105] "A living room with a huge wall-mounted screen, several chairs in front of it, and a series of board games on a shelf in the corner."

[0106] in, Figure 2 (a) shows the generated results of LayoutGPT from top to bottom; Figure 2 (b) shows the generation results of HOLODECK from top to bottom; Figure 2 (c) in the diagram shows the generated results of LayoutVLM from top to bottom; Figure 2 (d) in the image above and below shows the results generated by SceneWeaver. Figure 2 (e) in the image shows the results generated by ArtiScene from top to bottom; Figure 2 The (f) in the figure, from top to bottom, represents the results generated by the method in this embodiment.

[0107] Experimental results are as follows Figure 2 Compared with Table 1, the method (Ours) in this embodiment achieves better overall performance in terms of semantic consistency, physical feasibility and visual quality, and the generated scene has a more natural layout, more reasonable functional organization and better overall visual effect.

[0108] Table 1 compares the proposed method with other methods for 3D scenes with rectangular boundaries on the SceneEval dataset.

[0109]

[0110] Comparative experiment on the generation of 3D scenes with irregular boundaries: In the irregular boundary experiment, the same text requirement was applied to both rectangular boundary and irregular boundary scenarios.

[0111] The first line of input text reads: "This space comprises three rooms. The spacious living room features an L-shaped sectional sofa, opposite which is a wall-mounted TV, and in front of the sofa is a rectangular coffee table with a tray and napkins on it. The dining room has a retractable dining table surrounded by several comfortable chairs, and a stylish sideboard with glasses and soda. Two wall lamps provide soft lighting, and a small serving cart in the corner is piled with wine bottles. The modern kitchen has a central island with four bar stools on one side. A deli platter sits on the island, surrounded by several plates, and a wine cabinet is in the corner."

[0112] The second line of input text reads: "In this cozy bedroom, a full-size bed is placed against the wall, with a bedside table on each side. A delicate little alarm clock sits on the left bedside table. In the corner, there's a small desk and a comfortable chair for studying or working. Opposite the bed, there's a spacious dressing table providing extra storage space. Next to the bedroom is a living room with a recliner and a footstool facing a low coffee table, on which sits a small vase and some magazines. In the corner is a large bookshelf filled with books and board games, while a large sofa provides ample space for gatherings. Next to the living room is a small game room specifically designed for entertainment, containing a game console, two beanbag chairs, and a small shelf for storing controllers and headphones."

[0113] like Figure 3 As shown in (a), ArtiScene, a scene generation method based on a single global image, is prone to problems such as content exceeding boundaries, missing local regions, severe object occlusion, and unstable 3D reconstruction when faced with irregular boundaries. In contrast, as... Figure 3 As shown in (b) in this embodiment, the method decomposes the complex scene into multiple local generation tasks by dividing the functional area and generating local perspectives. This can better adapt to irregular boundary constraints and generate a more complete 3D scene with more reasonable object relationships.

[0114] To verify the functions of each key module in this invention, ablation experiments were conducted on the partitioned scene design module and the relation-constrained 3D structure estimation module. Experimental results show that without partitioned scene design, the generated results are more susceptible to occlusion from a single global image and insufficient boundary control, leading to problems such as object boundary violations, missing regions, and unclear functional organization. After introducing partitioned scene design, the semantic coverage and local image visibility of functional regions are improved, object boundary violations are significantly reduced, and the overall scene quality is improved.

[0115] Furthermore, after introducing the relation-constrained 3D structure estimation module, the estimation of the position, orientation, and scale of small objects and occluded objects in the scene is more accurate, the support relationships, proximity relationships, and boundary relationships between objects are more reasonable, and object interpenetration and local pose errors are alleviated. When both the partitioned scene design module and the relation-constrained 3D structure estimation module are used simultaneously, the system can reduce boundary crossing and spatial conflict problems while maintaining high semantic consistency, ultimately obtaining a 3D scene that better meets irregular boundary constraints and user needs. The results are as follows: Figure 4 As shown in Table 2: Table 2 Ablation Experiments; OC represents the relation-constrained 3D structure estimation module, and VP represents the partitioned scene design module.

[0116] Example 2 Based on the method described in Embodiment 1, the purpose of this embodiment is to provide a partitioned 3D scene generation system oriented towards irregular boundary constraints, including: The partitioned scene design module is used to obtain the floor plan of the interior space and user text requirements, and to plan functional areas based on the floor plan and user text requirements to obtain multiple functional areas with semantic descriptions and spatial constraints. The perspective determination module is used to adaptively determine the local observation perspective of each functional area based on the relative relationship between its area range and the boundary structure in the planar structure diagram. The image completion module is used to complete the corresponding functional area based on the semantic description and spatial constraints under the local observation perspective, and generate a local scene image. The relation-constrained 3D estimation module is used to perform object recognition and segmentation on the local scene image, extract the spatial structure data of each object for pose optimization, and obtain a local 3D object set through relation constraints. The scene fusion module is used to uniformly map the optimized local 3D object sets of each functional area to the global coordinate system for scene fusion, generating a 3D scene model diagram.

[0117] Example 3 The purpose of this embodiment is to provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the above-described method.

[0118] Example 4 The purpose of this embodiment is to provide a computer-readable storage medium.

[0119] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the steps of the above method.

[0120] The steps and methods involved in the devices of the above embodiments correspond to those in Embodiment 1. For specific implementation details, please refer to the relevant description section of Embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood as including any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and enabling the processor to perform any of the methods in this invention.

[0121] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computer devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computer device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. The present invention is not limited to any particular combination of hardware and software.

[0122] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A partitioned 3D scene generation method for irregular boundary constraints, characterized in that, include: Obtain the floor plan of the interior space and the user's textual requirements; Based on the aforementioned planar structure diagram and user text requirements, functional area planning is performed to obtain multiple functional areas with semantic descriptions and spatial constraints. For each functional area, the local observation angle of each functional area is adaptively determined based on the relative relationship between its area range and the boundary structure in the planar structure diagram. Based on the semantic description and spatial constraints, image completion is performed on the corresponding functional area from the local observation perspective to generate a local scene image; The local scene image is subjected to object recognition and segmentation, the spatial structure data of each object is extracted for pose optimization, and a local three-dimensional object set is obtained through relation constraints. The optimized local 3D object sets of each functional area are uniformly mapped to the global coordinate system for scene fusion, generating a 3D scene model diagram.

2. The partitioned 3D scene generation method for irregular boundary constraints as described in claim 1, characterized in that, The functional area includes the area range, functional semantic description, object list, and description of the spatial relationships between objects or between objects and building structures.

3. The partitioned 3D scene generation method for irregular boundary constraints as described in claim 1, characterized in that, The adaptive determination of the local observation angle of each functional area includes: setting candidate viewing angles in the four diagonal directions of each functional area; calculating the degree of overlap between each side of the functional area and the exterior wall of the room to obtain the wall coverage ratio; Candidate viewing angles are evaluated from three dimensions: near-end openness, far-end wall stability, and regional visibility. Scores are obtained through a scoring function, and the candidate viewing angle with the highest score is selected as the optimal local observation angle.

4. The partitioned 3D scene generation method for irregular boundary constraints as described in claim 3, characterized in that, The scoring function is: in, Candidate directions; For proximal openness score; Score the stability of the far-end wall; The visibility score is assigned to the region. , , These are the corresponding weighting coefficients.

5. The partitioned 3D scene generation method for irregular boundary constraints as described in claim 1, characterized in that, The image completion of the corresponding functional area under the local observation view specifically includes: rendering a blank local scene image with functional area color labels under the selected view; generating completion prompts based on the semantic description of the functional area, the object list, the spatial relationship description and the target area color labels; inputting the blank local scene image and the completion prompts into the image completion model to complete the image of the corresponding functional area.

6. The partitioned 3D scene generation method for irregular boundary constraints as described in claim 1, characterized in that, The extraction of spatial structure data of each object for pose optimization specifically includes: using a visual language model to identify objects and segment them into masks, combining monocular depth estimation and camera parameter back projection to obtain visible point clouds; estimating the initial pose through re-rendering matching, and using object transformation parameters as variables to construct a joint optimization objective to obtain a local 3D object set.

7. The partitioned 3D scene generation method for irregular boundary constraints as described in claim 6, characterized in that, The joint optimization objectives include point cloud alignment constraints, spatial relationship constraints, interleaving constraints, boundary constraints, and regularization constraints.

8. A partitioned 3D scene generation system for irregular boundary constraints, characterized in that, include: The partitioned scene design module is used to obtain the floor plan of the interior space and user text requirements, and to plan functional areas based on the floor plan and user text requirements to obtain multiple functional areas with semantic descriptions and spatial constraints. The perspective determination module is used to adaptively determine the local observation perspective of each functional area based on the relative relationship between its area range and the boundary structure in the planar structure diagram. The image completion module is used to complete the corresponding functional area based on the semantic description and spatial constraints under the local observation perspective, and generate a local scene image. The relation-constrained 3D estimation module is used to perform object recognition and segmentation on the local scene image, extract the spatial structure data of each object for pose optimization, and obtain a local 3D object set through relation constraints. The scene fusion module is used to uniformly map the optimized local 3D object sets of each functional area to the global coordinate system for scene fusion, generating a 3D scene model diagram.

9. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the partitioned 3D scene generation method for irregular boundary constraints as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it performs the steps of the partitioned 3D scene generation method for irregular boundary constraints as described in any one of claims 1-7.