An indoor three-dimensional scene data automatic construction method and system for interactive tasks

Abstract

This invention relates to an automatic construction method and system for indoor 3D scene data oriented towards interactive tasks, belonging to the field of computer vision and virtual simulation. Addressing the problems of limited scale, insufficient layout realism, missing interactive attributes, and reliance on manual construction in existing indoor 3D scene datasets, this invention proposes an automatic generation method based on multi-source indoor structural information. This method obtains room topology and functional zoning by parsing floor plans, 3D structural data, or scanned models; it generates a furniture candidate set by combining room type and statistical layout priors, and uses multi-constraint optimization to achieve reasonable placement, ensuring functionality, accessibility, and geometric consistency. Simultaneously, it evaluates and optimizes through collision detection, reachability analysis, and path connectivity verification. This method can efficiently construct large-scale, semantically complete, layout-realistic indoor 3D scene data that supports complex interactive tasks, and is suitable for scenarios such as embodied agent training.

Description

Technical Field

[0001] This invention relates to an automatic construction method and system for indoor 3D scene data oriented towards interactive tasks, specifically to a method and system that can automatically generate large-scale 3D indoor scene data with realistic layout features and interactive attributes from multi-source structural information, belonging to the fields of computer vision, virtual simulation and human-computer interaction technology. Background Technology

[0002] In recent years, with the rapid development of technologies such as virtual reality (VR / AR), intelligent robots, embodied agents, and 3D visual understanding, the demand for indoor 3D scene data with realistic layout, rich semantics, and interactive capabilities has been continuously increasing. In indoor environments, intelligent agents need to perform various interactive tasks such as navigation, transportation, opening and closing, grasping, and organizing in complex spaces. Therefore, high-quality indoor 3D datasets play a crucial role in the training and evaluation of intelligent systems.

[0003] Traditional indoor 3D scene data primarily comes from scanning reconstruction or manual modeling. While scanned data retains the geometric information of the real scene, it is prone to missing, distorted, or incomplete structures due to limitations such as equipment noise, occlusion, and spatial scale. Furthermore, because furniture is in a fixed state during real-world photography, scanned data is essentially static, lacking interactive attributes such as "drawer opening and closing," "door rotation," and "table and chair movement," thus failing to meet the needs of intelligent agents for action planning and interactive operations in dynamic environments. On the other hand, while manual modeling can generate geometrically complete models, it is costly to build, relies heavily on specialized personnel, and is difficult to scale.

[0004] To address the issue of insufficient data scale, some simulation platforms have attempted to generate indoor scenes using random placement. However, due to the lack of real-life spatial arrangement patterns (such as beds typically being placed against walls, tables and chairs appearing in combination, and furniture requiring passageways), these scenes are usually unrealistic in layout, limiting their performance in high-level semantic reasoning and embodied interaction tasks. Furthermore, without reasonable physical constraints, randomly generated furniture positions often suffer from collisions, mutual occlusion, and insufficient accessibility, further reducing their usability.

[0005] In real-world indoor environments, interactive objects are ubiquitous, such as cabinets, doors, windows, drawers, and movable objects. These objects possess interactive features including joint constraints, open / closed states, and multifunctional components. However, most existing datasets do not model these interactive attributes, preventing agents from learning the behavioral patterns of manipulable objects in real life. Furthermore, existing datasets still lack semantic understanding of scene hierarchy, object functional categories, and interactive point annotations, hindering the system's ability to model object attachment relationships and functional paths, thus limiting its application in complex tasks such as "finding and opening the refrigerator" or "organizing the desk."

[0006] In summary, existing 3D indoor scene datasets suffer from varying degrees of deficiency in terms of scale, layout rationality, functional attribute completeness, and interactive information modeling, failing to meet the training requirements of multi-task, multimodal, and complex interactive scenarios. Especially against the backdrop of the rapid development of embodied intelligence and interactive AI models, there is an urgent need for a method and system capable of automatically constructing indoor 3D scene data with realistic layout, complete semantics, rich interactive attributes, and physical consistency. This would fill the gaps in existing technologies and support agents in learning and reasoning in more realistic and operable environments. Summary of the Invention

[0007] The purpose of this invention is to overcome the shortcomings of existing technologies and creatively propose an automatic construction method and system for indoor 3D scene data oriented towards interactive tasks. This invention fills the gap in the current lack of indoor 3D scene data and methods with realistic layout patterns, complete semantic structures, and interactive attributes, while achieving high automation efficiency without requiring extensive manual modeling.

[0008] The innovation of this invention lies in its proposed large-scale automatic construction scheme for indoor 3D scenes, generating 3D indoor environment data with realistic layouts, complete semantic structures, and rich interactive objects. By parsing indoor structural information, we automatically identify room types and spatial relationships, generate a candidate list of furniture based on the distribution patterns of real-life scenes, and then automatically place furniture and objects using layout optimization strategies. This allows for the acquisition of high-quality indoor environments with realistic scene distributions without human intervention. Furthermore, this invention incorporates a functional attribute modeling mechanism in the object construction stage. By constructing interactive attributes such as opening and closing joints, rotating parts, and movable characteristics for furniture and everyday objects, the automatically generated scenes not only contain static geometric structures but also support complex intelligent agent interactions. Since it eliminates the need for manual modeling of each item or manual annotation of interactive information, this invention can efficiently construct large-scale, interactive, and semantically consistent indoor 3D scene data.

[0009] Furthermore, this invention includes a novel method for automatic layout and interaction modeling of indoor scenes. This method utilizes semantic relationships, natural layout patterns, and spatial accessibility constraints within the scene, combined with a layout generation mechanism based on multi-constraint optimization. To better ensure the physical rationality of the scene, we also incorporate collision detection, ergonomic accessibility analysis, and path traversability verification modules, ensuring that the generated scene geometrically conforms to the layout distribution of the real environment. Regarding the construction of interactive attributes, we automatically identify the functional categories and structural features of objects and embed information such as joint structures, handle positions, and open / closed states into the 3D model, enabling the generated scene to meet the interactive characteristics required by the intelligent agent when performing tasks such as grasping, moving, rotating, and switching. Experimental results demonstrate that the data construction method proposed in this invention has significant advantages in layout rationality, interaction richness, and physical consistency, effectively improving the generalization ability and operational performance of the intelligent agent in complex indoor tasks.

[0010] To achieve the above objectives, the present invention adopts the following technical solution:

[0011] An automatic construction method and system for indoor 3D scene data for interactive tasks includes two stages: structural analysis and scene generation. Specifically, it includes the following steps:

[0012] Step 1: Data building phase.

[0013] Specifically, step 1 includes the following steps:

[0014] Step 1.1: Collect multi-source indoor structural information.

[0015] The structural information can include various sources such as publicly available interior floor plan data, 3D scan data, apartment structure data, CAD interior modeling files, and sample images of interior layouts from the internet, to ensure that the collected structural types and spatial topology are consistent with real-life scenarios, thereby providing sufficient basic information for subsequent automatic scene construction.

[0016] Step 1.2: Analyze the collected structural information and establish a room space framework.

[0017] When processing structural data, to ensure that the analytical results do not introduce significant geometric errors in spatial layout, precise geometric analysis of the structural data is required, including wall locations, door and window positions, and functional area divisions. During the analysis process, the room area ratios, opening directions, and spatial connectivity are calibrated to ensure that the generated spatial structure maintains topological consistency with the actual interior environment. Furthermore, in establishing the spatial framework, the input data needs to be scaled and aligned to ensure that subsequent furniture and objects can be automatically constructed within the same spatial system. All data analysis processes are completed in a controlled environment to guarantee the accuracy and consistency of the structural information.

[0018] Step 2: Scene generation stage.

[0019] Using the interior structural framework data obtained in step 1, furniture layout, semantic object generation, and interactive attribute construction in the scene can be automatically generated in this step.

[0020] Specifically, step 2 includes the following steps:

[0021] Step 2.1: Perform semantic mapping on the input spatial framework and generate a set of candidate objects.

[0022] The input spatial structure is defined as a three-dimensional region of size R × C × S. Here, R and C represent the length and width of the space, respectively, and S represents the semantic division of the spatial hierarchy. To map the spatial region to specific room types, predefined layout rules and a semantic classification model are first applied to extract low-level structural features from each spatial block. By stacking and matching these features, the functional label of the overall room can be obtained. At this point, the space is mapped to a high-dimensional semantic space containing several candidate furniture categories and arrangement combinations.

[0023] Step 2.2: After completing the spatial semantic embedding, perform layout generation and interaction modeling on the overall space.

[0024] All spatial data is sequentially processed through K layout planning layers and an object composition layer. Each planning layer contains M layout constraint calculation modules, which aggregate the spatial prior relationships between room functions and furniture. Then, according to the multi-level layout structure, the initial positions of the furniture are searched using a joint optimization strategy of scale scaling and layout constraints, and the number and placement density of objects are dynamically adjusted. For the sake of layout symmetry and scene integrity, the generation stage also includes K object placement calibration layers and a spatial expansion layer, which complete the final arrangement through spatial occupancy inference and accessibility inference. At the same time, to supplement the local geometric information that may be lost during the layout search process, the system will integrate the structural analysis results of the previous stage to form a stable spatial arrangement. Then, the output of the layout generation layer is converted into renderable scene fragments.

[0025] Step 2.3: Finally, interactive attributes are embedded into the generated multi-object scene to construct an operable 3D environment.

[0026] At this stage, the system analyzes the functional characteristics of furniture and objects, automatically adding parameters such as rotational joints, translational joints, handle positions, and state switching to handle the interaction needs of the intelligent agent during task execution. The system further fuses these attributes to ensure the generated scene supports multiple interactions and meets task feasibility requirements.

[0027] The usage method of the above system is as follows:

[0028] Step 1: Collect indoor space data with real structural features, and identify and reconstruct the spatial structure through a unified analysis module;

[0029] Step 2: After completing the analysis and calibration of the spatial structure, the layout generation module automatically generates furniture placement schemes for different room types and constructs corresponding semantic object sets to complete the overall scene construction;

[0030] Step 3: After completing the geometric construction of the scene, perform functional analysis on each type of object in the scene and map them to a unified interactive attribute template space to further construct an interactive object set that can support multi-task learning.

[0031] Step 4: Reproject the completed scene using features, and use the interactive attribute fusion module to correct the operation points of the overall scene, finally exporting an interactive 3D indoor scene.

[0032] Step 5: The evaluation system reads the reference layout rules, functional attributes, and spatial accessibility indicators, compares and calculates the deviations and rationality indicators in the scene generation process, and sends the results back to the system parameters for updating. Each scene generation module adjusts its parameters according to the indicators sent back by the evaluation system, completing one construction iteration;

[0033] Step 6: Execute steps 3 to 5 to generate the final interactive 3D indoor scene, and send the scene and reference rules to the evaluation system to calculate the corresponding indicators.

[0034] Beneficial effects

[0035] Compared with the prior art, the present invention has the following advantages:

[0036] 1. This invention collects and analyzes multi-source indoor structural data and automatically generates three-dimensional indoor scenes that conform to the layout rules of real life within a unified spatial framework. It can directly obtain rich room types, layout relationships and object distributions from real spatial structures, and can efficiently supplement the shortcomings of existing indoor three-dimensional scene datasets in terms of scale, type and structural rationality.

[0037] This invention includes a novel method for generating indoor scene layouts. This method utilizes the semantic relationships and spatial distribution self-similarity between objects in a real scene, and combines a layout module based on multi-constraint optimization, thereby incorporating effective information in both the overall layout and local structure.

[0038] The method of this invention can efficiently generate large-scale 3D indoor scene data with complete semantic and interactive attributes without relying on manual modeling or annotation, reducing human intervention and significantly improving data construction efficiency. Attached Figure Description

[0039] Figure 1 This is a flowchart of the method of the present invention.

[0040] Figure 2 This is a detailed schematic diagram of the present invention. Detailed Implementation

[0041] To better illustrate the purpose and advantages of the present invention, the method of the invention will be further described below in conjunction with the accompanying drawings and examples.

[0042] Step 1: Collect indoor space data containing realistic structural features, and then perform unified analysis and reconstruction. The collected spatial data types need to include various real-world interior layouts, including residential, office, and commercial scenarios. The included structural categories should cover as many types of housing as possible in real life, and include spatial information on different areas, functional zones, and unit topologies to ensure that the constructed 3D scene has good robustness and generalization ability for multi-task interaction.

[0043] Step 2: After determining the spatial coordinate system and scene scale, the furniture objects inside the rooms are automatically generated and placed to construct the overall 3D scene. After processing each spatial structure, the scene frame is kept static, and a corresponding list of furniture candidates is automatically generated according to different spatial functions and layout rules, such as beds, wardrobes, and desks in bedrooms. Subsequently, based on layout constraints such as the shape of the space, opening direction, and passable areas, these objects are automatically placed in appropriate positions. After arranging each room, the system automatically switches to the next space and repeats the above process until the data for the entire house structure is constructed.

[0044] Step 3: After completing the construction of the structure and furniture layout, semantic enhancement and interactive attribute modeling are performed on the scene. First, the object semantic parsing module and attribute generation module are applied to the scene to extract the functional features of each type of object and map the data to a higher-dimensional functional space. Then, the geometric features, functional attributes, and spatial relationships of all objects are merged to form a semantic map of the overall 3D scene.

[0045] Next, a multi-level functional analysis is performed on the furniture and its attachments, with corresponding interaction self-similarity calculated at each level to reweight the functional dependencies between objects. Since performing global interaction reasoning on the entire scene incurs significant computational overhead, and given the large number of objects, directly using global reasoning is impractical. Furthermore, while this approach has strong overall functional modeling capabilities, it often overlooks local interaction features, which are crucial for real-world operational scenarios.

[0046] To extract local functional features with low computational cost, this invention applies a multi-level functional reasoning mechanism based on local windows and a structural analysis method based on deep geometric convolution in the interactive attribute modeling module. Through this method, the system can extract local geometric features through structural relationships and simultaneously establish long-range associations using cross-object functional self-similarity.

[0047] The above detailed description further illustrates the purpose, technical solution, and beneficial effects of the invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for automatically constructing indoor 3D scene data for interactive tasks, characterized in that, Includes the following steps: Step 1: Collect multi-source interior structural data, identify the walls, doors, windows and spatial topology of the interior space through the structural analysis module, and perform scale unification and coordinate alignment to establish a unified interior space structural framework. Step 2: Construct an indoor 3D scene generation model, and generate 3D indoor scene data that conforms to the real layout rules based on the indoor space structure framework, including a semantic mapping candidate object generation and a multi-constraint layout optimization module; Step 3: Construct an indoor scene interaction modeling model for interactive tasks, introduce automatic embedding of interactive attributes based on functional analysis and consistency constraint optimization, and use the parsed indoor structural data and generated 3D indoor scene data for construction; Step 4: Save the construction parameters, output an interactive 3D scene based on the generated 3D indoor scene data, and complete the evaluation of physical consistency and interaction rationality indicators.

2. The method as described in claim 1, characterized in that, The structural analysis in step 1 includes: spatial geometric analysis: obtaining boundary information of the interior space, wall positions and door and window opening parameters; topological relationship analysis: determining the connectivity between rooms and the division of functional areas; structural alignment: normalizing the scale and aligning the coordinate system of interior structural data from different sources to achieve a unified spatial representation.

3. The method as described in claim 1, characterized in that, The indoor 3D scene generation model in step 2 includes: a candidate object generation module, which generates a set of candidate furniture and objects based on room type and spatial semantics; and a multi-constraint layout optimization module, which performs joint layout optimization on candidate objects through space occupancy constraints, collision constraints and accessibility constraints to generate a reasonable 3D indoor scene.

4. The method as described in claim 3, characterized in that, The accessibility constraint in the multi-constraint layout optimization module is used to ensure that the generated 3D indoor scene has reserved access areas that meet the access needs of human beings or intelligent agents.

5. The method as described in claim 1, characterized in that, The interactive modeling model in step 3 includes: a functional analysis module, which analyzes the categories and structural features of furniture and objects; an interactive attribute embedding module, which automatically generates rotational joints, translational joints, and state switching parameters for the furniture and objects; and a consistency constraint optimization module, which jointly optimizes interactive attributes and scene geometry to ensure the feasibility of interactive operations.

6. The method as described in claim 5, characterized in that, The interactive attributes include at least one of the following: rotational joint parameters, translational joint parameters, opening / closing state parameters, handle position parameters, and object mobility attributes.

7. The method as described in claim 5, characterized in that, The consistency constraint optimization includes: Physical consistency constraints based on collision detection; interaction feasibility constraints based on spatial reachability.

8. An automatic indoor 3D scene data construction system for interactive tasks, characterized in that, include: Interior structure analysis subsystem: used to collect and analyze multi-source interior structure data to establish a unified interior space structure framework; Scene generation subsystem: Generates 3D indoor scene data based on semantic mapping and multi-constraint layout optimization; Interactive modeling subsystem: Constructs interactive 3D indoor scenes through functional analysis and interactive attribute embedding; Parameter saving and result evaluation subsystem: Saves construction parameters and evaluates the physical consistency and interactive rationality of the generated 3D indoor scenes.

Images