A multi-modal artificial intelligence interactive 3D decoration method and system
Patent Information
- Application Number
- CN202610969121.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-07-01
- Publication Date
- 2026-08-21
AI Technical Summary
然而，该流程通常在识别出家具类型后即告终止，识别结果无法作为一个可摆放的三维对象，被自动、精确地放入用户当前的三维房间场景中
[0054]与现有技术相比，本发明的有益效果是：通过让大模型输出结构化意图，再由前端根据实时场景进行空间计算，因此能够兼顾自然语言交互的灵活性和3D场景执行的稳定性。
Smart Images
Figure CN122615971A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of artificial intelligence technology, specifically a multimodal AI-interactive 3D decoration method and system. Background Technology
[0002] With the rapid development of Large Language Models (LLM), speech recognition, computer vision, generative imaging, and Web 3D rendering technologies, the home decoration design software field is undergoing a profound transformation from traditional computer-aided design (CAD) to intelligent and natural interaction modes. Users increasingly expect to quickly build and modify 3D home decoration plans through more intuitive and low-barrier methods such as "decorate a room with one sentence," "upload furniture pictures to generate 3D objects that can be placed," and "enter prompt words to generate decorative paintings."
[0003] Currently, existing 3D home decoration systems or solutions can be mainly divided into the following categories, each with its own significant drawbacks:
[0004] Traditional 3D home decoration software, such as Kujiale and 3D Home, offers powerful room modeling, furniture model loading, material editing, and scene rendering capabilities. However, their core interaction heavily relies on users manually selecting furniture from a catalog and placing it in 3D space by dragging, rotating, and fine-tuning parameters with the mouse. This operating mode is not only costly to learn and inefficient, but also makes it difficult for ordinary users to quickly and accurately express high-level, natural spatial layout intentions such as "placing the TV opposite the sofa" or "placing the shoe cabinet next to the entrance door."
[0005] Home renovation question-and-answer and suggestion systems based on large models attempt to incorporate large language models to understand users' natural language input. However, these systems typically only output AI at the level of generating text suggestions, describing renovation styles, or recommending furniture lists, failing to translate the understanding results into concrete, executable operations in a 3D scene. More importantly, if the large language model is asked to directly output the 3D spatial coordinates of furniture, the generated coordinates are highly susceptible to unfeasible or unsafe layouts, such as furniture passing through walls, going out of bounds, blocking doorways, overlapping, or being placed in the wrong room, because the model itself lacks the ability to perceive real-time room boundaries, doorway passage areas, the precise bounding box size of furniture, and the positions of existing furniture in the current scene.
[0006] Image-based interior design rendering tools use generative AI models to directly generate 2D interior design renderings based on user-provided room photos or text prompts. While these tools offer quick visual references, they only generate static images. Users cannot interactively edit, move, or modify the furniture or materials in the images, nor can they generate structured 3D scene data for subsequent detailed design.
[0007] Furniture image recognition and product recommendation systems typically terminate after a user uploads a furniture image. The system uses a visual model to identify the furniture's category, color, or style, and then returns the identification result or links to similar products. However, this process usually ends after identifying the furniture type, and the identification result cannot be automatically and accurately placed into the user's current 3D room scene as a configurable 3D object.
[0008] In summary, while existing technical solutions address specific aspects of the interior design process, none have established a complete closed loop encompassing multimodal input (natural language, voice, images, prompts) to structured intent parsing, front-end spatial security calculations, automated 3D scene execution, and data persistence and recovery. In particular, bridging the gap between natural language understanding results and precise 3D spatial calculations, ensuring that AI-generated intents can be translated into stable, executable, and security-compliant 3D scene operations, and supporting continuous editing, saving, and recovery, remains a critical technical challenge in the field of interactive 3D home decoration design. Summary of the Invention
[0009] The purpose of this invention is to provide a multimodal artificial intelligence interactive 3D decoration method and system to solve the problems mentioned in the background art.
[0010] To achieve the above objectives, the present invention provides the following technical solution:
[0011] A multimodal AI-interactive 3D interior design method, the method comprising:
[0012] Construct a 3D decoration scene;
[0013] Get the current room, room data, door data, existing furniture data, and current scene summary, and send them to the backend along with the user's text commands as the input context for the large language model;
[0014] The large language model outputs structured decoration intent data based on the input context;
[0015] Select the corresponding execution logic based on the structured decoration intention data;
[0016] Candidate locations are calculated based on room boundaries, doorway access areas, wall thickness, and existing occupied areas. After detection, these locations are written into a 3D decoration scene object, and a scoring result is generated based on the written 3D scene object.
[0017] As a further improvement of the present invention, the structured decoration intent data includes at least one of the following intent types: creating furniture, creating and placing furniture, moving furniture, moving furniture across rooms, deleting furniture, modifying furniture, and batch operations.
[0018] As a further improvement of the present invention, the step of calculating candidate positions based on room boundaries, doorway passage areas, wall thickness, and existing occupied areas, and then writing them into the 3D scene object after detection, specifically includes:
[0019] Candidate locations are generated based on the target room boundaries, wall spacing, doorway access area, existing furniture occupancy area, and target furniture bounding box.
[0020] For each candidate location, an inspection is performed to determine whether the furniture at that location exceeds the room boundary, encroaches on the doorway, overlaps with existing furniture, or meets the preset wall distance and passage space requirements.
[0021] If a candidate location fails the detection, boundary clipping, precise avoidance, or grid search are performed to generate new candidate locations and detect them again until a safe location that passes the detection is obtained.
[0022] As a further improvement to the present invention, the step of calculating candidate positions based on room boundaries, doorway passage areas, wall thickness, and existing occupied areas, and then writing them into the 3D scene object after detection, further includes:
[0023] For the placement pattern near the door, read the center point of the door, the wall it is on, the door width and the passage depth at the door, calculate the passage area at the door and generate candidate positions on the side of the door;
[0024] For automatic placement mode, first perform random sampling of safety points, then perform precise avoidance and grid search as a fallback.
[0025] If a position that passes the detection cannot be obtained after boundary clipping, precise avoidance, or grid search, a fallback position is used, and the reason for failure is recorded.
[0026] As a further improvement to the present invention, it also includes a step of arranging furniture relative to its position:
[0027] Locate reference furniture based on reference information, and read the bounding box of the reference furniture and the target furniture;
[0028] Normalize directional terms into global directions;
[0029] The target center point position of the target furniture is calculated based on the center point of the reference furniture, the size of the reference furniture, the size of the target furniture, and the spacing parameters.
[0030] The target center point position is tested for safe placement.
[0031] As a further improvement to the present invention, it also includes a batch furniture operation step:
[0032] For instructions involving multiple furniture items, the large language model returns structured decoration intent data of the batch operation type, which contains multiple single furniture operations arranged in the order expressed by the user.
[0033] Each single furniture operation is broken down into independent add, move, delete, or modify operations and executed sequentially.
[0034] Assign batch numbers and batch totals to batch operations, and generate distributed placement positions based on room size, furniture size, and existing occupied areas, so that multiple pieces of furniture are distributed at intervals in the target room.
[0035] If a sub-operation fails, record the sub-operation number, intent type, and error message, and retain the results of other successfully executed operations.
[0036] As a further improvement of the present invention, the step of generating a scoring result based on the written 3D scene object includes:
[0037] Obtain furniture, room, and door data for the current room or the entire house, perform deterministic rule detection to determine whether furniture is out of bounds, occupies the doorway, overlaps with each other, exceeds the density threshold, and is unobstructed, and generate deterministic detection results;
[0038] The deterministic detection results and scene summary are sent to the large language model, which then generates a rating level, problem description, cause explanation, and optimization suggestions.
[0039] If the large language model call fails, a backup score is generated based on the deterministic detection results.
[0040] As a further improvement to the present invention, it also includes the steps of generating and mounting artificial intelligence-based decorative paintings on walls:
[0041] After the user enters a decorative painting prompt and selects the target wall, the image generation service is invoked to generate a decorative painting image and return the image address.
[0042] A decorative painting object is created based on the image address, the image address is written into the furniture data, and the rotation angle and wall-fitting position of the decorative painting object are calculated based on the wall orientation selected by the user.
[0043] Save the decorative painting object as a furniture object.
[0044] As a further improvement to the present invention, it also includes a step of image furniture recognition and scene placement:
[0045] The furniture information in the image is identified and parsed into structured fields, which include furniture type, furniture name, color, style, material and confidence level.
[0046] Based on the furniture type, match existing furniture catalogs or default furniture models to generate matching results;
[0047] After the user confirms the matching results, the identified furniture will be written into the 3D scene object as placeable objects.
[0048] This invention also provides a multimodal artificial intelligence interactive 3D decoration system, the system comprising:
[0049] The scene building module is used to construct 3D decoration scenes;
[0050] The data acquisition module is used to acquire the current room, room data, door data, existing furniture data, and current scene summary, which are sent to the backend along with the user's text commands as the input context for the large language model.
[0051] The intent generation module is used by the large language model to output structured decoration intent data based on the input context;
[0052] The execution module is used to select the corresponding execution logic based on the structured decoration intention data;
[0053] The rendering generation module is used to calculate candidate positions based on room boundaries, doorway passage areas, wall thickness, and existing occupied areas. After detection, these positions are written into the 3D decoration scene object, and a scoring result is generated based on the written 3D scene object.
[0054] Compared with the prior art, the beneficial effects of the present invention are: by allowing a large model to output structured intent, and then having the front end perform spatial calculations based on the real-time scene, it is possible to balance the flexibility of natural language interaction and the stability of 3D scene execution.
[0055] Introducing detection of room boundaries, furniture enclosures, wall distances, existing furniture occupancy areas, and doorway passage areas during furniture placement can reduce issues such as furniture going out of bounds, penetrating walls, overlapping, and blocking doors, making the automated decoration results more consistent with real-world usage scenarios.
[0056] By incorporating AI-generated decorative paintings, image-based furniture recognition, and AI-based layout scoring into the same 3D decoration process, the generated content and recognition results are no longer merely static displays, but rather become three-dimensional scene objects that can be hung, placed, saved, and restored. Attached Figure Description
[0057] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention.
[0058] Figure 1 This is an overall architecture diagram provided for an embodiment of the present invention.
[0059] Figure 2 This is a flowchart of the multimodal decoration instruction processing provided in an embodiment of the present invention.
[0060] Figure 3 A flowchart illustrating safe placement and doorway avoidance provided in an embodiment of the present invention.
[0061] Figure 4 This is a schematic diagram of relative furniture placement provided for an embodiment of the present invention.
[0062] Figure 5 The AI layout scoring flowchart is provided for an embodiment of the present invention.
[0063] Figure 6 This is a flowchart illustrating the AI-generated decorative painting and its wall mounting process, as provided in an embodiment of the present invention.
[0064] Figure 7 This is a flowchart of image furniture recognition and scene placement provided in an embodiment of the present invention.
[0065] Figure 8 This is a flowchart illustrating the process of saving and restoring user renovation progress as provided in an embodiment of the present invention. Detailed Implementation
[0066] To make the technical problems to be solved, the technical solutions, and the beneficial effects of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present invention and are not intended to limit the present invention.
[0067] In this embodiment of the invention, a multimodal artificial intelligence interactive 3D decoration method is provided, the method comprising:
[0068] Construct a 3D decoration scene;
[0069] Get the current room, room data, door data, existing furniture data, and current scene summary, and send them to the backend along with the user's text commands as the input context for the large language model;
[0070] The large language model outputs structured decoration intent data based on the input context;
[0071] Select the corresponding execution logic based on the structured decoration intention data;
[0072] Candidate locations are calculated based on room boundaries, doorway access areas, wall thickness, and existing occupied areas. After detection, these locations are written into a 3D decoration scene object, and a scoring result is generated based on the written 3D scene object.
[0073] This embodiment includes a user input terminal, a browser front-end 3D scene, back-end services, artificial intelligence services, and a database. The user input terminal is used to receive text commands, voice commands, and upload furniture images and decorative painting prompts; the browser front-end is used to render 3D rooms, furniture, doors, walls, and decorative objects, and perform spatial location calculations; the back-end services are used for user authentication, interface scheduling, AI service calls, and data storage; the artificial intelligence services include large language model semantic parsing, speech recognition, image generation, and visual furniture recognition; the database is used to store user, room, furniture details, generation records, and decoration progress.
[0074] Users can input text or voice commands. For voice commands, the system first converts them into text using a speech recognition service. The backend combines the user's text, the current room, room structure, door data, and existing furniture data into a context and sends it to the large language model. The large language model does not directly output the final 3D coordinates but instead outputs a structured JSON intent. This JSON intent contains fields such as operation type, target furniture, target room, placement mode, relative reference object, color, size, rotation, or batch operation. After receiving the JSON intent, the frontend performs preview, spatial calculation, and specific 3D operations based on the current scene data.
[0075] Core idea: AI is responsible for understanding intent, the front end is responsible for spatial computation, and the back end is responsible for service orchestration and data persistence.
[0076] In the current embodiment, the 3D rendering engine of this invention can be Three.js; without changing the natural language intent parsing, safe placement and scene object management process, it can also be replaced with Unity, Unreal, WebGPU or other 3D rendering engines.
[0077] In the current embodiment, the three-dimensional decoration scene model of the present invention can adopt GLB or GLTF; while maintaining the consistency of furniture catalog, size information, bounding box calculation and scene loading logic, it can also be adapted to OBJ, FBX or other three-dimensional model formats through model conversion or loader.
[0078] The large language model of this invention can be DeepSeek or other models with structured JSON output capabilities; the speech recognition service can be replaced with other ASR services; the image generation service can be Doubao, Stable Diffusion, Midjourney, DALL·E or other text-to-image models; and the image furniture recognition can be achieved using a large visual model, an object detection model, an image classification model, or manually labeled assisted recognition.
[0079] The safe placement detection algorithm of this invention can be implemented by means of candidate point search, grid search, collision detection, constraint optimization or heuristic rules; the doorway passage area can be represented as a rectangle, a sector, a path buffer or other detectable area, as long as it can determine whether the furniture occupies the necessary passage space.
[0080] like Figure 1 and Figure 2 As shown, the processing of multimodal decoration instructions can be further subdivided into the following steps:
[0081] Input normalization: Receives text or voice commands; voice input is converted to text via a voice recognition interface and then written into the command input box, or it can directly trigger the AI command processing flow. Text input is normalized based on the current room, room alias, and keywords at the entrance.
[0082] Scene context assembly: The front end collects the current room, room data, door data, existing furniture data, and current scene summary, and sends them to the back end along with user commands; the back end uses this data as a scene snapshot and passes it to the large language model for prompt words.
[0083] Structured Intent Generation: The large language model only outputs structured decoration intent data (JSON intent) and does not directly calculate the final 3D rendering engine world coordinates (Three.js world coordinates). The structured decoration intent data includes: creating furniture, creating and placing furniture, moving furniture, moving furniture across rooms, deleting furniture, modifying furniture, batch operations, and unsupported operations. The system performs basic validation and normalization on the intent type, target room, target furniture, and placement mode; if the reference object does not exist or the operation is not supported, it returns an "unsupported" status or an execution failure message.
[0084] Front-end executor distribution: The front-end selects the execution logic based on the JSON intent, such as adding, moving, deleting, modifying, batch operations, relative furniture placement or placement near the door; for some previewable operations, a placement preview state can be generated first, and then the user can confirm or execute directly.
[0085] Space calculation and submission: The front end uses the placement tool to calculate candidate positions based on the room boundaries, doorway passage area, wall thickness, and existing occupied area; after passing the detection, the candidate positions are written into the 3D scene object; if the detection fails, the corresponding error message or reason information is returned, such as: exceeding the room boundaries, occupying the doorway passage area or blocking the area, not finding the specified door, no safe placement position next to the door, etc.
[0086] Progress saving: When a user saves their progress, the front end first saves the furniture details in batches and populates the database IDs, and then saves a scene snapshot containing furniture position, rotation, scaling, color, model path, image address and room information (scene_data), which is convenient for subsequent recovery and re-scoring.
[0087] JSON intent example:
[0088] {
[0089] "intent": "create_and_place",
[0090] "target": {"room_id": "living-room"},
[0091] "furniture": {
[0092] "name": "square coffee table",
[0093] "catalog_id": 7,
[0094] "type": "table"
[0095] },
[0096] "placement": {
[0097] "mode": "relative_to_furniture",
[0098] "reference": {
[0099] "room_id": "living-room",
[0100] "name": "Three-seater sofa",
[0101] "catalog_id": 1,
[0102] "type": "sofa"
[0103] },
[0104] "direction": "south",
[0105] "alignment": "center",
[0106] "gap": 0.5
[0107] },
[0108] Message: "Create a square coffee table in the living room and place it in front of the three-seater sofa."
[0109] }
[0110] Using JSON intent as an intermediate representation decouples the natural language understanding results from 3D scene execution.
[0111] As a preferred embodiment of the present invention, the step of calculating candidate positions based on room boundaries, doorway passage areas, wall thickness, and existing occupied areas, and then writing them into the 3D scene object after detection, specifically includes:
[0112] Candidate locations are generated based on the target room boundaries, wall spacing, doorway access area, existing furniture occupancy area, and target furniture bounding box.
[0113] For each candidate location, an inspection is performed to determine whether the furniture at that location exceeds the room boundary, encroaches on the doorway, overlaps with existing furniture, or meets the preset wall distance and passage space requirements.
[0114] If a candidate location fails the detection, boundary clipping, precise avoidance, or grid search are performed to generate new candidate locations and detect them again until a safe location that passes the detection is obtained.
[0115] Also includes:
[0116] For the placement pattern near the door, read the center point of the door, the wall it is on, the door width and the passage depth at the door, calculate the passage area at the door and generate candidate positions on the side of the door;
[0117] For automatic placement mode, first perform random sampling of safety points, then perform precise avoidance and grid search as a fallback.
[0118] If a position that passes the detection cannot be obtained after boundary clipping, precise avoidance, or grid search, a fallback position is used, and the reason for failure is recorded.
[0119] In this embodiment, as Figure 3 As shown, the front-end spatial calculation generates candidate locations based on the target room boundary, wall spacing, doorway passage area, existing furniture occupied area, and target furniture bounding box. The system checks each candidate location to determine whether it exceeds the room boundary, intrudes into the doorway passage area, overlaps with existing furniture, and retains the necessary wall spacing and passage space. If a candidate location is invalid, the system performs boundary clipping, precise avoidance, or mesh search until an executable and safe location is obtained, and then returns the final scene coordinates and reason code.
[0120] The entrance area is designated as a no-occupancy zone for testing to prevent furniture from blocking the door or affecting passage.
[0121] The system generates the desired location based on the target room size, wall distance, furniture size, and placement pattern. For the "near the door" mode, the system reads data such as the center point of the door, the wall it is on, the door width, and the passage depth at the doorway, calculates the passage area at the doorway, and generates candidate locations on the side of the door. For automatic placement modes such as "automatic," "safe automatic," or "random safe," the system first performs random sampling of safe points, and then performs precise avoidance and grid search as a fallback if necessary.
[0122] The placement detection process determines whether furniture is too large, exceeds room boundaries, or enters doorway or other registered occupied areas. If the initial location is unavailable, the system performs boundary trimming, precise avoidance, or grid search. If an ideal location still cannot be found, a fallback location is used, and the corresponding reason information is recorded, such as: furniture exceeding room boundaries has been trimmed, occupied areas or doorway have been avoided, the nearest safe candidate location has been used, or the system has returned to the fallback location in the center of the room. Furniture overlap and doorway obstruction issues are further detected during the layout scoring phase.
[0123] As a preferred embodiment of the present invention, it further includes a step of arranging furniture relative to its position:
[0124] Locate reference furniture based on reference information, and read the bounding box of the reference furniture and the target furniture;
[0125] Normalize directional terms into global directions;
[0126] The target center point position of the target furniture is calculated based on the center point of the reference furniture, the size of the reference furniture, the size of the target furniture, and the spacing parameters.
[0127] The target center point position is tested for safe placement.
[0128] In this embodiment, for relative placement instructions such as "place in front of the sofa," "near the bedside table," and "place next to the dining table," the system first identifies the target furniture, reference furniture, and directional words. Then, it reads the center point, width, depth, height, and bounding box of the reference furniture. The system combines the target furniture dimensions, directional words, and spacing parameters to calculate the target center position and sends this position into the safe placement process to ensure that the relative position conforms to the user's intention and does not cause boundary overflow, overlap, or blockage of doors.
[0129] like Figure 4 As shown, when placing furniture relative to each other, the system first finds the reference furniture based on the "reference" field, and then reads the bounding boxes of the reference furniture and the target furniture. Currently, the system normalizes directions such as "front, back, left, right" into global directions such as south, north, west, and east, and calculates the target center point based on the center point of the reference furniture, the size of the reference furniture, the size of the target furniture, and the spacing parameters.
[0130] For example, if a user inputs "Place the coffee table in front of the sofa", the large language model returns the intent "relative to furniture placement". After the front end finds the sofa object, it places the coffee table south (in front) of the sofa, at a distance equal to half the depth of the sofa, half the depth of the coffee table, and the spacing. Then, this position continues to enter the safe placement detection to reduce the possibility of going out of bounds, pressing against doors, or entering obstructed areas.
[0131] The relative placement is not determined by directly providing coordinates, but rather by a combination of factors including the surrounding furniture, directional information, and safety checks.
[0132] As a preferred embodiment of the present invention, it also includes a batch furniture operation step:
[0133] For instructions involving multiple furniture items, the large language model returns structured decoration intent data of the batch operation type, which contains multiple single furniture operations arranged in the order expressed by the user.
[0134] Each single furniture operation is broken down into independent add, move, delete, or modify operations and executed sequentially.
[0135] Assign batch numbers and batch totals to batch operations, and generate distributed placement positions based on room size, furniture size, and existing occupied areas, so that multiple pieces of furniture are distributed at intervals in the target room.
[0136] If a sub-operation fails, record the sub-operation number, intent type, and error message, and retain the results of other successfully executed operations.
[0137] In this embodiment, for an instruction involving multiple pieces of furniture, the system generates a "batch operation" intent and expands each operation into a separate furniture creation or movement operation. The front end assigns a sequence number and a total number to the batch operations and generates distributed placement positions based on room size, furniture size, and existing occupied area, so that multiple pieces of furniture can be automatically spaced apart in the target room, avoiding them all being stacked in the same position.
[0138] In batch operations, the large language model returns structured data of intent type "batch operations," where the operation array lists multiple individual operations in the order expressed by the user. The front end expands each operation into a separate add, move, delete, or modify operation and executes them sequentially.
[0139] When multiple automatic placement operations target the same room, the front end assigns batch numbers and batch totals to these operations and generates distributed locations within the room to prevent multiple pieces of furniture from being stacked in the center of the room by default. If a sub-operation fails, the system records the sub-operation's sequence number, intent type, and error message, while retaining the results of other successfully executed operations.
[0140] As a preferred embodiment of the present invention, the step of generating a scoring result based on the written 3D scene object includes:
[0141] Obtain furniture, room, and door data for the current room or the entire house, perform deterministic rule detection to determine whether furniture is out of bounds, occupies the doorway, overlaps with each other, exceeds the density threshold, and is unobstructed, and generate deterministic detection results;
[0142] The deterministic detection results and scene summary are sent to the large language model, which then generates a rating level, problem description, cause explanation, and optimization suggestions.
[0143] If the large language model call fails, a backup score is generated based on the deterministic detection results.
[0144] In this embodiment, as Figure 5 As shown, when a user requests a layout rating, the system first collects data on furniture, rooms, and doors in the current room or the entire house. It then performs deterministic rule detection to determine if furniture is out of bounds, blocking doors, overlapping, has excessive density, or obstructs traffic flow. The backend then sends the detection results and a scene summary to a large language model, which provides a rating, problem description, explanation of the cause, and optimization suggestions.
[0145] The layout scoring employs a two-stage structure: front-end deterministic detection and back-end AI scoring and interpretation. The front-end detects boundary violations, doorway occupancy, furniture embedded in rooms, bathroom furniture placement, furniture overlap, furniture density, and passageway issues, generating deterministic detection results. The back-end scoring model must use these deterministic detection results as the hard rule basis. If the AI call fails, a backup score is used, generating a score based on the deterministic detection results.
[0146] The rating results include a total score, grade, sub-categories, highlights, issues, and suggestions. It only provides ratings, explanations, and suggestions; it does not directly return automatically executable furniture movement commands or modify the current scene.
[0147] As a preferred embodiment of the present invention, it also includes the steps of generating and wall-mounting artificial intelligence decorative paintings:
[0148] After the user enters a decorative painting prompt and selects the target wall, the image generation service is invoked to generate a decorative painting image and return the image address.
[0149] A decorative painting object is created based on the image address, the image address is written into the furniture data, and the rotation angle and wall-fitting position of the decorative painting object are calculated based on the wall orientation selected by the user.
[0150] Save the decorative painting object as a furniture object.
[0151] In this embodiment, as Figure 6 As shown, after the user inputs a decorative painting prompt and selects a target wall, the backend calls an image generation service to generate a decorative painting image and saves the image as an accessible resource. The frontend creates a decorative painting object based on the wall orientation, wall position, user selection point, and preset size, loads the generated image as a texture onto the 3D wall plane, and writes the decorative painting object into the scene data, enabling it to be saved, reloaded, and edited further.
[0152] In the decorative painting generation process, the backend calls the image generation service to generate image files and returns the image address; the frontend creates a decorative painting object of type "decorative painting" based on the image address, writes the image address into the furniture data, and calculates the rotation angle and wall-fitting position of the decorative painting according to the north, south, east or west wall direction selected by the user.
[0153] When saving progress, the AI-generated decorative painting is treated as a regular furniture object, saving its image address, whether it is an AI-generated marker, position, rotation, scale, model path, and room it belongs to. When the user logs in again or loads the progress, the front end asynchronously recreates the object based on the furniture data in the scene snapshot, restoring its position, rotation, scale, and image address.
[0154] As a preferred embodiment of the present invention, it also includes a step of image furniture recognition and scene placement:
[0155] The furniture information in the image is identified and parsed into structured fields, which include furniture type, furniture name, color, style, material and confidence level.
[0156] Based on the furniture type, match existing furniture catalogs or default furniture models to generate matching results;
[0157] After the user confirms the matching results, the identified furniture will be written into the 3D scene object as placeable objects.
[0158] In this embodiment, as Figure 7 As shown, after a user uploads a furniture image, the backend calls a visual model to identify the furniture type, color, style, material, confidence level, and description information in the image, and parses it into structured JSON. The system filters invalid objects, matches them with furniture catalogs or 3D model paths, and after user confirmation, generates placeable furniture objects. Then, the system enters the safe placement process, placing the identified furniture into the target room.
[0159] In the image-based furniture recognition process, the system sends the user-uploaded image to the visual model in Base64 encoded format or in the form of a Uniform Resource Identifier (URI). The visual model then returns structured fields such as furniture type, furniture name, color, style, material, confidence level, and description. If the image is not furniture or the furniture type is not in the supported list, the model returns an empty array or an unknown type.
[0160] The front-end or back-end can match the returned furniture type with an existing furniture catalog or default furniture model, and add it to the scene as a regular furniture object after user confirmation; for recognition failure or interface error, the system returns an error message; for recognition results with low confidence, the system displays confidence information, and users can still choose to place a single recognition object or place all recognition objects in batches.
[0161] like Figure 8 As shown, when a user saves their renovation progress, the front-end first collects the furniture details in the current scene, calls the batch save interface to write them to the furniture table, and the back-end returns the database ID of each piece of furniture. After the front-end populates the object IDs, it submits a complete scene snapshot, including the current room, furniture position, rotation, scaling, color, model path, and wall decoration objects. When the user logs in next time, the system reads the scene snapshot and furniture details, asynchronously loads the 3D model, and restores the spatial state.
[0162] The save and restore process uses a structure that combines furniture details and scene snapshots. During saving, the front end collects the client ID, database ID, type, name, position, scale, rotation, color, model path, image address, AI-generated tags, and room information for each piece of furniture in the scene. First, it calls the batch save furniture interface; the furniture table actually saves mainly the type, position, scale, color, and model path to the database. Then, it calls the progress save interface to write the scene snapshot data.
[0163] During restoration, the system reads the scene snapshot furniture data from the progress data, asynchronously calls the furniture loading logic to recreate the furniture objects, and restores their position, rotation, scale, color, image address, and room affiliation. If an object lacks room information, the system will synchronize the room based on the current position; if the model path is empty, it will load the furniture according to its type or default configuration.
[0164] Example Scenario: A user says in the living room, "Generate a square coffee table and place it in front of the three-seater sofa." The system converts the speech to text and sends it to the backend. The large language model returns the "Create and Place" intent, with the placement mode set to "relative to furniture placement," and the reference pointing to the three-seater sofa in the living room. The frontend locates the sofa object, calculates candidate positions for the coffee table based on its south-facing orientation and spacing, and then writes it into the scene after passing a safe placement check. After the user saves the page, the system saves the coffee table's client ID, database ID, position, rotation, scale, model path, and room information; the next time the user logs in, the system restores the object based on the furniture data in the scene snapshot.
[0165] First, synchronize the furniture details, then save a complete scene snapshot to ensure that the object identity and space state are consistent during restoration.
[0166] This invention also provides a multimodal artificial intelligence interactive 3D decoration system, the system comprising:
[0167] The scene building module is used to construct 3D decoration scenes;
[0168] The data acquisition module is used to acquire the current room, room data, door data, existing furniture data, and current scene summary, which are sent to the backend along with the user's text commands as the input context for the large language model.
[0169] The intent generation module is used by the large language model to output structured decoration intent data based on the input context;
[0170] The execution module is used to select the corresponding execution logic based on the structured decoration intention data;
[0171] The rendering generation module is used to calculate candidate positions based on room boundaries, doorway passage areas, wall thickness, and existing occupied areas. After detection, these positions are written into the 3D decoration scene object, and a scoring result is generated based on the written 3D scene object.
[0172] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A multimodal AI-interactive 3D decoration method, characterized in that, The method includes: Construct a 3D decoration scene; Get the current room, room data, door data, existing furniture data, and current scene summary, and send them to the backend along with the user's text commands as the input context for the large language model; The large language model outputs structured decoration intent data based on the input context; Select the corresponding execution logic based on the structured decoration intention data; Candidate locations are calculated based on room boundaries, doorway access areas, wall thickness, and existing occupied areas. After detection, these locations are written into a 3D decoration scene object, and a scoring result is generated based on the written 3D scene object.
2. The multimodal AI-interactive 3D decoration method according to claim 1, characterized in that, The structured decoration intent data includes at least one of the following intent types: create furniture, create and place, move furniture, move furniture across rooms, delete furniture, modify furniture, and batch operations.
3. The multimodal AI-interactive 3D decoration method according to claim 1, characterized in that, The step of calculating candidate locations based on room boundaries, doorway passage areas, wall thickness, and existing occupied areas, and then writing them into the 3D scene object after detection, specifically includes: Candidate locations are generated based on the target room boundaries, wall spacing, doorway access area, existing furniture occupancy area, and target furniture bounding box. For each candidate location, an inspection is performed to determine whether the furniture at that location exceeds the room boundary, encroaches on the doorway, overlaps with existing furniture, or meets the preset wall distance and passage space requirements. If a candidate location fails the detection, boundary clipping, precise avoidance, or grid search are performed to generate new candidate locations and detect them again until a safe location that passes the detection is obtained.
4. The multimodal AI interactive 3D decoration method according to claim 3, characterized in that, The steps of calculating candidate locations based on room boundaries, doorway access areas, wall thickness, and existing occupied areas, and then writing them into the 3D scene object after detection, also include: For the placement pattern near the door, read the center point of the door, the wall it is on, the door width and the passage depth at the door, calculate the passage area at the door and generate candidate positions on the side of the door; For automatic placement mode, first perform random sampling of safety points, then perform precise avoidance and grid search as a fallback. If a position that passes the detection cannot be obtained after boundary clipping, precise avoidance, or grid search, a fallback position is used, and the reason for failure is recorded.
5. The multimodal AI-interactive 3D decoration method according to claim 1, characterized in that, It also includes steps for arranging relative furniture positions: Locate reference furniture based on reference information, and read the bounding box of the reference furniture and the target furniture; Normalize directional terms into global directions; The target center point position of the target furniture is calculated based on the center point of the reference furniture, the size of the reference furniture, the size of the target furniture, and the spacing parameters. The target center point position is tested for safe placement.
6. The multimodal AI-interactive 3D decoration method according to claim 1, characterized in that, It also includes steps for handling bulk furniture: For instructions involving multiple furniture items, the large language model returns structured decoration intent data of the batch operation type, which contains multiple single furniture operations arranged in the order expressed by the user. Each single furniture operation is broken down into independent add, move, delete, or modify operations and executed sequentially. Assign batch numbers and batch totals to batch operations, and generate distributed placement positions based on room size, furniture size, and existing occupied areas, so that multiple pieces of furniture are distributed at intervals in the target room. If a sub-operation fails, record the sub-operation number, intent type, and error message, and retain the results of other successfully executed operations.
7. The multimodal AI-interactive 3D decoration method according to claim 1, characterized in that, The steps for generating a rating result based on the written 3D scene object include: Obtain furniture, room, and door data for the current room or the entire house, perform deterministic rule detection to determine whether furniture is out of bounds, occupies the doorway, overlaps with each other, exceeds the density threshold, and is unobstructed, and generate deterministic detection results; The deterministic detection results and scene summary are sent to the large language model, which then generates a rating level, problem description, cause explanation, and optimization suggestions. If the large language model call fails, a backup score is generated based on the deterministic detection results.
8. The multimodal AI interactive 3D decoration method according to claim 1, characterized in that, It also includes the steps for generating and mounting AI-powered decorative paintings on walls: After the user enters a decorative painting prompt and selects the target wall, the image generation service is invoked to generate a decorative painting image and return the image address. A decorative painting object is created based on the image address, the image address is written into the furniture data, and the rotation angle and wall-fitting position of the decorative painting object are calculated based on the wall orientation selected by the user. Save the decorative painting object as a furniture object.
9. The multimodal AI-interactive 3D decoration method according to claim 1, characterized in that, It also includes image furniture recognition and scene placement steps: The furniture information in the image is identified and parsed into structured fields, which include furniture type, furniture name, color, style, material and confidence level. Based on the furniture type, match existing furniture catalogs or default furniture models to generate matching results; After the user confirms the matching results, the identified furniture will be written into the 3D scene object as placeable objects.
10. A multimodal AI interactive 3D decoration system, used to implement the multimodal AI interactive 3D decoration method according to any one of claims 1-9, characterized in that, The system includes: The scene building module is used to construct 3D decoration scenes; The data acquisition module is used to acquire the current room, room data, door data, existing furniture data, and current scene summary, which are sent to the backend along with the user's text commands as the input context for the large language model. The intent generation module is used by the large language model to output structured decoration intent data based on the input context; The execution module is used to select the corresponding execution logic based on the structured decoration intention data; The rendering generation module is used to calculate candidate positions based on room boundaries, doorway passage areas, wall thickness, and existing occupied areas. After detection, these positions are written into the 3D decoration scene object, and a scoring result is generated based on the written 3D scene object.