Intelligent home decoration system and method based on mixed reality

Abstract

The invention discloses an intelligent home decoration system and method based on mixed reality, and the system comprises a data collection module which comprises a space scanning module, a 3D model generation module and a space feature recognition module; the intelligent assistant module comprises an AI drive design suggestion module based on deep learning, an automatic layout optimization module, a personalized recommendation engine and a design rule check module; the multi-party cooperation module comprises a cross-device synchronization module, a real-time multi-user interaction module, a remote cooperation support module and a role authority management module; and the visual presentation module comprises a mixed reality rendering module, a real-time light and shadow simulation module, a material reality presentation module and a virtual image presentation module. According to the method, the design efficiency is greatly improved, the design period is shortened, design errors and rework are reduced, the project cost is reduced, the design quality is improved, the whole-course visualization and traceability of the design process are realized, and the working mode of traditional home decoration design is changed.

Description

Technical Field

[0001] The present invention relates to the field of mixed reality technology and intelligent building design, and in particular to an intelligent home decoration design system based on mixed reality and an implementation method thereof. Background Art

[0002] With the improvement of living standards and the increase of personalized needs, the home decoration design industry is facing unprecedented challenges and opportunities. Traditional design methods can no longer meet the needs of modern customers for efficient, accurate and personalized design. At present, existing technologies mainly rely on CAD software and 2D drawings, but these tools are difficult to intuitively display the design effects, resulting in difficulties for customers to understand, low communication efficiency, and increased design modification times and costs. Although virtual reality (VR) technology provides a certain degree of immersive experience, it is disconnected from the actual space, lacks real-time interaction capabilities, and cannot make instant design adjustments and decisions in the actual environment. Although augmented reality (AR) applications can overlay virtual elements in the actual space, they have limited interactivity and lack of in-depth design tools and collaboration functions, making it difficult to support complex design processes and multi-party collaboration.

[0003] The traditional designer-client communication model relies on face-to-face meetings and multiple rounds of communication, which is time-consuming and difficult to capture and implement in a timely manner. Design process management also lacks systematic records and tools, which makes it difficult to track design decisions and is not conducive to solution optimization and subsequent maintenance. The main challenges facing the industry include: how to make customers understand and participate in the design process more intuitively, how to improve the efficiency of real-time collaboration between designers and customers, how to modify and optimize designs quickly and at low cost, how to use artificial intelligence technology to assist design decisions, and how to ensure the accurate implementation of design solutions in actual construction.

[0004] Although virtual reality (VR) technology provides an immersive experience to a certain extent, it is disconnected from the actual space and lacks real-time interaction capabilities, which cannot support designers to make instant adjustments and decisions in the actual environment. Augmented reality (AR) technology can overlay virtual elements in the actual space, but the interactivity of existing AR applications is relatively limited, lacks professional design tool support, and is difficult to meet the complex design process and multi-party collaboration needs.

[0005] Mixed reality (MR) technology provides designers and customers with a brand new solution through its powerful ability to integrate virtual and real things. Artificial intelligence and machine learning are increasingly used in the design field, and advances in cloud computing and real-time rendering technology provide support for high-quality visualization. The home improvement industry urgently needs a comprehensive design platform that can integrate these advanced technologies, provide an immersive experience, support real-time collaboration, and have intelligent assistance functions. Summary of the invention

[0006] In response to the above technical problems, the present invention provides a smart home improvement system based on mixed reality and an implementation method thereof. The system realizes the intelligence, visualization and collaboration of the design process by integrating advanced mixed reality technology, artificial intelligence algorithms and collaborative design tools.

[0007] The first embodiment of the present invention provides a smart home improvement system based on mixed reality, and the system includes: a data acquisition module, including a space scanning module, a 3D model generation module and a space feature recognition module, which is used to capture the real-time environment and generate real-time grids through the space scanning module, and build a 3D model through the 3D model generation module, and the space feature recognition module is used for accurate semantic segmentation to provide key information for subsequent design; an intelligent assistant module, including an AI-driven design suggestion module based on deep learning, an automatic layout optimization module, a personalized recommendation engine and a design rule checking module, which is used to analyze user preferences, budgets and lifestyles, achieve layout optimization, color matching and style recommendations, and recommend suitable furniture, decorations and materials; a multi-party collaboration module, including a cross-device synchronization module, a real-time multi-user interaction module, a remote collaboration support module and a role authority management module; a visualization presentation module, including a mixed reality rendering module, a real-time light and shadow simulation module, a material reality presentation module and a virtual image display module.

[0008] Optionally, the system also includes standardized interfaces and APIs for connecting to third-party systems.

[0009] Optionally, the spatial scanning module is a sensor and aiming point tracking system of the Quest 3 device.

[0010] Optionally, the personalized recommendation engine utilizes collaborative filtering and content-based algorithms to analyze user preferences, budgets, and lifestyles, and accurately recommends suitable furniture, decorations, and materials.

[0011] Optionally, the design rule checking module automatically verifies whether the recommended solution complies with building codes and safety standards, identifies potential conflicts and provides improvement suggestions.

[0012] Optionally, the intelligent assistant module also includes a natural speech processing module for responding to user voice commands and converting the voice commands into operation commands to achieve human-computer interaction.

[0013] Optionally, the mixed reality rendering module is used for real-time integration of virtual design elements and actual space, and supports multi-platform adaptive rendering.

[0014] A second embodiment of the present invention provides a smart home improvement method based on mixed reality, the method comprising any of the smart home improvement systems based on mixed reality described above, the method further comprising: a data acquisition module for real-time environment capture and real-time grid generation, and a 3D model is constructed through a 3D model generation module, and a spatial feature recognition module is used for accurate semantic segmentation; an intelligent assistant module analyzes user preferences, budgets and lifestyles, achieves layout optimization, color matching and style recommendations, and recommends suitable furniture, decorations and materials; a multi-party collaboration module is used for real-time data synchronization and collaborative editing across devices, allowing users to make design modifications and discussions in a virtual environment through natural interaction; a visualization presentation module integrates virtual design elements into the actual space through a mixed reality rendering module, and users view design details in the virtual mixed reality, or edit the scene layout in real time through intuitive gesture control.

[0015] Optionally, the method also includes: the intelligent assistant module also includes a natural speech processing module, which is used to respond to the user's voice commands, convert the voice commands into operation commands, and realize human-computer interaction.

[0016] The technical solution sensor and tracking system provided by the embodiment of the present invention realizes accurate environment capture, combines the optimization algorithm to quickly build a high-quality 3D model, and applies machine learning to improve the model. The intelligent assistant module integrates the AI-driven design suggestion module to provide real-time creative inspiration, layout optimization and personalized recommendations; the multi-party collaboration module is used for real-time interaction and collaboration in a virtual environment, supports multimodal interaction (such as gestures, sight and voice control), greatly improves collaboration efficiency and management capabilities, and improves decision-making efficiency. The visualization presentation module realizes the digitization and visualization of the entire process from spatial scanning to final rendering output, ensuring the accurate communication and execution of design intent. The present invention greatly improves design efficiency, shortens the design cycle, reduces design errors and rework, reduces project costs, improves design quality, realizes full visualization and traceability of the design process, and changes the working method of traditional home improvement design. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is a structural schematic diagram of a smart home improvement system based on mixed reality in the present invention.

[0018] Figure 2 The present invention is a flow chart of a smart home improvement method based on mixed reality. DETAILED DESCRIPTION

[0019] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0020] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "connected" and "connection" should be understood in a broad sense, for example, it can be directly connected, or indirectly connected through an intermediate medium, or it can be the internal connection of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances. In addition, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0021] Please refer to Figure 1 As shown, the present invention provides a smart home decoration system based on mixed reality, which aims to solve the problems of low efficiency, communication difficulties and insufficient customer participation in traditional home decoration design. The system includes a data acquisition module, an intelligent assistant module, a multi-party collaboration module and a visual presentation module. The data acquisition module includes a space scanning module, a 3D model generation module and a space feature recognition module, which is used to capture the real-time environment and generate real-time grids through the space scanning module, and build a 3D model through the 3D model generation module. The space feature recognition module is used for accurate semantic segmentation to provide key information for subsequent design; the intelligent assistant module includes an AI-driven design suggestion module based on deep learning, an automatic layout optimization module, a personalized recommendation engine and a design rule checking module, which is used to analyze user preferences, budgets and lifestyles, realize layout optimization, color matching and style recommendations, and recommend suitable furniture, decorations and materials; the multi-party collaboration module includes a cross-device synchronization module, a real-time multi-user interaction module, a remote collaboration support module and a role authority management module; the visual presentation module includes a mixed reality rendering module, a real-time light and shadow simulation module, a material reality presentation module and a virtual image display module. These four modules are tightly integrated to form a complete closed loop of the design process, realizing the intelligence and visualization of the entire process from spatial data acquisition to final design presentation.

[0022] In one embodiment of the present invention, the system further includes a standardized interface and API for connecting to a third-party system, and the spatial scanning module is a sensor and aiming point tracking system of the Quest 3 device.

[0023] As the basis of the intelligent home improvement design collaboration system based on mixed reality, the data acquisition module of the present invention integrates advanced spatial perception and 3D scanning technology. The system uses the sensors and aiming point tracking system of the Quest 3 device to achieve accurate environment capture and real-time mesh generation. By combining model optimization technology, the system can quickly build high-quality, texture-rich 3D models to ensure the integrity and accuracy of the 3D models. The spatial feature recognition function driven by deep learning realizes accurate semantic segmentation and provides key information for subsequent design. The system supports multimodal interaction and real-time editing, and realizes the seamless integration of virtual and reality through spatial mapping technology. In addition, the open architecture of the present invention allows the import of custom models and the integration of third-party resources, which greatly enhances the flexibility and application scope of the system.

[0024] The smart assistant module integrates advanced artificial intelligence and machine learning technologies to provide users with all-round design support. The AI-driven design suggestion module can provide creative inspiration such as layout optimization, color matching and style recommendation in real time. The automatic layout optimization module optimizes algorithms, space utilization and user preferences, intelligently adjusts furniture placement, and provides the best space solution. The system's personalized recommendation engine uses collaborative filtering and content-based algorithms to deeply analyze user preferences, budgets and lifestyles, and accurately recommends suitable furniture, decorations and materials. The innovative design rule check module automatically verifies whether the plan complies with building codes and safety standards, identifies potential conflicts and provides improvement suggestions. Through the natural language processing module, it is used to respond to user voice commands, convert voice commands into operation commands, and realize human-computer interaction. In addition, the integrated machine vision algorithm can extract design elements from inspiration pictures uploaded by users to further enrich the design plan. The smart design assistant continuously learns and adapts to user feedback, allowing users to instantly preview and experience design effects in the headset.

[0025] The multi-party collaboration module uses a head display as an interactive device and supports PC and mobile access. Through advanced network synchronization technology and distributed data processing, real-time data synchronization and collaborative editing across devices are achieved. Users can make design modifications and discussions in a virtual environment through natural interactions such as gestures, sight, and voice. The system supports virtual pointers, spatial markers, and real-time voice communications, and integrates an intelligent conference assistant function to automatically record discussion points and design decisions. Innovative role permission management and collaborative version control ensure the security and traceability of the collaboration process. The remote collaboration function breaks through geographical restrictions and supports remote teams to communicate instantly in the same virtual space. Real-time rendering technology and adaptive data transmission strategies ensure consistent visual experience and smooth collaboration on different devices. This module significantly improves design efficiency and communication quality, provides customers with a high sense of participation and decision transparency, and realizes a truly customer-centric collaborative design process.

[0026] As the core component of the system, the visualization module fully utilizes the advantages of the high-performance display and optical system of the head display to provide users with an immersive and highly realistic design experience. The system uses advanced mixed reality rendering technology to seamlessly integrate virtual design elements into the actual space. Users can freely explore design details in a fully immersive VR environment or in virtual mixed reality, or edit scene layouts in real time through intuitive gesture control. The multi-view preview function ensures that the design effect is fully displayed, and the optimized performance ensures a smooth interactive experience.

[0027] The present invention uses mixed reality technology to seamlessly integrate virtual design elements into the actual space, allowing users to experience and modify the design scheme in an immersive way. The present invention uses advanced sensors and tracking systems to achieve accurate environmental capture, combines optimization algorithms to quickly build high-quality 3D models, and applies machine learning to improve the model. The intelligent assistant module integrates an AI-driven design suggestion module to provide real-time creative inspiration, layout optimization and personalized recommendations, greatly improving design efficiency and quality. The multi-party collaboration module is used for real-time interaction and collaboration in a virtual environment, supports multi-modal interaction (such as gestures, sight and voice control), and is equipped with an intelligent conference assistant and a version control system, which greatly improves collaboration efficiency and management capabilities, breaks geographical restrictions, and improves decision-making efficiency. The present invention realizes the real-time integration of virtual design elements and actual space, supports multi-platform adaptive rendering, and ensures a consistent high-quality visual experience. The visualization presentation module realizes the digitization and visualization of the entire process from space scanning to final rendering output, ensuring the accurate communication and execution of design intent. The standardized interface and API (application programming interface) of the present invention can help the system to seamlessly exchange data with external software and services, expand system functions and promote industry collaboration. The present invention significantly improves the efficiency and accuracy of home improvement design, enhances customer engagement and satisfaction, and provides a new paradigm for the digital transformation of the entire home improvement industry.

[0028] Please refer to Figure 2 As shown, the present invention provides a smart home improvement method based on mixed reality, and the method comprises the following steps:

[0029] Step S10, the data acquisition module captures the real-time environment and generates the real-time mesh, and constructs the 3D model through the 3D model generation module. The spatial feature recognition module is used for accurate semantic segmentation to provide key information for subsequent design;

[0030] Step S20, the intelligent assistant module analyzes the user's preferences, budget and lifestyle, optimizes the layout, recommends color matching and style, and recommends suitable furniture, decorations and materials;

[0031] Step S30, the multi-party collaboration module is used for real-time data synchronization and collaborative editing across devices, allowing users to modify and discuss designs in a virtual environment through natural interaction;

[0032] Step S40: The visualization module integrates the virtual design elements into the actual space through the mixed reality rendering module. The user views the design details in the virtual mixed reality or edits the scene layout in real time through intuitive gesture control.

[0033] In one embodiment of the present invention, the method further comprises:

[0034] The intelligent assistant module also includes a natural speech processing module for responding to the user's voice commands, converting the voice commands into operation commands, and realizing human-computer interaction.

[0035] First, a high-precision 3D scan of the actual space is performed using a mixed reality head-mounted display device. This process not only captures the geometric information of the space, but also identifies the fixed structures, windows, doors, and existing furniture in the space through advanced computer vision algorithms. This information lays the foundation for subsequent virtual design and ensures that the virtual design fits the actual space perfectly. The head-mounted display scanning performed by the present invention using mixed reality technology provides a highly accurate spatial data foundation for subsequent design work. This three-dimensional scanning technology not only captures the geometric information of the space, but also identifies the fixed structures and material features in the space through advanced computer vision algorithms, providing a real environmental background for virtual design.

[0036] After the 3D scan is completed, designers and customers can "enter" this virtually reconstructed space. Through mixed reality technology, participants can feel as if they are actually there, and can intuitively feel the scale and atmosphere of the space. Designers can design in real time in this virtual environment, move virtual furniture, adjust materials and colors, and even change wall structures. Every change can be presented immediately, allowing customers to experience the design effect in real time. This instant feedback mechanism greatly improves decision-making efficiency and reduces rework caused by poor communication in traditional design. The "enter" step marks the transition of the design process from the traditional two-dimensional plane to an immersive three-dimensional experience. Designers and customers can experience the design plan immersively at the same time. This intuitive spatial perception greatly reduces communication errors and accelerates the decision-making process. In this virtual environment, participants can make design adjustments in real time, such as moving furniture, changing materials, adjusting light, etc., and every change can be presented immediately, allowing creativity and feedback to form an instant loop.

[0037] The multi-party collaboration module allows designers to invite owners or other stakeholders to join the virtual space at any time. Participants can use intuitive gestures and voice commands to interact with the environment, mark points of interest, and make modification suggestions. The system will record all interactions and decision-making processes in real time, forming a detailed design log for subsequent review and modification. This collaboration mode breaks the time and space limitations in traditional design, making remote collaboration as natural and efficient as face-to-face communication. This multi-party participation not only enriches the professionalism of the design, but also greatly improves the customer's sense of participation and the accuracy of decision-making. The system supports real-time voice communication, gesture annotation and virtual drawing, making remote collaboration as natural and efficient as face-to-face communication.

[0038] Throughout the process, the intelligent assistant module always runs in the background to provide support for design decisions. The intelligent assistant module analyzes the user's operation patterns, learns preferences, and provides suggestions when appropriate. For example, when the designer chooses a certain material, AI may suggest a matching color scheme or remind of possible budget overruns. This intelligent assistant module not only improves design quality, but also helps control project costs and time.

[0039] Through standardized data interfaces and APIs, the present invention can seamlessly connect to various third-party systems, such as material management software, budget calculation tools, or building information modeling (BIM) systems, ensuring the consistency of design information and avoiding data silos. It also provides possibilities for the digital transformation of the entire home improvement industry. Specifically, designers can directly call the latest data from the material library and calculate costs in real time; engineers can perform structural analysis to ensure the feasibility of the design; and they can even connect to smart home systems to consider future intelligent needs during the design phase.

[0040] The system provides a variety of rendering and output options. In addition to real-time rendering on mixed reality headsets, it also supports high-quality PC rendering for producing beautiful renderings. Lightweight viewers for web and mobile allow customers to view and share design plans at any time. The system can also automatically generate design description documents and project presentations, greatly reducing the paperwork of designers. The system's multi-platform rendering and output capabilities further enhance its practicality and flexibility. From highly immersive headset rendering, to sophisticated PC rendering, to convenient web and mobile access, the system ensures that design results can be effectively presented and used in a variety of scenarios. This not only meets the needs of professional display, but also provides comprehensive reference materials for subsequent construction, procurement and home life.

[0041] Finally, the system's output function converts the results of the entire design process into a variety of formats that can be directly applied. In addition to conventional pictures, documents and PPT, the system also supports the output of custom formats to meet the special needs of different projects and customers. This comprehensive output capability ensures that the design intent can be accurately communicated to the construction team, minimizing the information loss from design to implementation.

[0042] The present invention greatly improves design efficiency, shortens the design cycle, reduces design errors and rework, reduces project costs, improves design quality, realizes full visualization and traceability of the design process, and changes the working method of traditional home decoration design. It integrates the originally fragmented design links into a coherent and transparent collaborative process, greatly improving design efficiency and quality. Through mixed reality technology and intelligent collaboration platform, it eliminates the communication barriers between customers and designers, transforming home decoration design from a professional service to a co-creation experience. At the same time, its open ecosystem provides unlimited possibilities for innovation in the entire industry, and promotes home decoration design to develop in a more intelligent, personalized and sustainable direction. This design method is not only applicable to conventional residential decoration projects, but can also be extended to commercial space design, landscape design, and even urban planning. It represents an important direction for the digital transformation of the design industry and has the potential to reshape the future of the entire architecture and interior design industry.

[0043] The present invention can use VR technology to create a virtual design environment, or use mobile devices based on AR to create a virtual design environment, and even use a holographic projection solution. The spatial capture technology of the present invention can be replaced by image-based 3D reconstruction, portable laser scanning or deep camera array deployment. The multi-party collaboration module can be based on a cloud system, support asynchronous collaboration mode or be integrated into a social media platform. Rendering technology can pursue physical-based real-time rendering, cloud rendering services or stylized rendering effects. In terms of platform architecture, a closed ecosystem, a decentralized platform based on blockchain or a microservice architecture can be developed.

[0044] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A smart home improvement system based on mixed reality, characterized in that: The system comprises: Data acquisition module, including space scanning module, 3D model generation module and space feature recognition module, which is used for real-time environment capture and real-time mesh generation through the space scanning module, and building 3D models through the 3D model generation module. The space feature recognition module is used for accurate semantic segmentation to provide key information for subsequent design; Intelligent assistant module, including AI-driven design suggestion module based on deep learning, automatic layout optimization module, personalized recommendation engine and design rule checking module, which is used to analyze user preferences, budget and lifestyle, realize layout optimization, color matching and style recommendation, and recommend suitable furniture, decorations and materials; Multi-party collaboration module, including cross-device synchronization module, real-time multi-user interaction module, remote collaboration support module and role permission management module; The visualization presentation module includes a mixed reality rendering module, a real-time light and shadow simulation module, a material reality presentation module and a virtual image display module.

2. The smart home improvement system based on mixed reality according to claim 1, characterized in that: The system also includes standardized interfaces and APIs for connecting to third-party systems.

3. The smart home improvement system based on mixed reality according to claim 1, characterized in that: The spatial scanning module is the sensor and aiming point tracking system of the Quest 3 device.

4. The smart home improvement system based on mixed reality according to claim 1, characterized in that: The personalized recommendation engine uses collaborative filtering and content-based algorithms to analyze user preferences, budgets, and lifestyles, and accurately recommends suitable furniture, decorations, and materials.

5. The smart home improvement system based on mixed reality according to claim 1, characterized in that: The design rule checking module automatically verifies that recommended solutions comply with building codes and safety standards, identifies potential conflicts and provides suggestions for improvements.

6. The smart home improvement system based on mixed reality according to claim 1, characterized in that: The intelligent assistant module also includes a natural speech processing module for responding to the user's voice commands, converting the voice commands into operation commands, and realizing human-computer interaction.

7. The smart home improvement system based on mixed reality according to claim 1, characterized in that: The mixed reality rendering module is used for real-time integration of virtual design elements and actual space, and supports multi-platform adaptive rendering.

8. A smart home improvement method based on mixed reality, characterized in that: The method includes the mixed reality-based smart home improvement system according to any one of claims 1 to 7, and further includes: The data acquisition module captures the real-time environment and generates real-time meshes, and builds 3D models through the 3D model generation module. The spatial feature recognition module is used for accurate semantic segmentation; The smart assistant module analyzes user preferences, budget and lifestyle to optimize layout, recommend color matching and style, and recommend suitable furniture, decorations and materials; The multi-party collaboration module is used for real-time data synchronization and collaborative editing across devices, allowing users to modify and discuss designs in a virtual environment through natural interaction; The visualization presentation module integrates virtual design elements into the actual space through the mixed reality rendering module. Users can view the design details in virtual mixed reality or edit the scene layout in real time through intuitive gesture control.

9. The smart home improvement method based on mixed reality according to claim 8, characterized in that: The method further comprises: The intelligent assistant module also includes a natural speech processing module for responding to the user's voice commands, converting the voice commands into operation commands, and realizing human-computer interaction.

Images