A virtual scene control method and system based on MR large space

Abstract

The present invention proposes a virtual scene control method and system based on MR large space. It belongs to the field of mixed reality technology. It uses a spatial scanning device to scan the actual physical space, generate spatial mapping data, and initialize the basic framework of the virtual scene according to the spatial mapping data; based on the user's preset scene template and real-time sensor data, it dynamically generates elements in the virtual scene, and uses artificial intelligence algorithms to optimize the layout of scene elements; through a variety of interactive methods, the virtual scene is controlled in real time, and the user's interactive behavior is analyzed using a deep learning algorithm, and the user's intention is predicted based on the analysis results. Through a high-precision sensor array and advanced data processing technology, a virtual scene that is highly matched with the real environment can be created to provide users with an immersive experience.

Description

Technical Field

[0001] The present invention proposes a virtual scene control method and system based on MR large space, belonging to the technical field of mixed reality. Background Art

[0002] With the rapid development of virtual reality (VR) and mixed reality (MR) technologies, the construction and control of virtual scenes has become one of the core issues in this field. In existing technologies, the control of virtual scenes mainly relies on preset scenes and fixed interaction methods, which lacks flexibility and real-time performance, and is difficult to meet users' immersive experience needs in large spaces. Summary of the invention

[0003] The present invention provides a virtual scene control method and system based on MR large space, which is used to solve the problems mentioned in the above background technology:

[0004] The present invention proposes a virtual scene control method based on MR large space, the method comprising:

[0005] S1. Scan the actual physical space using a space scanning device to generate space mapping data, and initialize the basic framework of the virtual scene according to the space mapping data;

[0006] S2, based on the user's preset scene template and real-time sensor data, dynamically generate elements in the virtual scene, and use artificial intelligence algorithms to optimize the layout of scene elements;

[0007] S3, through a variety of interactive methods, real-time control of virtual scenes, use deep learning algorithms to analyze user interaction behaviors, and predict user intentions based on the analysis results;

[0008] S4. Users can interact collaboratively in the same virtual scene through the collaborative interaction module, and the consistency of each user's perspective and the real-time nature of the interaction are guaranteed through network synchronization technology;

[0009] S5. Use a high-performance graphics processing unit to render the scene, overlay the virtual scene into the actual physical space through the MR device, and dynamically adjust the resolution and frame rate of the scene according to the device performance and network bandwidth.

[0010] Furthermore, the S1 includes:

[0011] S11, through the high-precision sensor array, the physical space is scanned in all directions to obtain the original data of the physical space;

[0012] S12, performing time synchronization and spatial calibration on the collected raw data, eliminating errors between different sensors, and removing noise through filtering algorithms;

[0013] S13, segmenting and clustering the point cloud data using the point cloud library, extracting key features, and constructing the basic terrain grid of the virtual scene using 3D modeling software based on the pre-processed spatial data;

[0014] S14. Using the terrain texture system, texture is added to the virtual terrain according to the actual material of the physical space;

[0015] S15. Accurately model obstacles so that the obstacles in the virtual scene are consistent with the actual space;

[0016] S16. Automatically adjust the virtual scene boundary according to the user's activity range through boundary detection and dynamic adjustment mechanism.

[0017] Furthermore, the S2 includes:

[0018] S21. The user selects or customizes a template in the scene template library according to needs and builds a scene element library. Through AR marking technology, the user quickly generates scene elements by scanning specific markers;

[0019] S22, obtaining environmental data in real time through a sensor data interface, and dynamically adjusting the performance of elements in the virtual scene according to the environmental data;

[0020] S23, through the integration of IoT devices, the smart home status is reflected in the virtual scene in real time, the user's historical behavior data is analyzed through deep learning algorithms, and user preferences are predicted based on the analysis results;

[0021] S24. Combine genetic algorithm or simulated annealing algorithm to intelligently layout scene elements and optimize visual effects and functionality;

[0022] S25. Through the user feedback mechanism, users are allowed to fine-tune the layout results and continuously optimize the AI ​​algorithm.

[0023] Furthermore, the S3 includes:

[0024] S31, constructing an interactive interface to integrate multiple interactive modes, including gesture recognition, voice recognition, and eye tracking; and unifying the interactions of different MR devices through a cross-platform compatible interactive engine;

[0025] S32, automatically adjust the interactive options according to the user's current location and scene elements through context-aware algorithms, and perform time series analysis of user interaction behaviors through the Transformer model;

[0026] S33. Construct an intent prediction model based on context information; and update the intent prediction model in real time based on user feedback and new interaction data.

[0027] Further, the S32 includes:

[0028] Use MR devices to capture the precise coordinates, orientation angles, and surrounding environment data of the user's current position in real time, and synchronize the collected scene element information;

[0029] Analyze the spatial relationship between the user's current location and scene elements, and dynamically adjust the visibility and accessibility of interactive options based on the spatial relationship;

[0030] Prioritize scene elements based on the user's historical behavior, current task goals, and the characteristics of scene elements; place high-priority elements in the center of the user's field of vision or in a more accessible location;

[0031] Convert the user's interactive behavior into time series data and use the Transformer model to model the time series of the user's interactive behavior;

[0032] Use the self-attention mechanism to capture the dependencies between different time steps and train the model through historical interaction data;

[0033] Use the trained Transformer model to analyze the user's current interactive behavior in real time and identify the user's next action or potential needs;

[0034] Combine contextual information with the prediction results of the Transformer model to build an intent prediction model, and dynamically adjust the interactive interface and options based on the prediction results;

[0035] Design a user feedback mechanism to collect user feedback on the interaction experience, and update and optimize the intent prediction model in real time based on user feedback and new interaction data.

[0036] Furthermore, the S4 includes:

[0037] S41. Through the collaborative interaction framework based on distributed systems and the adoption of microservice architecture, different functional modules are split into independent services. Each independent service interacts with each other through a lightweight communication protocol and performs fine-grained role division, assigning different roles according to the user's responsibilities and permissions in the virtual scene.

[0038] S42. Through the permission verification mechanism, when a user initiates an interaction request, their role and permission are verified, and a dynamic permission adjustment function is introduced to allow administrators to adjust the user's permissions according to real-time conditions; through a collaborative tool set and an interactive protocol, multiple users can interact with each other;

[0039] S43. Through network transmission technology, the user's perspective and interactive data are synchronized with low latency; through the network quality monitoring mechanism, network indicators are monitored in real time; and the transmission strategy is dynamically adjusted according to the network status;

[0040] S44. Adopt optimistic concurrency control and pessimistic concurrency control strategies, and manage data changes in combination with version control mechanisms;

[0041] S45. When a data conflict is detected through the conflict detection and resolution algorithm, the conflict is automatically resolved according to the preset conflict resolution rules;

[0042] S46. Synchronize the state through state snapshot, incremental update or hybrid mode; based on the state rollback function, when an unresolvable conflict or user error is detected, allow the system to roll back to the last stable state;

[0043] S47. When the user performs an interactive operation, the operation result and state change are fed back through a real-time feedback mechanism; and errors are handled based on the error handling process;

[0044] S48. Through the scene element locking mechanism, when a user is editing a scene element, the element is automatically locked. Through the element unlocking function, when the user completes editing and saves, the element is automatically unlocked to allow other users to continue editing.

[0045] Furthermore, the S41 includes:

[0046] Identify the core design principles of distributed systems and introduce containerization technology and orchestration tools to automate the deployment and management of services;

[0047] Design a distributed data storage solution and introduce a caching mechanism;

[0048] Analyze business requirements and divide the system into multiple independent functional modules, each of which is responsible for a single responsibility;

[0049] Encapsulate each functional module as an independent microservice, and deploy and scale the service independently on demand;

[0050] Inter-service interaction is carried out through the RESTful API lightweight communication protocol. Multiple roles are defined according to the user's responsibilities and permissions in the virtual scene; specific permissions and operation scopes are assigned to each role.

[0051] Furthermore, the S5 includes:

[0052] S51, GPU-based rendering engine, which accelerates the rendering process of complex scenes through parallel computing on GPU, and dynamically adjusts the rendering accuracy according to the distance and importance of scene elements through level of detail technology;

[0053] S52, through the scene segmentation and asynchronous loading mechanism, the large scene is divided into multiple small blocks, loaded and rendered on demand, and the rendering parameters are dynamically adjusted according to the device performance, network bandwidth and user settings based on the intelligent resource management system;

[0054] S53, monitor device performance and user interaction behavior in real time, predict future performance requirements through machine learning algorithms, and preload resources and optimize performance in advance;

[0055] S54. Users can adjust the rendering effect according to their needs and preferences through configurable rendering setting options, and record and analyze the performance indicators of the rendering process through performance monitoring and reporting tools, and make further optimizations based on the analysis results.

[0056] Furthermore, the S52 includes:

[0057] The complexity of the entire scene is analyzed through AI algorithms. Based on the scene complexity evaluation results, adaptive meshing is used to divide the large scene into multiple independent and interrelated small blocks.

[0058] Assign priority to each scene block based on the camera position, user line of sight, distance and importance of scene elements.

[0059] Combined with user movement path prediction and user behavior pattern analysis, the scene blocks that the user is about to enter are loaded in advance. For network applications, streaming transmission technology is used to transmit necessary data blocks in real time according to the user's current field of view, while releasing no longer needed memory resources;

[0060] Through the intelligent cache mechanism, the cache size and content are dynamically adjusted according to the access frequency and importance of the scene blocks, and the GPU model, CPU performance, memory size and network bandwidth of the device are monitored in real time to automatically adjust the rendering parameters;

[0061] Learn user preferences through machine learning algorithms to provide each user with a personalized rendering experience and dynamically adjust resource loading priorities based on current scene requirements;

[0062] Through monitoring tools, key performance indicators are recorded in real time, and rendering strategies are dynamically adjusted based on performance monitoring results.

[0063] The present invention proposes a virtual scene control system based on MR large space, the system comprising:

[0064] Space scanning module: uses space scanning equipment to scan the actual physical space, generates space mapping data, and initializes the basic framework of the virtual scene according to the space mapping data;

[0065] Optimized layout module: Based on the user's preset scene templates and real-time sensor data, it dynamically generates elements in the virtual scene and optimizes the layout of the scene elements using artificial intelligence algorithms;

[0066] Intention prediction module: Through various interactive methods, the virtual scene is controlled in real time, the user's interactive behavior is analyzed using deep learning algorithms, and the user's intention is predicted based on the analysis results;

[0067] Collaborative interaction module: Users can interact collaboratively in the same virtual scene through the collaborative interaction module, and the consistency of each user's perspective and the real-time nature of the interaction are guaranteed through network synchronization technology;

[0068] Scene rendering module: Use a high-performance graphics processing unit to render the scene, overlay the virtual scene into the actual physical space through the MR device, and dynamically adjust the resolution and frame rate of the scene according to the device performance and network bandwidth.

[0069] The beneficial effects of the present invention are as follows: through high-precision sensor arrays and advanced data processing technology, a virtual scene that is highly matched with the real environment can be created to provide users with an immersive experience; based on user presets and real-time data, elements in the virtual scene are dynamically generated and adjusted to ensure the real-time and personalization of the scene to meet the needs of different users; the integration of multiple interaction methods and the application of deep learning algorithms make the interaction between users and the virtual environment more intuitive and natural, while being able to predict user intentions and enhance user experience; multiple users work together in the same virtual scene, and network synchronization technology is used to ensure the real-time interaction and consistency of perspective, which is suitable for remote collaboration and team projects; high-performance graphics processing units and intelligent resource management ensure high-quality rendering effects, while dynamically adjusting according to device performance and network conditions to ensure a smooth user experience; through learning from user behavior and analyzing scene complexity, the system can adaptively adjust rendering and loading strategies to provide a customized virtual experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 It is a step diagram of the method of the present invention;

[0071] Figure 2 This is a system module diagram of the present invention. DETAILED DESCRIPTION

[0072] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0073] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. The embodiments described are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0074] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0075] One embodiment of the present invention, as Figure 1 As shown, a virtual scene control method based on MR large space, the method comprising:

[0076] S1. Use a space scanning device to scan the actual physical space, generate space mapping data, and initialize the basic framework of the virtual scene according to the space mapping data; including terrain, obstacles and boundaries.

[0077] S2, based on user-preset scene templates and real-time sensor data, dynamically generate elements in the virtual scene, such as buildings, vegetation, and props, and use artificial intelligence algorithms to optimize the layout of scene elements;

[0078] S3, through a variety of interactive methods, including gesture recognition, voice recognition and eye tracking, to control the virtual scene in real time, use deep learning algorithms to analyze the user's interactive behavior, and predict the user's intention based on the analysis results;

[0079] S4. Users can interact collaboratively in the same virtual scene through the collaborative interaction module, and the consistency of each user's perspective and the real-time nature of the interaction are guaranteed through network synchronization technology;

[0080] S5. Use high-performance graphics processing units (GPUs) to render scenes, overlay virtual scenes into actual physical spaces through MR devices, and dynamically adjust the resolution and frame rate of scenes based on device performance and network bandwidth.

[0081] The working principle of the above technical solution is: use spatial scanning equipment (such as laser radar, depth camera, etc.) to perform three-dimensional scanning of the actual physical space to capture geometric information such as terrain, obstacles and boundaries in the space; generate spatial mapping data based on the spatial data obtained by scanning. These data are used to initialize the basic framework of the virtual scene to ensure that the virtual scene can accurately reflect the structure and layout of the real space; combine the user's preset scene template (such as urban landscape, forest environment, etc.) and real-time sensor data (such as weather, time changes, etc.) to dynamically generate elements in the virtual scene, such as buildings, vegetation, roads, props, etc.; use artificial intelligence algorithms to optimize the layout of the generated scene elements to ensure that the scene not only conforms to physical laws, but also has visual rationality and aesthetics. This includes considerations of factors such as the distance, orientation, and lighting effects between elements. Provide multiple interaction methods such as gesture recognition, voice recognition, and eye tracking, so that users can interact with virtual scenes in a natural and intuitive way; use deep learning algorithms to analyze users' interactive behaviors and identify users' operating intentions and preferences. By learning from historical data, the system can more accurately predict the user's next intention, thereby providing a more intelligent interactive experience; support multi-user collaborative interaction in the same virtual scene, such as co-editing scenes, role-playing, etc.; ensure the consistency of each user's perspective and the real-time nature of the interaction through network synchronization technology. Even users in different geographical locations can feel the interactive experience as if they were in the same space; use high-performance graphics processing units (GPUs) to render virtual scenes in real time to ensure high definition and smoothness of the scenes; dynamically adjust the resolution and frame rate of the scene according to the performance of the MR device and the actual network bandwidth. When the device performance is insufficient or the network bandwidth is limited, the continuity and stability of the interaction are guaranteed by reducing the resolution or frame rate.

[0082] The effect of the above technical solution is: the actual physical space data obtained by the space scanning device can generate high-precision space mapping data. This provides a solid foundation for the basic framework of the virtual scene, so that the virtual scene can highly restore the terrain, obstacles and boundaries of the real space, and enhance the user's immersion; based on the user's preset scene template and real-time sensor data, the system can dynamically generate elements in the virtual scene, such as buildings, vegetation and props. This flexibility allows users to quickly build and adjust virtual scenes according to actual needs; using artificial intelligence algorithms to optimize the layout of scene elements not only ensures the rationality of the scene (such as avoiding collisions, maintaining balance, etc.), but also improves the aesthetics of the scene. This intelligent processing greatly reduces the user's manual adjustment work and improves the efficiency and quality of scene creation; provides gesture recognition, voice recognition and eye tracking and other interactive methods, so that users can interact with virtual scenes in a more natural and intuitive way. This diversified interaction method enhances the user's experience and sense of participation; uses deep learning algorithms to analyze the user's interactive behavior and predicts the user's intention based on the analysis results. This intelligent prediction function enables the system to respond to user needs in advance and provide more personalized services; it supports multiple users to interact collaboratively in the same virtual scene, such as jointly editing scenes and role-playing. This collaborative interaction function promotes cooperation and communication between users and improves the efficiency of team work; it ensures the consistency of each user's perspective and the real-time nature of interaction through network synchronization technology. Even users in different geographical locations can feel the interactive experience as if they are in the same space, which enhances the closeness and coherence of team collaboration; it uses high-performance graphics processing units (GPUs) for scene rendering to ensure high definition and smoothness of virtual scenes. This high-quality rendering effect provides users with a more realistic visual experience; it dynamically adjusts the resolution and frame rate of the scene according to the performance of the MR device and the actual network bandwidth. This dynamic adjustment function ensures that users can obtain a stable and smooth interactive experience in different environments, improving the adaptability and stability of the system.

[0083] In one embodiment of the present invention, the S1 includes:

[0084] S11, through a high-precision sensor array, including LiDAR, depth camera, RGB camera, environmental sensor, etc., the physical space is scanned in all directions to obtain the original data of the physical space;

[0085] S12, performing time synchronization and spatial calibration on the collected raw data, eliminating errors between different sensors, and removing noise through filtering algorithms (such as Kalman filtering);

[0086] S13, using a point cloud library (such as PCL) to segment and cluster the point cloud data, extract key features, the key features including terrain and obstacles; based on the pre-processed spatial data, using 3D modeling software to construct a basic terrain grid for the virtual scene;

[0087] S14. Add textures to the virtual terrain based on the actual material of the physical space (such as grass, sand, and cement) through the terrain texture system;

[0088] S15. Accurately model obstacles, including shape, size, position, etc., so that obstacles in the virtual scene are consistent with the actual space;

[0089] S16. Automatically adjust the virtual scene boundary according to the user's activity range through boundary detection and dynamic adjustment mechanism.

[0090] The working principle of the above technical solution is: use a high-precision sensor array including LiDAR (laser radar), depth camera, RGB camera and environmental sensor to scan the physical space in all directions. These sensors can capture the three-dimensional geometric information, color information and environmental parameters (such as light, temperature, etc.) of the space; the raw data of the physical space obtained by the above sensors is the basis for building the virtual scene, which contains various details and features of the physical space; due to the use of multiple sensors, there may be time differences and position deviations between them. Therefore, it is necessary to synchronize the collected raw data in time and space to eliminate these errors; process the data through filtering algorithms (such as Kalman filtering) to remove noise and unnecessary interference information and improve data quality; use point cloud libraries (such as PCL) to further process the point cloud data, including segmentation, clustering and other steps to extract key features (such as terrain, obstacles); based on the pre-processed spatial data, use 3D modeling software to build the basic terrain grid of the virtual scene. This step is the key step to convert the three-dimensional information of the physical space into a three-dimensional model in the virtual scene; in order to enhance the realism of the virtual scene, it is necessary to add corresponding textures to the virtual terrain according to the actual materials of the physical space (such as grass, sand, and cement). This includes selecting appropriate texture images and applying them to the virtual terrain to achieve a visual effect consistent with the physical space; accurately modeling obstacles in the physical space, including the extraction and reconstruction of information such as shape, size, and position. This step is the key to ensuring that obstacles in the virtual scene are consistent with the actual space, which helps to enhance the user's immersion and interactive experience; the boundaries of the physical space are automatically identified through algorithms, and the corresponding boundaries are set in the virtual scene accordingly. This helps prevent users from "walking out" of the scene range during the interaction process; dynamically adjust the boundaries of the virtual scene according to changes in the user's activity range. This mechanism ensures that no matter how the user moves, they can stay within the effective range of the virtual scene, thereby maintaining the continuity and stability of the interaction.

[0091] The effects of the above technical solutions are: by integrating multiple high-precision sensors such as LiDAR, depth camera, RGB camera and environmental sensor, it is possible to achieve all-round and high-precision scanning of the physical space. This ensures that the acquired raw data of the physical space is highly accurate and rich, providing a solid foundation for subsequent processing and modeling; the collected raw data is synchronized in time and space, eliminating the errors between different sensors and ensuring the consistency and accuracy of the data; the noise is removed by using filtering algorithms (such as Kalman filtering), which effectively improves the data quality and reduces the interference factors in subsequent processing; the point cloud data is efficiently processed using a point cloud library (such as PCL), including segmentation, clustering and other steps, to quickly extract key features (such as terrain, obstacles). This not only improves processing efficiency, but also ensures that the extracted features are accurate and reliable; based on the pre-processed spatial data, the basic terrain grid of the virtual scene is constructed using 3D modeling software. This process is highly automated and can quickly generate a virtual terrain that is highly consistent with the physical space; the texture of actual materials, such as grass, sand, concrete, etc., is added to the virtual terrain through the terrain texture system. This greatly enhances the realism of the virtual scene, allowing users to have a more realistic visual experience during the interaction process; accurately model obstacles in the physical space, including accurate restoration of information such as shape, size, and position. This helps ensure that the user's interactive experience in the virtual scene is consistent with the experience in the physical space, improving the user's sense of security and immersion; through boundary detection and dynamic adjustment mechanisms, the boundaries of the virtual scene are automatically adjusted according to the user's activity range. This flexibility ensures that no matter how the user moves, they can stay within the effective range of the virtual scene, avoiding the situation where the user "walks out" of the scene, thereby ensuring the continuity and stability of the interaction.

[0092] In one embodiment of the present invention, the S16 includes:

[0093] Integrate user location tracking systems (such as GPS, indoor positioning technology), user behavior analysis modules (monitoring movements, speed, etc. through sensors), and dynamic environmental change monitoring (such as lighting and weather simulation), and integrate multi-source data through data fusion algorithms (such as Bayesian networks and deep learning models);

[0094] Based on historical data and current user behavior, machine learning algorithms (such as LSTM and GRU) are used to predict the user's possible future movement paths and stay areas.

[0095] Based on the prediction results, the temporary boundaries of the virtual scene are automatically planned. According to the user's current position, sight direction and predicted path, the content in the virtual scene is prioritized, and the area that the user is about to enter or pay attention to is loaded and rendered first;

[0096] Through the real-time loading and unloading mechanism of scene content, when the user moves to a new area, the content of the old area is unloaded and the content of the new area is quickly loaded, which reduces memory usage and improves performance.

[0097] Dynamically adjust environmental parameters in the virtual scene, such as lighting, shadows, clouds, etc., based on actual weather data (such as API acquisition) or user preferences. Adjust the position, size or properties of obstacles in real time based on the user's interaction with the virtual environment (such as collision detection, physical interaction, etc.);

[0098] Through questionnaire surveys, behavioral data analysis and other methods, we regularly collect user feedback on virtual scene boundary adjustment, content loading speed, environmental realism, etc. Based on user feedback, we use A / B testing to continuously optimize boundary detection algorithms, scene loading strategies and environmental adaptability adjustment mechanisms.

[0099] The working principle of the above technical solution is: by integrating multi-source data such as user location tracking system (such as GPS, indoor positioning technology), user behavior analysis module (monitoring movement, speed, etc.) and dynamic environmental change monitoring (such as lighting, weather simulation), and integrating these data using data fusion algorithms (such as Bayesian network, deep learning model). This step ensures that the system can obtain comprehensive and accurate user and environmental information; based on historical data and user current behavior, the user's possible future movement path and stay area are predicted through machine learning algorithms (such as LSTM, GRU). This predictive ability enables the system to perceive user needs in advance and make corresponding adjustments; based on the prediction results, the system automatically plans the temporary boundaries of the virtual scene. These boundaries are designed to cover the user's main activity area while maintaining a reasonable range of the scene to avoid unnecessary resource loading. By adjusting the boundaries in real time, the system can ensure that users can get the best interactive experience at any location; the content in the virtual scene is prioritized according to the user's current location, line of sight direction and predicted path. Prioritize loading and rendering the area that the user is about to enter or pay attention to, thereby improving the efficiency of resource utilization; when the user moves to a new area, the system will unload the content of the old area and quickly load the content of the new area. This real-time loading and unloading mechanism reduces memory usage and improves system performance; dynamically adjusts environmental parameters (such as lighting, shadows, clouds, etc.) in the virtual scene according to actual weather data (such as obtained through API) or user preferences. This dynamic adaptability enables the virtual scene to more realistically reflect changes in the real world; adjust the position, size or attributes of obstacles in real time according to the interaction between the user and the virtual environment (such as collision detection, physical interaction, etc.). This instant feedback mechanism enhances the interactivity between the user and the virtual environment; regularly collect user feedback on virtual scene boundary adjustment, content loading speed, environmental realism, etc. through questionnaires, behavioral data analysis, etc.; based on user feedback, use A / B testing and other methods to continuously optimize boundary detection algorithms, scene loading strategies and environmental adaptability adjustment mechanisms. This continuous optimization mechanism enables the system to continuously adapt to changes in user needs and realize personalized and intelligent virtual scene construction.

[0100] The effects of the above technical solutions are as follows: by integrating user location tracking and behavior analysis modules, combined with machine learning algorithms to predict the user's future path, the system can respond to the user's movement in real time, load and render the area that the user is about to enter in advance, greatly reducing loading delays and improving the smoothness of the user experience; the automatically planned temporary boundaries cover the user's main activity area, while avoiding unnecessary resource loading, ensuring that the user always interacts in the most optimized virtual environment; dynamically adjusting environmental parameters according to actual weather data or user preferences, and adjusting obstacles in real time according to user interactions, making the virtual scene closer to reality and improving the naturalness and immersion of the interactive experience; the real-time loading and unloading mechanism of scene content effectively reduces memory usage and improves system performance. The content of a certain area is loaded only when the user is about to enter or pay attention to it, thus avoiding unnecessary waste of resources; based on historical data and predictions of the user's current behavior, the system can prepare and optimize the resources that the user may need in advance, further improving the efficiency of resource utilization; the system can adjust environmental parameters and obstacle attributes according to user preferences to achieve personalized customization of virtual scenes and meet the personalized needs of different users; by regularly collecting user feedback and continuously optimizing boundary detection algorithms, scene loading strategies and environmental adaptability adjustment mechanisms using methods such as A / B testing, the system can continuously learn and improve and become more intelligent; real-time adjustment of environmental parameters and obstacle positions makes the user's interaction with the virtual environment more natural and smooth, reducing the sense of abruptness and unreality; through intelligent loading and predictive optimization, the user's waiting time for loading is reduced, avoiding user distraction or dissatisfaction caused by loading delays; the entire optimization process is based on a large amount of user data and real-time environmental data, making decisions more scientific and reasonable. Through data analysis, the system can accurately identify user needs and problems, and optimize them in a targeted manner.

[0101] In one embodiment of the present invention, the S2 includes:

[0102] S21. Users select or customize templates in the scene template library according to their needs, and build a scene element library, including buildings, vegetation, props and other types. Each type contains multiple styles and material options. Through AR marking technology, users can quickly generate scene elements by scanning specific markers;

[0103] S22. Obtain environmental data in real time (such as light intensity, temperature and humidity, and weather conditions) through the sensor data interface, and dynamically adjust the performance of elements in the virtual scene according to the environmental data; for example, adjust the color and brightness of vegetation according to weather changes to simulate rain and snow effects.

[0104] S23, through the integration of IoT devices, the smart home status is reflected in the virtual scene in real time, the user's historical behavior data is analyzed through deep learning algorithms (such as convolutional neural network CNN), and user preferences are predicted based on the analysis results;

[0105] S24. Combine genetic algorithm or simulated annealing algorithm to intelligently layout scene elements and optimize visual effects and functionality;

[0106] S25. Through the user feedback mechanism, users are allowed to fine-tune the layout results and continuously optimize the AI ​​algorithm.

[0107] The working principle of the above technical solution is as follows: the user first selects an existing template as a basis in the scene template library according to the needs, or completely customizes a new template; the user then builds a scene element library, which contains various types of elements such as buildings, vegetation, props, etc., and each type contains multiple styles and material options. Through AR marking technology, users can quickly scan specific tags to generate the required scene elements, which greatly improves the efficiency and convenience of scene construction; through the sensor data interface, the system obtains data such as light intensity, temperature and humidity, and weather conditions in the environment in real time; according to the acquired environmental data, the system dynamically adjusts the performance of elements in the virtual scene. For example, the color and brightness of vegetation are adjusted according to weather changes, rain and snow effects are simulated, etc., so that the virtual scene is closer to the real environment; through IoT device integration, the system reflects the status of smart home in real time in the virtual scene. In this way, users can not only control smart home devices in the physical space, but also intuitively see the changes in device status in the virtual scene; use deep learning algorithms (such as convolutional neural network CNN) to analyze user historical behavior data, and predict user preferences based on the analysis results. This predictive capability helps the system better understand user needs and provide more personalized services; combined with optimization algorithms such as genetic algorithms or simulated annealing algorithms, the system intelligently lays out scene elements. These algorithms continuously adjust the position and size of elements through iterative calculations to optimize visual effects and functionality; the optimization goals usually include improving the aesthetics of the scene, enhancing the convenience of user interaction, and ensuring the coordination between scene elements; through the user feedback mechanism, the system allows users to fine-tune the layout results and make suggestions for improvement. This feedback mechanism enables the system to continuously collect user opinions and make targeted improvements; based on user feedback and data analysis of layout results, the system continuously optimizes the AI ​​algorithm. This continuous optimization mechanism helps to improve the accuracy and efficiency of the algorithm, allowing the system to better meet user needs.

[0108] The effects of the above technical solutions are as follows: users can select or customize templates according to their needs, and build a scene element library containing a variety of styles and material options, which provides users with great flexibility and personalized space, making the virtual scene more in line with the user's aesthetics and usage habits; by acquiring environmental data in real time, the system can dynamically adjust the performance of elements in the virtual scene, such as adjusting the color and brightness of vegetation according to weather changes, simulating rain and snow effects, etc., making the virtual scene closer to the real environment, and improving the user's immersion and experience quality; the smart home status is reflected in the virtual scene in real time, so that users can directly control and manage smart home devices in the virtual environment, improving the convenience and intelligence level of home life; combined with optimization algorithms such as genetic algorithms or simulated annealing algorithms, the system can intelligently layout scene elements and optimize visual effects and functionality. This intelligent layout not only improves the beauty of the scene, but also enhances the convenience and comfort of user interaction; by analyzing user historical behavior data through deep learning algorithms, the system can predict user preferences, thereby providing more personalized services and recommendations. This predictive ability helps to improve user satisfaction and loyalty; using AR tagging technology, users can quickly generate scene elements by scanning specific tags, which greatly improves the efficiency and convenience of scene construction. Users can easily build and adjust virtual scenes without complicated operations or professional skills; the system can dynamically adjust the performance of elements in the virtual scene based on real-time environmental data. This dynamic adjustment mechanism makes the virtual scene more flexible and practical, and can adapt to changes in different scenes and environments; through the user feedback mechanism, the system can collect user feedback on layout results and AI algorithms, so as to make targeted improvements and optimizations. This continuous improvement mechanism helps to improve the performance and user experience of the system, maintain the competitiveness and market position of the system; based on user feedback and data analysis results, the system can continuously optimize the AI ​​algorithm and improve the accuracy and efficiency of the algorithm. This optimization process is a virtuous circle, which enables the system to continuously learn and improve, and provide users with better services and experiences.

[0109] In one embodiment of the present invention, the S23 includes:

[0110] Through the smart device discovery protocol, it automatically scans and identifies smart home devices connected to the network (such as smart light bulbs, temperature control systems, security cameras, etc.) and completes the initial configuration;

[0111] Build a unified data exchange standard so that the virtual scene system can receive status data (such as switch status, brightness level, temperature value, etc.) from smart home devices in real time and accurately;

[0112] Collect user data through multiple channels, including user interaction logs and sensor data (such as human body sensors and environmental sensors), and pre-process the collected user data;

[0113] Based on the preprocessed data, we extract features that are valuable for predicting user behavior, such as time patterns, device usage frequency, and environmental variables. We use convolutional neural networks to capture the timing and complexity of user behavior.

[0114] The model is trained through historical behavior data, the model performance is evaluated through cross-validation, and the model parameters are continuously adjusted to optimize the prediction accuracy. The trained deep learning model is deployed into the system to make real-time predictions of the user's current and future behavior.

[0115] Based on the prediction results, the performance of smart home elements in the virtual scene is automatically adjusted; for example, when it is predicted that the user is about to return home, the lights are turned on in advance and the indoor temperature is adjusted to a comfortable range; or the playlist of the audio system is automatically adjusted according to the user's entertainment preferences.

[0116] Collect user feedback on the effects of scene adaptation through user feedback mechanisms (such as satisfaction surveys and behavior feedback buttons). Combine user feedback with actual effect evaluation to regularly re-evaluate the predictive performance of the deep learning model, and adjust the model structure or retrain as needed.

[0117] The working principle of the above technical solution is as follows: through this protocol, the system can automatically scan and identify smart home devices connected to the network, such as smart bulbs, temperature control systems, security cameras, etc. This process includes the discovery, identification and automatic processing of the initial configuration of the device to ensure that the device can be successfully connected to the system and work normally; build a set of unified data exchange standards so that the virtual scene system can receive status data from different smart home devices in real time and accurately. These data include the switch status, brightness level, temperature value, etc. of the device, which are important bases for subsequent user behavior analysis and prediction; collect user data through multiple channels such as user interaction logs and sensor data (such as human body sensors and environmental sensors). These data cover multiple aspects such as users' daily behavior and environmental changes, providing a rich data source for subsequent data analysis and prediction; pre-process the collected user data, including data cleaning, denoising, format unification and other steps to ensure data quality and accuracy. This is the basis for subsequent feature extraction and model training. Based on the preprocessed data, extract valuable features for user behavior prediction, such as time pattern (regularity of user's daily activities), device usage frequency (user's preference for specific devices), environmental variables (light, temperature and humidity, etc.). Use deep learning technologies such as convolutional neural network (CNN) to capture the timing and complexity of user behavior. Real-time prediction of user's current and future behavior is achieved by building a suitable network structure and parameter settings. Use historical behavior data to train the model and evaluate the model performance through cross-validation. According to the evaluation results, the model parameters are continuously adjusted to optimize the prediction accuracy. Finally, the trained deep learning model is deployed to the system for practical application. According to the prediction results of the deep learning model, the performance of smart home elements in the virtual scene is automatically adjusted. For example, when it is predicted that the user is about to go home, turn on the lights in advance and adjust the indoor temperature to a comfortable range; or automatically adjust the playlist of the sound system according to the user's entertainment preferences. This adaptive adjustment makes the virtual scene closer to the actual needs and living habits of users; collect user feedback on the scene adaptive effect through user feedback mechanisms (such as satisfaction surveys and behavior feedback buttons). These feedbacks are important bases for evaluating model performance and improving models. The prediction performance of deep learning models is regularly re-evaluated based on user feedback and actual effect evaluation results. The model structure is adjusted or the model is retrained based on the evaluation results to optimize its performance. This continuous optimization mechanism ensures that the model can adapt to changing user needs and environmental changes.

[0118] The effect of the above technical solution is: through the smart device discovery protocol, smart home devices can automatically scan, identify and complete the initial configuration without manual settings by users, which greatly improves the access speed and convenience of the devices. This seamless integration makes the smart home system easier to use and maintain; based on the real-time collected smart home device status data and user behavior prediction results, the system can automatically adjust the performance of smart home elements in the virtual scene. For example, turning on the lights in advance, adjusting the indoor temperature, etc., these intelligent adjustments provide users with a more comfortable and personalized living environment and improve the living experience; by capturing the timing and complexity of user behavior through deep learning technologies such as convolutional neural networks, the system can accurately predict the user's current and future behavior. This predictive ability enables the system to prepare in advance to meet user needs, improve the initiative and response speed of the service; based on the prediction results, the system can automatically adjust the settings of smart home devices to optimize the user's living experience. For example, automatically adjusting the playlist of the audio system according to the user's entertainment preferences, this personalized service enhances user satisfaction and loyalty; collecting user data through multiple channels such as user interaction logs and sensor data provides a rich data source for the system. These data not only help improve the accuracy of predictions, but also provide strong support for the continuous optimization of the system; collect user feedback on the effect of scene adaptation through the user feedback mechanism, and regularly re-evaluate the prediction performance of the deep learning model based on the actual effect evaluation results. This feedback and evaluation mechanism ensures that the system can continuously learn and improve, maintain its competitiveness and adaptability; building a unified data exchange standard enables the system to easily access new smart home devices and receive their status data in real time. This flexibility and scalability provides broad space and possibilities for the future development of the system; through intelligent prediction and automatic adjustment of the settings of smart home devices, the system can achieve energy conservation and emission reduction while meeting user needs. For example, when it is predicted that the user is about to leave the room, the lights and air conditioners are automatically turned off. This intelligent management helps to reduce energy consumption and carbon emissions.

[0119] In one embodiment of the present invention, S3 includes:

[0120] S31, constructing an interactive interface to integrate multiple interactive modes, including gesture recognition, voice recognition, and eye tracking; and unifying the interactions of different MR devices through a cross-platform compatible interactive engine;

[0121] S32, automatically adjust the interactive options according to the user's current location and scene elements through context-aware algorithms, and perform time series analysis of user interaction behaviors through the Transformer model;

[0122] S33. Build an intent prediction model based on contextual information (such as scene elements, user location, and historical behavior); and update the intent prediction model in real time based on user feedback and new interaction data.

[0123] The working principle of the above technical solution is as follows: the system builds an interactive interface that integrates multiple interactive methods such as gesture recognition, voice recognition, and eye tracking. This multimodal interactive method allows users to choose the most suitable interactive means according to their habits and preferences, which improves the flexibility and convenience of interaction; in order to ensure the consistency of the interactive experience between different MR devices, the system adopts a cross-platform compatible interactive engine. The engine can adapt to the hardware characteristics and operating systems of various MR devices, and convert the user's interactive instructions into a unified format for processing, thereby achieving seamless switching and consistent experience on different devices; the system obtains the user's current location information and scene element status in real time through a context-aware algorithm. Based on this information, the system can automatically adjust the interactive options so that the user can perform the required operations more conveniently in the current scene. For example, when a user approaches a virtual object, the system may display interactive options related to the object; in order to understand the user's interactive behavior more deeply, the system uses the Transformer model to perform time series analysis on the user's behavior data. The model can capture the temporal dependency between user behaviors, thereby more accurately predicting the user's next operation. This predictive capability helps the system prepare in advance and provide a smoother interactive experience; combined with contextual information (such as scene elements, user location, historical behavior, etc.), the system builds an intent prediction model. This model can analyze the user's current interactive behavior and contextual environment to predict the user's true intention. For example, when a user frequently looks at a virtual button and is about to reach out, the system may predict that the user wants to click the button and prepare in advance; in order to ensure the accuracy and adaptability of the intent prediction model, the system adopts a real-time update mechanism. By collecting user feedback and new interaction data, the system can continuously optimize and adjust the model parameters and structure. This continuous optimization process enables the system to better adapt to changes in user behavior and environment, and provide more accurate and personalized services.

[0124] The effect of the above technical solution is: by integrating gesture recognition, voice recognition, eye tracking and other interactive methods, users can choose the most suitable interactive means according to their preferences and habits. This multimodal interaction method not only improves the flexibility of interaction, but also makes the user's operation in the MR environment more natural and intuitive, thereby improving the overall user experience; the use of a cross-platform compatible interaction engine makes the interactive experience between different MR devices consistent. No matter which MR device the user uses, they can enjoy the same or similar interactive effects, which greatly reduces the user's learning cost and adaptation difficulty, and improves user satisfaction; through the context-aware algorithm, the system can obtain the user's current location and scene element information in real time, and automatically adjust the interaction options accordingly. This intelligent adjustment enables users to perform the required operations more conveniently in the current scene, improving the efficiency and accuracy of the interaction; by combining context information and historical behavior data to build an intention prediction model, the system can predict the user's intention in advance and prepare. At the same time, the model is updated in real time based on user feedback and new interaction data to ensure the accuracy and adaptability of the prediction. This personalized service enables the system to better meet the needs and expectations of users, and improves user loyalty and stickiness; through the Transformer model, the system can capture the time dependency between user behaviors through time series analysis of user interaction behaviors, providing strong support for subsequent intention prediction and interaction optimization. This data analysis capability helps the system better understand the user's behavior patterns and habits, so as to make more accurate decisions and optimizations; the intention prediction model is updated in real time based on user feedback and new interaction data, ensuring the continuous optimization and iteration of the system. This dynamic adjustment mechanism enables the system to continuously adapt to user changes and needs, maintain its competitiveness and advancement; automatically adjust the interaction options according to the user's current location and scene elements, reducing the time and energy spent by users on finding suitable interaction methods, and improving the efficiency of interaction; through the intention prediction model, the user's intention is predicted in advance and prepared, which reduces the user's waiting and confirmation time and improves the accuracy and fluency of interaction.

[0125] In one embodiment of the present invention, the S32 includes:

[0126] Use MR devices (such as HMD, sensors, etc.) to capture the precise coordinates of the user's current position, orientation angle and surrounding environment data (such as lighting, sound, etc.) in real time, and synchronize the collected scene element information, including the location, size, type (such as furniture, walls, digital objects, etc.) and properties (such as interactivity, clickable areas, etc.) of objects.

[0127] Analyze the spatial relationship between the user's current location and scene elements, such as distance, relative position, etc., and dynamically adjust the visibility and accessibility of interactive options based on the spatial relationship;

[0128] Prioritize scene elements based on the user's historical behavior, current task goals, and the characteristics of scene elements (such as importance and urgency); place high-priority elements in the center of the user's field of vision or in a more accessible location;

[0129] Convert user interaction behaviors (such as gesture operations, voice commands, eye movement trajectories, etc.) into time series data, and use the Transformer model to model the time series of user interaction behaviors;

[0130] Use the self-attention mechanism to capture the dependencies between different time steps and train the model through historical interaction data;

[0131] Use the trained Transformer model to analyze the user's current interactive behavior in real time and identify the user's next action or potential needs;

[0132] Combine contextual information with the prediction results of the Transformer model to build an intent prediction model, and dynamically adjust the interactive interface and options based on the prediction results;

[0133] Design a user feedback mechanism to collect user feedback on the interaction experience, and update and optimize the intent prediction model in real time based on user feedback and new interaction data.

[0134] The working principle of the above technical solution is: using MR devices (such as head-mounted displays HMD, sensors, etc.) to capture the precise coordinates, orientation angles, and surrounding environment data (such as lighting, sound, etc.) of the user's current position in real time. At the same time, synchronously collect information on scene elements, including the location, size, type (such as furniture, walls, digital objects, etc.) and properties (such as interactivity, clickable areas, etc.) of objects; conduct in-depth analysis of the spatial relationship between the user's current position and scene elements, such as distance, relative position, etc. Based on these spatial relationships, the system can dynamically adjust the visibility and accessibility of interaction options. For example, when a user approaches an object, the interaction options related to the object may become more obvious or accessible; based on the user's historical behavior, current task goals, and the characteristics of scene elements (such as importance, urgency), the system prioritizes scene elements. High-priority elements are placed in the center of the user's field of view or in a more accessible position to improve the efficiency and accuracy of the interaction; convert the user's interactive behavior (such as gesture operations, voice commands, eye movement trajectories, etc.) into time series data. The Transformer model is used to model these time series data to capture the complex relationship between user behaviors; the self-attention mechanism in the Transformer model is used to capture the dependencies between different time steps. This mechanism enables the model to understand the overall trend and potential pattern of user behavior, thereby improving the accuracy of prediction; the Transformer model is trained through historical interaction data so that it can learn the laws and characteristics of user behavior. After training, the model can perform real-time analysis of the user's current interaction behavior; the trained Transformer model is used to perform real-time analysis of the user's current interaction behavior to identify the next step or potential needs that the user may take; the intention prediction model is constructed by combining contextual information and the prediction results of the Transformer model. The model can predict the user's true intention and dynamically adjust the interaction interface and options based on the prediction results; based on the output of the intention prediction model, the system can automatically adjust the interaction interface and options to better meet the user's needs and expectations. For example, if it is predicted that the user is about to perform a specific operation, the system can display relevant interaction options or prompt information in advance; a user feedback mechanism is designed to collect user feedback on the interaction experience. These feedbacks include user evaluations and suggestions on the interactive interface, options, and system response speed; the intention prediction model is updated and optimized in real time based on user feedback and new interaction data. This continuous optimization mechanism enables the system to continuously adapt to user changes and needs, and improve the accuracy and personalization of interactions.

[0135] The effects of the above technical solution are: the user's position and surrounding environment data are captured in real time through the MR device, and the scene element information is synchronized, ensuring that the virtual elements seen by the user are highly integrated with the real environment. This immediacy and accuracy enhance the naturalness of the user experience, making the user feel as if they are in a real mixed reality world; the visibility and accessibility of the interactive options are dynamically adjusted based on the spatial relationship between the user and the scene elements, so that the user can access and use these options more conveniently. This personalized adjustment improves the fluency of the interaction and reduces the time and energy wasted by the user in finding a suitable interaction method; the user's historical behavior, current task goals and the characteristics of the scene elements are prioritized, and high-priority elements are placed in the center of the user's field of vision or in a more accessible position. This intelligent recommendation mechanism enables users to find the required information or functions faster, improving the efficiency and accuracy of the interaction; the intention prediction model is constructed by combining the context information and the prediction results of the Transformer model, and the interactive interface and options are dynamically adjusted according to the prediction results. This personalized interaction method can better meet the actual needs and preferences of users and improve the satisfaction and loyalty of the user experience; the user's interactive behavior is converted into time series data, and the Transformer model is used for modeling. The self-attention mechanism is used to capture the dependencies between different time steps and understand the overall trend and potential patterns of user behavior. This method not only improves the predictability of user behavior, but also helps the system to understand the user's needs and intentions more deeply; the trained Transformer model is used to analyze the user's current interactive behavior in real time, and feedback and adjustments are made based on the analysis results. This instant feedback mechanism enables the system to respond more quickly to changes in user needs and continuously optimize its performance and services; a user feedback mechanism is designed to collect user feedback on the interactive experience. These feedbacks provide an important reference for the improvement and optimization of the system. By actively listening to the user's voice and continuously improving its own services, the system can continuously improve the satisfaction and loyalty of the user experience; the intention prediction model is updated and optimized in real time based on user feedback and new interaction data. This continuous optimization mechanism enables the system to continuously adapt to changes in users and changes in needs, and maintain its competitiveness and advancement.

[0136] In one embodiment of the present invention, the S4 includes:

[0137] S41. Through a collaborative interaction framework based on a distributed system and a microservice architecture, different functional modules (such as user management, scene management, interactive data processing, etc.) are split into independent services to improve the scalability and maintainability of the system. Each independent service interacts with each other through a lightweight communication protocol (such as RESTfulAPI or gRPC) to ensure data consistency and service availability. Fine-grained role division is also carried out, and different roles (such as administrator, editor, observer, etc.) are assigned according to the user's responsibilities and permissions in the virtual scene;

[0138] S42. Through the permission verification mechanism, when a user initiates an interaction request, their role and permission are verified to ensure that only users with corresponding permissions can perform specific operations; and introduce a dynamic permission adjustment function to allow administrators to adjust user permissions according to real-time conditions to respond to emergencies or special needs. Through a collaborative tool set, including chat systems, shared whiteboards, collaborative editors, etc., various forms of collaborative interaction are supported. And an interactive protocol is used to enable multiple users to interact with each other.

[0139] S43. Use network transmission technology (such as WebRTC or QUIC protocol) to synchronize user perspective and interactive data with low latency. Use network quality monitoring mechanism to monitor network indicators in real time, including network bandwidth, latency, and packet loss rate. Dynamically adjust transmission strategy (such as reducing resolution, compressing data, etc.) according to network conditions. The network conditions are calculated by the following formula:

[0140] Network status score = α*(1-delay rate)+β*(1-packet loss rate)+γ*bandwidth utilization;

[0141] α, β, and γ are weight coefficients of delay rate, packet loss rate, and bandwidth utilization, respectively, and α+β+γ=1;

[0142] Delay rate = (actual delay / maximum allowed delay);

[0143] Packet loss rate = (number of lost packets / total number of sent packets);

[0144] Bandwidth utilization = (actual bandwidth usage / total bandwidth);

[0145] S44. Adopt optimistic concurrency control (OCC) and pessimistic concurrency control (PCC) strategies, and combine version control mechanisms to manage data changes;

[0146] S45. When a data conflict is detected by the conflict detection and resolution algorithm, the conflict is automatically resolved according to a preset conflict resolution rule (such as a priority rule, a timestamp rule, etc.);

[0147] S46. Synchronize the state through state snapshot, incremental update or hybrid mode; based on the state rollback function, when an unresolvable conflict or user error is detected, allow the system to roll back to the last stable state;

[0148] S47. When users perform interactive operations, feedback on operation results and status changes should be provided through a real-time feedback mechanism; errors should be handled based on the error handling process; for recoverable errors, automatic or manual recovery plans should be provided; for unrecoverable errors, detailed error information should be recorded for subsequent analysis.

[0149] S48. Through the scene element locking mechanism, when a user is editing a scene element, the element is automatically locked. Through the element unlocking function, when the user completes editing and saves, the element is automatically unlocked to allow other users to continue editing.

[0150] The working principle of the above technical solution is as follows: by splitting different functional modules into independent services, each service is responsible for specific business logic and data processing, which improves the scalability and maintainability of the system. These services interact with each other through lightweight communication protocols (such as RESTfulAPI or gRPC) to ensure data consistency and service availability. According to the responsibilities and permissions of users in the virtual scene, fine-grained role division (such as administrator, editor, observer, etc.) is carried out. This division helps to achieve refined management of permissions, ensuring that only users with corresponding permissions can perform specific operations; when users initiate interaction requests, their roles and permissions are verified through the permission verification mechanism to ensure security. At the same time, the dynamic permission adjustment function is introduced to allow administrators to flexibly adjust user permissions according to real-time situations to respond to emergencies or special needs; chat systems, shared whiteboards, collaborative editors and other tools are provided to support various forms of collaborative interaction. The interactive protocol is adopted to ensure that the interactive data between multiple users can be transmitted smoothly, and key elements such as data verification, error handling, and retry mechanism are included to improve the reliability and stability of the interaction; network transmission technology (such as WebRTC or QUIC protocol) is used to achieve low-latency synchronization of user perspective and interactive data to ensure smooth interactive experience between users; through real-time monitoring of network indicators such as network bandwidth, delay, packet loss rate, etc., the transmission strategy is dynamically adjusted (such as reducing resolution, compressing data, etc.) to adapt to different network conditions to ensure the stability and reliability of data transmission; optimistic concurrency control (OCC) and pessimistic concurrency control (PCC) strategies are adopted, combined with the version control mechanism to manage data changes. This strategy helps to reduce data conflicts and maintain data consistency; when data conflicts are detected, the conflicts are automatically resolved according to the preset conflict resolution rules (such as priority rules, timestamp rules, etc.). These rules ensure that conflicts can be handled in a timely and effective manner to avoid affecting the collaborative work of users; the status is synchronized through state snapshots, incremental updates or hybrid modes to ensure that all users can see the latest status information; when unsolvable conflicts or user misoperations are detected, the system is allowed to roll back to the previous stable state. This mechanism provides security and avoids data loss or damage caused by erroneous operations. When users perform interactive operations, the real-time feedback mechanism provides feedback on the operation results and state changes to improve the user's interactive experience. For recoverable errors, automatic or manual recovery solutions are provided. For unrecoverable errors, detailed error information is recorded for subsequent analysis. This mechanism helps to quickly locate problems and take appropriate solutions. When a user is editing a scene element, the element is automatically locked to prevent other users from editing at the same time and causing conflicts. When the user completes the editing and saves it, the element is automatically unlocked to allow other users to continue editing. This mechanism ensures the continuity and consistency of the editing process.

[0151] The effect of the above technical solution is: different functional modules are split into independent services, so that the system can flexibly expand or reduce each service as needed. When a service needs to be updated or maintained, it can be done separately without affecting the normal operation of other services. This architecture greatly improves the scalability and maintainability of the system; different roles are assigned according to the responsibilities and permissions of users in the virtual scene, and refined management of permissions is achieved. This helps prevent unauthorized users from accessing sensitive information or performing dangerous operations; when a user initiates an interaction request, the role and permission are verified through the permission verification mechanism to ensure that only users with corresponding permissions can perform specific operations. This enhances the security of the system; allows administrators to adjust user permissions according to real-time conditions to respond to emergencies or special needs. This flexibility enables the system to better adapt to changing environments and needs; provides tools such as chat systems, shared whiteboards, and collaborative editors to support various forms of collaborative interaction. These tools enable users to communicate and collaborate more conveniently, improving work efficiency; low-latency synchronization of user perspective and interactive data is achieved through network transmission technology, ensuring smooth interaction experience between users; when users perform interactive operations, feedback on operation results and state changes is provided through real-time feedback mechanisms, enabling users to understand the operation results in a timely manner and make corresponding adjustments; optimistic concurrency control (OCC) and pessimistic concurrency control (PCC) strategies are adopted, and data changes are managed in combination with version control mechanisms. These strategies help reduce data conflicts and maintain data consistency; data conflicts are automatically resolved through conflict detection and resolution algorithms, reducing problems and disputes caused by conflicts; the state is synchronized through state snapshots, incremental updates or hybrid modes, and the system is allowed to roll back to the previous stable state when unresolvable conflicts or user errors are detected. This improves the reliability and stability of the system; errors are handled based on the error handling process, and automatic or manual recovery solutions are provided. This helps reduce interruptions and losses caused by errors and improves user experience; conflicts and confusion caused by multiple users editing the same element at the same time are prevented through the scene element locking mechanism. This mechanism ensures the continuity and consistency of the editing process and improves user experience satisfaction.The above formula combines three key indicators: latency rate, packet loss rate, and bandwidth utilization, which makes the evaluation result more comprehensive and can reflect the real status of the network from multiple angles. By converting the abstract network quality into specific values, the system can make automatic decisions based on the score, such as dynamically adjusting the transmission strategy to ensure service continuity and user experience. When the network status score is low, the system can automatically take measures based on the score, such as reducing video resolution, compressing data transmission, etc., to reduce bandwidth consumption and delay and ensure the normal operation of core functions. Through the WebRTC or QUIC protocol, combined with the network status score, low-latency synchronization of real-time interactive data can be achieved, enhancing multi-user The system can improve the smoothness and efficiency of user collaboration; when the score is lower than a certain threshold, the system can trigger the error handling process to provide users with automatic or manual recovery solutions to reduce user experience interruptions caused by network problems; by monitoring network conditions, it can predict possible connection problems in advance and reduce the occurrence of data conflicts. Combined with conflict detection and resolution algorithms, it ensures data consistency and service availability; when network conditions deteriorate seriously, the state snapshot and rollback functions allow the system to quickly recover to a previous stable state to prevent data loss or system crashes; bandwidth utilization is part of the score, which encourages the system to allocate and use network bandwidth more efficiently and avoid waste when resources are limited.

[0152] In one embodiment of the present invention, the S41 includes:

[0153] Identify the core design principles of distributed systems, including high availability, scalability, fault tolerance, and data consistency. Introduce containerization technologies (such as Docker) and orchestration tools (such as Kubernetes) to automate the deployment and management of services;

[0154] Design a distributed data storage solution to ensure data security and high availability, and introduce a caching mechanism; reduce the pressure on database access and increase data access speed.

[0155] Analyze business needs and divide the system into multiple independent functional modules, such as user management, scenario management, interactive data processing, permission control, etc. Each module is responsible for a single responsibility; reduce system coupling and improve maintainability.

[0156] Encapsulate each functional module as an independent microservice, with an independent database (or database instance) and operating environment, and independently deploy and expand the service on demand;

[0157] Inter-service interaction is carried out through the RESTfulAPI lightweight communication protocol. According to the user's responsibilities and permissions in the virtual scene, multiple roles (such as administrator, editor, observer, etc.) are defined; specific permissions and operation scopes are assigned to each role.

[0158] The working principle of the above technical solution is: ensure that the system can continue to provide services when some components fail, and improve the reliability of the system through redundant deployment, load balancing and other strategies; the system should be able to flexibly cope with the growth of user volume and data volume, and easily increase service instances through microservice architecture and horizontal expansion; the system should have an error detection and recovery mechanism, which can automatically or manually restore services when errors occur to reduce interruption time; ensure the consistency and accuracy of data when multiple users or multiple services operate data at the same time, and avoid data conflicts or inconsistencies; package each microservice and its dependencies into an independent container to achieve lightweight, portable and consistent deployment of services; automate the management of container deployment, expansion, update and fault recovery and other life cycle operations to improve the system's operation and maintenance efficiency and stability; adopt distributed data storage solutions, such as distributed databases or data sharding technology, to ensure that data is redundantly stored among multiple nodes to improve data reliability and availability; introduce a cache layer (such as Redis, Memcached, etc.) to reduce the access pressure on the main database and increase data access speed. The cache layer can store hot data or query results to speed up data retrieval. We need to deeply understand business needs and divide the system into multiple independent functional modules. Each module is responsible for a single responsibility, which reduces system coupling and improves maintainability. Each functional module is encapsulated as an independent microservice, and each microservice has an independent database (or database instance) and operating environment. This ensures the independence of services, facilitates independent deployment and on-demand expansion. We use RESTful API as a lightweight communication protocol for service interaction to ensure data consistency and service availability. RESTful API is easy to understand and use, and supports multiple clients and platforms. Based on the responsibilities and permissions of users in virtual scenes, we define multiple roles (such as administrators, editors, observers, etc.), and assign specific permissions and operation scopes to each role. Through the permission verification mechanism, we ensure that only users with corresponding permissions can perform specific operations.

[0159] The effects of the above technical solutions are: through the design of distributed systems, it is ensured that the system can still provide services when some components or nodes fail, reducing the risk of single point failure and improving the stability and reliability of the system; introducing a fault-tolerant mechanism, it can automatically detect, isolate and recover errors when errors occur in the system, reducing the service interruption time caused by failures; dividing the system into multiple independent functional modules and encapsulating them into microservices, so that the system can flexibly expand or reduce the scale of each service according to business needs. This architecture supports horizontal expansion and can easily cope with the growth of user volume and data volume; through tools such as Docker and Kubernetes, the automatic deployment, management and expansion of services are realized. When it is necessary to increase service instances, new container instances can be quickly deployed, and resource scheduling and load balancing can be performed through orchestration tools; a reasonable distributed data storage solution is designed to ensure data security and high availability. At the same time, the introduction of a cache mechanism reduces the access pressure on the database, improves the data access speed, and speeds up the response time of the system; through RESTfulAPI, service-to-service interaction reduces the complexity and overhead of data transmission and improves the communication efficiency between services; the system is divided into multiple independent functional modules, each module is responsible for a single responsibility, reduces the system coupling, and improves maintainability. When a module needs to be modified or upgraded, it can be done independently without affecting the normal operation of other modules; multiple roles are defined according to the user's responsibilities and permissions in the virtual scene, and specific permissions and operation scopes are assigned to each role. This flexible role permission management mechanism enables the system to better adapt to different business needs and security requirements; the microservice architecture allows team members to develop different services in parallel, improving development efficiency. At the same time, each service can use the technology stack and tool chain that best suits its business needs, promoting technological innovation and team collaboration; by introducing containerization technology and orchestration tools, automated deployment and management of services are achieved, reducing the possibility of manual intervention and errors, and improving operation and maintenance efficiency.

[0160] In one embodiment of the present invention, S5 includes:

[0161] S51, GPU-based rendering engine, which accelerates the rendering process of complex scenes through parallel computing on GPU, dynamically adjusts the rendering accuracy according to the distance and importance of scene elements through Level of Detail (LOD) technology, and optimizes rendering performance;

[0162] S52. Through scene segmentation and asynchronous loading mechanism, large scenes are divided into multiple small blocks, which are loaded and rendered on demand, reducing memory consumption and increasing loading speed. Based on the intelligent resource management system, rendering parameters (such as resolution, frame rate, special effects level) are dynamically adjusted according to device performance (such as GPU model, memory size), network bandwidth and user settings.

[0163] S53, monitor device performance and user interaction behavior in real time, predict future performance requirements through machine learning algorithms, and preload resources and optimize performance in advance;

[0164] S54. Users can adjust the rendering effect according to their needs and preferences through configurable rendering setting options, and use performance monitoring and reporting tools to record and analyze performance indicators (such as frame rate, response time, resource usage, etc.) during the rendering process, and make further optimizations based on the analysis results.

[0165] The working principle of the above technical solution is as follows: the powerful parallel processing capability of GPU is used to accelerate the rendering process of complex scenes. GPU can process a large amount of data at the same time, significantly improving the rendering speed; through the level of detail (LOD) technology, the rendering accuracy is dynamically adjusted according to the distance and importance of the scene elements to the observer. Near and important objects are rendered with high details, while distant or less important objects are rendered with lower details to reduce the amount of calculation and optimize rendering performance; large scenes are divided into multiple small blocks (such as terrain, buildings, vegetation, etc.) to achieve on-demand loading and rendering. This segmentation method can reduce the initial loading time and allow users to gradually load new content while exploring the scene; an asynchronous loading mechanism is used to load data of other scene blocks in the background while rendering the current scene. This can reduce the user's waiting time and improve the user experience; dynamically adjust rendering parameters (such as resolution, frame rate, special effects level) according to device performance (such as GPU model, memory size), network bandwidth and user settings. This intelligent adjustment can ensure that users can get the best rendering effect under different devices and network environments; monitor device performance and user interaction behavior in real time, collect key performance indicators and user behavior data; use machine learning algorithms to analyze the collected data and predict future performance requirements. These predictions can help the system preload resources and tune performance in advance to meet upcoming performance challenges; based on the prediction results, load the resources that may be needed, such as scene blocks, textures, maps, etc. in advance. This can reduce the loading time when users explore the scene and improve fluency; provide user-configurable rendering setting options, allowing users to adjust the rendering effect according to their needs and preferences. These settings may include resolution, frame rate, special effects level, etc.; through performance monitoring and reporting tools, record and analyze performance indicators (such as frame rate, response time, resource occupancy, etc.) during the rendering process. These data can help developers understand the performance bottlenecks and optimization directions of the system; based on the results of performance monitoring, further optimization can be carried out. This may include adjusting rendering parameters, optimizing scene layout, improving resource loading strategies, etc.

[0166] The effects of the above technical solutions are as follows: parallel computing through GPU greatly improves the rendering speed of complex scenes. The parallel processing capability of GPU makes large-scale data processing efficient, thereby reducing rendering time; the level of detail (LOD) technology dynamically adjusts the rendering accuracy according to the distance and importance of scene elements, effectively reducing unnecessary calculations and further optimizing rendering performance; large scenes are divided into multiple small blocks, loaded and rendered on demand, significantly reducing memory consumption and increasing loading speed. Users can start exploring without waiting for the entire scene to load, which improves the user experience; dynamically adjust rendering parameters according to device performance, network bandwidth and user settings to ensure the best rendering effect in different environments. This flexibility not only improves the user experience, but also improves resource utilization; real-time monitoring of device performance and user interaction behavior, and prediction of future performance requirements through machine learning algorithms. This predictive capability enables the system to preload resources and optimize performance in advance, thus avoiding performance bottlenecks and freezes. Based on the predicted results, the resources that may be needed are loaded in advance, reducing the waiting time when users explore the scene and improving the system's response speed and fluency. Users can adjust the rendering effects according to their needs and preferences, such as resolution, frame rate, special effects level, etc. This personalized setting meets the different needs of different users for rendering quality. Users can customize the rendering effects according to their preferences, making the virtual scene more in line with user expectations, thereby enhancing the user experience. Performance indicators in the rendering process are recorded and analyzed through performance monitoring and reporting tools to help developers understand the system's performance bottlenecks and optimization directions. Further optimization based on the results of performance monitoring can continuously improve the system's rendering performance and user experience. This closed-loop optimization process ensures that the system can continue to maintain its optimal state.

[0167] In one embodiment of the present invention, the S52 includes:

[0168] The AI ​​algorithm is used to analyze the complexity of the entire scene, including geometric complexity (such as the number of polygons, texture resolution), lighting complexity (number of light sources, shadow generation requirements), and the number of dynamic elements. Based on the scene complexity assessment results, adaptive mesh division is used to divide the large scene into multiple independent and interrelated small blocks (Chunks); each block contains an appropriate amount of scene elements to ensure the balance of loading and rendering.

[0169] Assign priorities to each scene block based on the camera position, user line of sight, distance and importance of scene elements (such as characters, key props, etc.).

[0170] Combined with user movement path prediction and user behavior pattern analysis, the scene blocks that the user may enter are loaded in advance to reduce loading delays. For network applications, streaming transmission technology is used to transmit necessary data blocks in real time according to the user's current field of view, while releasing memory resources that are no longer needed;

[0171] Through the intelligent cache mechanism, the cache size and content are dynamically adjusted according to the access frequency and importance of the scene block, the loading speed is improved during repeated access, and the GPU model, CPU performance, memory size and network bandwidth of the device are monitored in real time to automatically adjust the rendering parameters (such as resolution, texture quality, shadow accuracy, etc.);

[0172] Through machine learning algorithms, it learns user preferences, provides a personalized rendering experience for each user, and dynamically adjusts resource loading priorities according to current scene requirements; for example, it reduces texture details of distant scenes and increases the rendering quality of nearby key elements to optimize overall rendering performance.

[0173] Through monitoring tools, key performance indicators are recorded in real time, including frame rate, response time, and resource usage. Based on the performance monitoring results, rendering strategies are dynamically adjusted, such as reducing the rendering complexity of non-critical areas, enabling or disabling specific rendering effects, etc., to maintain the stability and smoothness of game operation.

[0174] The working principle of the above technical solution is as follows: First, AI algorithms are used to comprehensively analyze the geometric complexity (such as the number of polygons, texture resolution), lighting complexity (number of light sources, shadow generation requirements) and number of dynamic elements of the entire scene; based on the evaluation results of scene complexity, adaptive meshing technology is used to divide large scenes into multiple independent and interrelated small blocks (Chunks). Each block contains an appropriate amount of scene elements to ensure the balance between loading and rendering; each scene block is assigned a priority based on the camera position, user line of sight, distance and importance of scene elements (such as characters, key props, etc.). This ensures that the parts that users are most likely to pay attention to are loaded and rendered first; combined with user movement path prediction and user behavior pattern analysis, the scene blocks that users may be about to enter are loaded in advance. For network applications, streaming technology is used to transmit necessary data blocks in real time according to the user's current field of view, while releasing no longer needed memory resources to save bandwidth and memory; through an intelligent caching mechanism, the cache size and content are dynamically adjusted according to the access frequency and importance of scene blocks. This can improve the loading speed during repeated visits; monitor the device's GPU model, CPU performance, memory size and network bandwidth in real time, and automatically adjust rendering parameters (such as resolution, texture quality, shadow accuracy, etc.) to adapt to the performance characteristics of different devices; use machine learning algorithms to learn user preferences and provide each user with a personalized rendering experience. For example, dynamically adjust resource loading priorities based on user habits, reduce texture details of distant scenes, and increase the rendering quality of nearby key elements to optimize overall rendering performance; use monitoring tools to record key indicator performance (such as frame rate, response time, resource occupancy) in real time; dynamically adjust rendering strategies based on performance monitoring results. For example, reduce the rendering complexity of non-critical areas, enable or disable specific rendering effects, etc. when performance is insufficient to maintain the stability and smoothness of game operation.

[0175] The effects of the above technical solutions are as follows: the complexity of the scene is accurately analyzed through the AI ​​algorithm, and the large scene is divided into small blocks by the adaptive mesh division technology, so as to realize the on-demand loading and rendering of resources, reduce unnecessary calculation and loading burden, and significantly improve the rendering efficiency; the scene blocks are assigned priorities according to factors such as the camera position and the user's sight direction, and are loaded in advance in combination with the prediction of the user's movement path, which further reduces the loading delay and improves the response speed of the scene; the user preferences are learned through the machine learning algorithm to provide each user with a personalized rendering experience. For example, the resource loading priority is dynamically adjusted according to the user's habits, and the rendering quality of key elements is optimized, so that the user experience is more in line with personal expectations; by real-time monitoring of key indicator performance and dynamically adjusting the rendering strategy, such as reducing the rendering complexity of non-critical areas, enabling or disabling specific rendering effects, etc., the stability and smoothness of the game operation are maintained, and the jamming and frame dropping phenomena are reduced; for network applications, the streaming transmission technology is used to transmit the necessary data blocks in real time according to the user's current field of view, while releasing the memory resources that are no longer needed, effectively saving bandwidth and memory. The intelligent cache mechanism dynamically adjusts the cache size and content according to the access frequency and importance of the scene blocks, improving the loading speed during repeated access; monitors device performance (such as GPU model, CPU performance, memory size and network bandwidth) in real time and automatically adjusts rendering parameters (such as resolution, texture quality, shadow accuracy, etc.), ensuring the best rendering effect on various devices and network environments while avoiding waste of resources; divides large scenes into multiple independent and interrelated small blocks (Chunks), making the system more modular. This design method not only facilitates the update and management of resources, but also improves the scalability and maintainability of the system; through real-time monitoring and dynamic adjustment mechanisms, the system can automatically adjust rendering strategies and resource configurations according to different scene requirements, user behavior, device performance and other factors, making the system more flexible and intelligent.

[0176] One embodiment of the present invention, as Figure 2 As shown, a virtual scene control system based on MR large space, the system includes:

[0177] Space scanning module: Use space scanning equipment to scan the actual physical space, generate space mapping data, and initialize the basic framework of the virtual scene according to the space mapping data; including terrain, obstacles and boundaries.

[0178] Optimized layout module: Based on the user's preset scene templates and real-time sensor data, it dynamically generates elements in the virtual scene, such as buildings, vegetation, and props, and uses artificial intelligence algorithms to optimize the layout of scene elements;

[0179] Intention prediction module: through a variety of interaction methods, including gesture recognition, voice recognition and eye tracking, the virtual scene is controlled in real time, the user's interaction behavior is analyzed using a deep learning algorithm, and the user's intention is predicted based on the analysis results;

[0180] Collaborative interaction module: Users can interact collaboratively in the same virtual scene through the collaborative interaction module, and the consistency of each user's perspective and the real-time nature of the interaction are guaranteed through network synchronization technology;

[0181] Scene rendering module: Use high-performance graphics processing units (GPUs) for scene rendering, overlay virtual scenes into actual physical spaces through MR devices, and dynamically adjust the resolution and frame rate of scenes based on device performance and network bandwidth.

[0182] The working principle of the above technical solution is: use spatial scanning equipment (such as laser radar, depth camera, etc.) to perform three-dimensional scanning of the actual physical space to capture geometric information such as terrain, obstacles and boundaries in the space; generate spatial mapping data based on the spatial data obtained by scanning. These data are used to initialize the basic framework of the virtual scene to ensure that the virtual scene can accurately reflect the structure and layout of the real space; combine the user's preset scene template (such as urban landscape, forest environment, etc.) and real-time sensor data (such as weather, time changes, etc.) to dynamically generate elements in the virtual scene, such as buildings, vegetation, roads, props, etc.; use artificial intelligence algorithms to optimize the layout of the generated scene elements to ensure that the scene not only conforms to physical laws, but also has visual rationality and aesthetics. This includes considerations of factors such as the distance, orientation, and lighting effects between elements. Provide multiple interaction methods such as gesture recognition, voice recognition, and eye tracking, so that users can interact with virtual scenes in a natural and intuitive way; use deep learning algorithms to analyze users' interactive behaviors and identify users' operating intentions and preferences. By learning from historical data, the system can more accurately predict the user's next intention, thereby providing a more intelligent interactive experience; support multi-user collaborative interaction in the same virtual scene, such as co-editing scenes, role-playing, etc.; ensure the consistency of each user's perspective and the real-time nature of the interaction through network synchronization technology. Even users in different geographical locations can feel the interactive experience as if they were in the same space; use high-performance graphics processing units (GPUs) to render virtual scenes in real time to ensure high definition and smoothness of the scenes; dynamically adjust the resolution and frame rate of the scene according to the performance of the MR device and the actual network bandwidth. When the device performance is insufficient or the network bandwidth is limited, the continuity and stability of the interaction are guaranteed by reducing the resolution or frame rate.

[0183] The effect of the above technical solution is: the actual physical space data obtained by the space scanning device can generate high-precision space mapping data. This provides a solid foundation for the basic framework of the virtual scene, so that the virtual scene can highly restore the terrain, obstacles and boundaries of the real space, and enhance the user's immersion; based on the user's preset scene template and real-time sensor data, the system can dynamically generate elements in the virtual scene, such as buildings, vegetation and props. This flexibility allows users to quickly build and adjust virtual scenes according to actual needs; using artificial intelligence algorithms to optimize the layout of scene elements not only ensures the rationality of the scene (such as avoiding collisions, maintaining balance, etc.), but also improves the aesthetics of the scene. This intelligent processing greatly reduces the user's manual adjustment work and improves the efficiency and quality of scene creation; provides gesture recognition, voice recognition and eye tracking and other interactive methods, so that users can interact with virtual scenes in a more natural and intuitive way. This diversified interaction method enhances the user's experience and sense of participation; uses deep learning algorithms to analyze the user's interactive behavior and predicts the user's intention based on the analysis results. This intelligent prediction function enables the system to respond to user needs in advance and provide more personalized services; it supports multiple users to interact collaboratively in the same virtual scene, such as jointly editing scenes and role-playing. This collaborative interaction function promotes cooperation and communication between users and improves the efficiency of team work; it ensures the consistency of each user's perspective and the real-time nature of interaction through network synchronization technology. Even users in different geographical locations can feel the interactive experience as if they are in the same space, which enhances the closeness and coherence of team collaboration; it uses high-performance graphics processing units (GPUs) for scene rendering to ensure high definition and smoothness of virtual scenes. This high-quality rendering effect provides users with a more realistic visual experience; it dynamically adjusts the resolution and frame rate of the scene according to the performance of the MR device and the actual network bandwidth. This dynamic adjustment function ensures that users can obtain a stable and smooth interactive experience in different environments, improving the adaptability and stability of the system.

[0184] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A virtual scene control method based on MR large space, characterized in that: The method comprises: S1. Scan the actual physical space using a space scanning device to generate space mapping data, and initialize the basic framework of the virtual scene according to the space mapping data; S2, based on the user's preset scene template and real-time sensor data, dynamically generate elements in the virtual scene, and use artificial intelligence algorithms to optimize the layout of scene elements; S3, through a variety of interactive methods, real-time control of virtual scenes, use deep learning algorithms to analyze user interaction behaviors, and predict user intentions based on the analysis results; S4. Users can interact collaboratively in the same virtual scene through the collaborative interaction module, and the consistency of each user's perspective and the real-time nature of the interaction are guaranteed through network synchronization technology; S5. Use a high-performance graphics processing unit to render the scene, overlay the virtual scene into the actual physical space through the MR device, and dynamically adjust the resolution and frame rate of the scene according to the device performance and network bandwidth; The S2 comprises: S21. The user selects or customizes a template in the scene template library according to needs and builds a scene element library. Through AR marking technology, the user quickly generates scene elements by scanning specific markers; S22, obtaining environmental data in real time through a sensor data interface, and dynamically adjusting the performance of elements in the virtual scene according to the environmental data; S23, through the integration of IoT devices, the smart home status is reflected in the virtual scene in real time, the user's historical behavior data is analyzed through deep learning algorithms, and user preferences are predicted based on the analysis results; S24. Combine genetic algorithm or simulated annealing algorithm to intelligently layout scene elements and optimize visual effects and functionality; S25. Fine-tune the layout results through the user feedback mechanism and continuously optimize the AI ​​algorithm.

2. According to claim 1, a virtual scene control method based on MR large space is characterized in that: Said S1 comprises: S11, through the high-precision sensor array, the physical space is scanned in all directions to obtain the original data of the physical space; S12, performing time synchronization and spatial calibration on the collected raw data, eliminating errors between different sensors, and removing noise through filtering algorithms; S13, segmenting and clustering the point cloud data using the point cloud library, extracting key features, and constructing the basic terrain grid of the virtual scene using 3D modeling software based on the pre-processed spatial data; S14. Using the terrain texture system, texture is added to the virtual terrain according to the actual material of the physical space; S15. Accurately model obstacles so that the obstacles in the virtual scene are consistent with the actual space; S16. Automatically adjust the virtual scene boundary according to the user's activity range through boundary detection and dynamic adjustment mechanism.

3. The method for controlling a virtual scene based on a large MR space according to claim 1, characterized in that: The S3 includes: S31, constructing an interactive interface to integrate multiple interactive modes, including gesture recognition, voice recognition, and eye tracking; and unifying the interactions of different MR devices through a cross-platform compatible interactive engine; S32, automatically adjust the interactive options according to the user's current location and scene elements through context-aware algorithms, and perform time series analysis of user interaction behaviors through the Transformer model; S33. Construct an intent prediction model based on context information; and update the intent prediction model in real time based on user feedback and new interaction data.

4. The method for controlling a virtual scene based on a large MR space according to claim 3, characterized in that: The S32 includes: Use MR devices to capture the precise coordinates, orientation angles, and surrounding environment data of the user's current position in real time, and synchronize the collected scene element information; Analyze the spatial relationship between the user's current location and scene elements, and dynamically adjust the visibility and accessibility of interactive options based on the spatial relationship; Prioritize scene elements based on the user's historical behavior, current task goals, and the characteristics of scene elements; place high-priority elements in the center of the user's field of vision or in a more accessible location; Convert the user's interactive behavior into time series data and use the Transformer model to model the time series of the user's interactive behavior; Use the self-attention mechanism to capture the dependencies between different time steps and train the model through historical interaction data; Use the trained Transformer model to analyze the user's current interactive behavior in real time and identify the user's next action or potential needs; Combine contextual information with the prediction results of the Transformer model to build an intent prediction model, and dynamically adjust the interactive interface and options based on the prediction results; Design a user feedback mechanism to collect user feedback on the interaction experience, and update and optimize the intent prediction model in real time based on user feedback and new interaction data.

5. The method for controlling a virtual scene based on a large MR space according to claim 1, characterized in that: The S4 comprises: S41. Through the collaborative interaction framework based on distributed systems and the adoption of microservice architecture, different functional modules are split into independent services. Each independent service interacts with each other through a lightweight communication protocol and performs fine-grained role division, assigning different roles according to the user's responsibilities and permissions in the virtual scene. S42. Through the authority verification mechanism, when a user initiates an interaction request, their role and authority are verified, and a dynamic authority adjustment function is introduced, so that the administrator can adjust the user's authority according to the real-time situation; through the collaborative tool set and the interactive protocol, multiple users can interact with each other; S43. Through network transmission technology, the user's perspective and interactive data are synchronized with low latency; through the network quality monitoring mechanism, network indicators are monitored in real time; and the transmission strategy is dynamically adjusted according to the network status; S44. Adopt optimistic concurrency control and pessimistic concurrency control strategies, and manage data changes in combination with version control mechanisms; S45, when a data conflict is detected by the conflict detection and resolution algorithm, the conflict is resolved according to a preset conflict resolution rule; S46. Synchronize the state through state snapshot, incremental update or hybrid mode; based on the state rollback function, when an unresolvable conflict or user error is detected, allow the system to roll back to the last stable state; S47. When the user performs an interactive operation, the operation result and state change are fed back through a real-time feedback mechanism; and errors are handled based on the error handling process; S48. Through the scene element locking mechanism, when a user is editing a scene element, the element is automatically locked. Through the element unlocking function, when the user completes editing and saves, the element is automatically unlocked to allow other users to continue editing.

6. The method for controlling a virtual scene based on a large MR space according to claim 5, characterized in that: The S41 includes: Identify the core design principles of distributed systems and introduce containerization technology and orchestration tools to automate the deployment and management of services; Design a distributed data storage solution and introduce a caching mechanism; Analyze business requirements and divide the system into multiple independent functional modules, each of which is responsible for a single responsibility; Encapsulate each functional module as an independent microservice, and deploy and scale the service independently on demand; Inter-service interaction is carried out through the RESTful API lightweight communication protocol. Multiple roles are defined according to the user's responsibilities and permissions in the virtual scene; specific permissions and operation scopes are assigned to each role.

7. The method for controlling a virtual scene based on a large MR space according to claim 1, characterized in that: The S5 comprises: S51, GPU-based rendering engine, which accelerates the rendering process of complex scenes through parallel computing on GPU, and dynamically adjusts the rendering accuracy according to the distance and importance of scene elements through level of detail technology; S52, through the scene segmentation and asynchronous loading mechanism, the large scene is divided into multiple small blocks, loaded and rendered on demand, and the rendering parameters are dynamically adjusted according to the device performance, network bandwidth and user settings based on the intelligent resource management system; S53, monitor device performance and user interaction behavior in real time, predict future performance requirements through machine learning algorithms, and preload resources and optimize performance in advance; S54. Users can adjust the rendering effect according to their needs and preferences through configurable rendering setting options, and record and analyze the performance indicators of the rendering process through performance monitoring and reporting tools, and make further optimizations based on the analysis results.

8. The method for controlling a virtual scene based on a large MR space according to claim 7, characterized in that: The S52 includes: The complexity of the entire scene is analyzed through AI algorithms. Based on the scene complexity evaluation results, adaptive meshing is used to divide the large scene into multiple independent and interrelated small blocks. Assign a priority to each scene block based on the camera position, user line of sight, distance and importance of scene elements. Combined with user movement path prediction and user behavior pattern analysis, the scene blocks that the user is about to enter are loaded in advance. For network applications, streaming transmission technology is used to transmit necessary data blocks in real time according to the user's current field of view, while releasing no longer needed memory resources; Through the intelligent cache mechanism, the cache size and content are dynamically adjusted according to the access frequency and importance of the scene blocks, and the GPU model, CPU performance, memory size and network bandwidth of the device are monitored in real time to automatically adjust the rendering parameters; Learn user preferences through machine learning algorithms to provide each user with a personalized rendering experience and dynamically adjust resource loading priorities based on current scene requirements; Through monitoring tools, key performance indicators are recorded in real time, and rendering strategies are dynamically adjusted based on performance monitoring results.

9. A system for implementing the MR large space based virtual scene control method as claimed in any one of claims 1 to 8, characterized in that: The system comprises: Space scanning module: uses space scanning equipment to scan the actual physical space, generates space mapping data, and initializes the basic framework of the virtual scene according to the space mapping data; Optimized layout module: Based on the user's preset scene templates and real-time sensor data, it dynamically generates elements in the virtual scene and optimizes the layout of the scene elements using artificial intelligence algorithms; Intention prediction module: Through various interactive methods, the virtual scene is controlled in real time, the user's interactive behavior is analyzed using deep learning algorithms, and the user's intention is predicted based on the analysis results; Collaborative interaction module: Users can interact collaboratively in the same virtual scene through the collaborative interaction module, and the consistency of each user's perspective and the real-time nature of the interaction are guaranteed through network synchronization technology; Scene rendering module: Use a high-performance graphics processing unit to render the scene, overlay the virtual scene into the actual physical space through the MR device, and dynamically adjust the resolution and frame rate of the scene according to the device performance and network bandwidth.

Images