Multi-sensory interactive system based on biophilic experience

Abstract

The multi-sensory interactive system of the present invention is based on a biophilic experience and includes an interactive experience module, a virtual natural scene module, a user feedback module, a multi-sensory stimulation module, a UI design and data visualization module, an experience feedback module and a central processing module. It can solve the shortcomings of existing technologies in tactile feedback simulation and the accuracy and precision between physiological reactions and emotions, thereby providing users with an immersive, multi-sensory and nature-friendly healing experience.

Description

Technical Field

[0001] The present invention relates to the field of human-computer interaction, and more particularly, to a multi-sensory interaction system based on biophilic experience. Background Art

[0002] With the acceleration of urbanization, the life pressure faced by urban residents is increasing. In order to alleviate this pressure, people have begun to look for various ways to relax and relieve stress. With the accelerated pace of life, the increase of competitive pressure and the advent of the information age, national mental health problems are becoming increasingly prominent. These problems not only affect the quality of life and happiness of individuals, but also bring challenges to social stability and development. However, existing stress relief methods are often ineffective or unable to provide a full range of biophilic experiences.

[0003] Existing technologies have not fully achieved realism and accuracy in simulating tactile feedback, and there are deficiencies in the accuracy and precision between physiological reactions and emotions. Summary of the Invention

[0004] In order to overcome the above-mentioned deficiencies of the prior art, the present invention designs a multi-sensory interactive system based on biophilic experience, which can effectively overcome the problems of the prior art.

[0005] In order to solve the above technical problems, the technical solutions of the present invention are as follows:

[0006] A multi-sensory interactive system based on biophilic experience, including an interactive experience module, a virtual natural scene module, a user feedback module, a multi-sensory stimulation module, a UI design and data visualization module, an experience feedback module, and a central processing module;

[0007] The interactive experience module is used to receive user voice and gesture input, and through voice recognition and image recognition technology, converts the user's intention into instructions that the system can understand, providing an intuitive and natural interaction method, allowing users to easily interact with the system. In combination with the UI design and data visualization module, it provides users with a friendly operation interface and feedback;

[0008] The virtual natural scene module uses 3D modeling and real-time rendering technology to create a realistic natural environment, enhance the user's sense of immersion, and dynamically adjust elements in the scene based on user interaction instructions and system control signals, including light and shadow changes and wind simulation, to provide users with an experience similar to or even exceeding the real environment, promoting physical and mental relaxation and pleasure for users;

[0009] The user feedback module uses biosensor technology to monitor the user's physiological parameters in real time, including heart rate and galvanic skin response, analyze the user's physiological data, assess the user's emotional state and stress level, and transmit the user feedback data to the system for optimizing system output and providing personalized suggestions;

[0010] The multi-sensory stimulation module uses multiple sensory stimulation methods to enhance the user's immersion and sense of reality in the virtual natural environment, providing visual, auditory and tactile multi-sensory stimulation, including stereo images, natural sounds and vibration feedback. The type and intensity of sensory stimulation are dynamically adjusted according to changes in the virtual natural scene and user behavior. Through multi-sensory stimulation, users can better perceive and understand the virtual environment and improve the effect of the interactive experience.

[0011] The UI design and data visualization module designs an intuitive and easy-to-use user interface to facilitate user interaction with the system. It presents user data and system status to users in a visual manner through charts and animations, and provides data analysis and mining functions to help users better understand their needs and system performance.

[0012] The experience feedback module collects and analyzes users' overall evaluations and suggestions on the multi-sensory interaction system. It collects user feedback through questionnaires, interviews, and social media channels, organizes and analyzes user feedback, identifies system problems and areas for improvement, and converts user feedback into specific improvement measures and product development suggestions, thereby continuously improving the user experience and satisfaction of the system.

[0013] The central processing module is used to receive input data from each module, perform unified processing and analysis, coordinate the work of each module based on user feedback and interactive instructions, ensure the smooth and efficient operation of the entire system, update the system status information in real time, and pass it to the UI design and data visualization module for display;

[0014] The interactive experience module is connected to the central processing module, the virtual natural scene module is connected to the central processing module, the virtual natural scene module is connected to the multi-sensory stimulation module, the user feedback module is connected to the central processing module, the multi-sensory stimulation module is connected to the central processing module, the UI design and data visualization module is connected to the central processing module, and the experience feedback module is connected to the central processing module.

[0015] Preferably, the interactive experience module includes a user initialization submodule, a device configuration and calibration submodule, a real-time interaction processing submodule, a situation simulation and triggering submodule, a feedback display and visualization submodule, a personalized option and community interaction submodule, and a user exit settlement submodule;

[0016] The user initialization submodule is used for initial user guidance and system initialization. When the user enters the healing space, it prompts the user to walk to the designated location through voice or interface guidance. After the user sits down, the large and small screens are activated, preparing to enter the virtual natural scene.

[0017] The device configuration and calibration submodule is used to configure and calibrate the experience device. When the small screen lights up, it prompts the user to wear the experience device and perform necessary calibration operations. It also monitors the sensor connection status and signal quality to ensure data accuracy.

[0018] The real-time interaction processing submodule is used to receive interaction commands input by the user through body movements, touch, and blowing, interpret these commands, convert them into signals that the system can recognize, transmit them to the central processing module for processing, and update the response in the virtual environment in real time, including the contraction of virtual plants and the flight of birds;

[0019] The scenario simulation and triggering submodule is used to manage the interaction logic and trigger conditions in different scenarios. It triggers corresponding changes in the virtual environment based on the user's interaction instructions and the current scenario. When the user touches the physical plant, the virtual plant shrinks; when the user blows, the flock of birds flies.

[0020] The feedback display and visualization submodule is responsible for displaying the user's emotional state and other related information on the screen. Based on the data collected by the experience device, it analyzes the user's emotional fluctuations and presents them in a visual manner on the small screen, providing real-time display of data such as experience duration and emotional indicators;

[0021] The personalized options and community interaction submodule provides personalized interaction options and relaxation training, allowing users to choose to continue relaxation, meditation and breathing training functions according to their needs, supports community functions, allows users to share experiences and relaxation training, and increases the social interactivity of the system;

[0022] The user exit settlement submodule performs necessary exit operations when the user takes off the device and prepares to leave, saves the user's experience data for subsequent analysis and improvement, and settles the user's experience fee.

[0023] Preferably, the virtual natural scene module includes a visual channel submodule, a falling leaf motion effect submodule, and a sound and visual synchronization submodule;

[0024] The visual channel submodule uses TouchDesigner software and L-System technology to construct complex natural elements, including plants, animals, and terrain. It uses high-resolution environment maps and lighting effects to enhance the realism of the scene and interactive elements, including a dynamic weather system and day and night changes, to provide rich visual content. The detailed steps are as follows:

[0025] S21.1, L-System builds a virtual plant framework

[0026] Define L-System rules. According to the characteristics of the desired plant (such as branches, leaves, etc.), define corresponding L-System rules. The L-System rules will determine the growth pattern and morphology of the plant.

[0027] S21.2. Create an L-System generator. In the TouchDesigner software, use the SOP (Surface Operators) functional element to create the L-System generator. The L-System generator will gradually generate the framework of the virtual plant according to the defined L-System rules.

[0028] S21.3. Optimizing the generated virtual plant framework by adjusting parameters of the L-System generator, such as the number of iterations and branch angles;

[0029] S21.4. Environment Mapping Technology Enhances Visual Realism Obtain an environment map. Obtain a high-quality environment map that contains rich lighting information and texture details to provide a realistic environment mapping effect for virtual plants.

[0030] S21.5. Creating an environment mapping material: In the TouchDesigner software, use the MAT (Material) function to create an environment mapping material, using the obtained environment map as input and setting corresponding material parameters, such as reflection intensity and highlights;

[0031] S21.6. Applying the created environment mapping material to the virtual plant frame, and adjusting the material parameters and lighting conditions so that the virtual plant can realistically reflect the light and shadow information of the surrounding environment, thereby enhancing visual realism;

[0032] S21.7. Optimize the visual effects of the virtual plants as needed, including adding shadow effects and adjusting lighting direction to enhance the realism of the virtual plants;

[0033] S21.8, real-time preview and rendering, adjusting and optimizing the appearance and environmental mapping effect of the virtual plant frame through the real-time preview function of the TouchDesigner software to ensure that the final rendering effect meets expectations;

[0034] The leaf-falling animation submodule is used to simulate the dynamic effect of falling leaves in the wind and respond to the input of the airflow sensor to adjust the motion state of the falling leaves. The detailed steps are as follows:

[0035] S22.1. Initialize the SOP process, create a new SOP network in Houdini, and set the basic working environment and parameters;

[0036] S22.2. Collect physical property data of fallen leaves, including mass, shape, and density, create a 3D fallen leaf model library containing fallen leaves of different types and states, select and import a fallen leaf model from the 3D fallen leaf model library, and adjust the size, position, and rotation of the model as needed;

[0037] S22.3. Integrate the airflow sensor, connect the airflow sensor and set up the data interface. Create a custom node or script in the SOP process to receive and process the airflow sensor data.

[0038] S22.4. Set up the leaf-falling dynamics. Add physics engine nodes to the SOP process, including gravity and wind nodes. Connect the airflow sensor data to the corresponding parameters of the wind node. Adjust the wind force based on the real-time data and adjust other physical parameters to simulate the real leaf-falling dynamics.

[0039] S22.5. Create a particle system. Add a particle system node to the SOP process, including a Scatter or POPSNetwork node. Use the falling leaf model as an instance object of the particle system. Set the number, speed, and rotation parameters of the particles as needed.

[0040] S22.6. Add visual effects. Add textures, lighting, and shadows to the fallen leaves to enhance the visual effect. Use Houdini's material system to assign appropriate materials to the fallen leaves. Adjust lighting and camera parameters to achieve the best visual effect.

[0041] The sound and visual synchronization submodule plays corresponding sound effects based on different elements in the visual scene, such as wind, water, and animal calls. It uses 3D sound technology to enable users to perceive the direction and distance of sounds, increasing the sense of immersion. It also includes sound feedback related to user interaction, such as the sound of footsteps when users walk in the scene. The detailed steps are as follows:

[0042] S23.1. Create the river and bird models. Create the river model in Houdini's SOPs network and the bird model using the POPs particle system. Set the initial position, velocity, and lifecycle properties of the particles.

[0043] S23.2. Set up the CHOP network. Create a new CHOP network in Houdini and import the position and velocity animation data of the flying bird particles into the CHOP network.

[0044] S23.3. Create an event trigger. Create an event trigger in the CHOP network and write logic to detect whether the flying bird particle meets the trigger conditions, such as entering the view or changing speed. When the conditions are met, trigger an event to play the sound effect.

[0045] S23.4. Import and synchronize sound effects. Import the bird chirping sound effect file into Houdini. Use Houdini's audio tools or third-party plug-ins to synchronize the sound effect with events in the CHOP network, ensuring that the sound effect's playback time is consistent with the animation timeline of the visual element.

[0046] Preferably, the multi-sensory stimulation module includes an ESP32 touch submodule, an atomization drive module submodule, a skin electrical sensor submodule and an airflow sensor submodule;

[0047] The ESP32 touch submodule provides touch input function, allowing users to communicate with the system intuitively and conveniently through the touch interactive interface. It can detect the user's touch position, touch force and touch duration information, provide user input data to the system, and combine with the software interface to realize menu selection and parameter setting operations;

[0048] The atomization drive module submodule controls the opening, closing, and adjustment of atomization equipment, such as humidifiers and atomizers, and simulates the fog and water mist effects in the natural environment according to system instructions or preset conditions, thereby enhancing the user's biophilia experience;

[0049] The galvanic skin sensor submodule measures the user's galvanic skin response (GSR), i.e., changes in skin resistance. GSR is a physiological indicator of emotional state and can be used to analyze the user's emotional changes and stress levels.

[0050] The airflow sensor submodule measures the airflow velocity, direction and pressure parameters in the environment, and provides the system with real-time environmental data so as to adjust the simulation effect or trigger corresponding functions.

[0051] Compared with the prior art, the beneficial effects of the technical solution of the present invention are as follows: the present invention designs a multi-sensory interaction system based on biophilic experience, and the multi-sensory stimulation module provides users with accurate tactile feedback according to the instructions of the virtual natural scene module. These feedbacks simulate the tactile experience in the real environment, such as the feeling of wind blowing, the temperature of water, etc., and draw on the multi-layer network model in the force tactile perception mechanism. The system transfers and maps tactile stimulation through different channels to ensure the accuracy and completeness of tactile information. The central processing module receives tactile stimulation information from the multi-sensory stimulation module, and combines the data of the virtual natural scene module to perform intelligent analysis and processing of the tactile stimulation to achieve synchronization and matching with the virtual environment. The central processing module has a fast response capability to ensure that tactile feedback can be provided in real time when the user interacts with the system, thereby enhancing the realism and accuracy of the user experience, and using biosensors for real-time monitoring. Measure the user's physiological parameters, such as heart rate, skin electrical response, etc., to ensure the accuracy and real-time nature of physiological data. Combined with the user's behavioral data and virtual environment information, the system can accurately analyze the user's emotional state and provide a multi-sensory experience that is more in line with the user's emotional state. The UI design and data visualization module displays the user's physiological data, emotional state and other information in the form of charts, graphs, etc., providing system developers with an intuitive analysis basis. Based on user feedback and data analysis, the UI design and data visualization module can optimize the system's interaction design to make it more in line with the user's physiological reactions and emotional needs. The experience feedback module collects user feedback on the system, such as satisfaction, improvement suggestions, etc., as an important reference for system optimization. Based on user feedback, system developers can continuously iterate and optimize the system to continuously improve the system's accuracy and precision in tactile feedback simulation and the relationship between physiological reactions and emotions. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the implementation methods of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the implementation methods or the description of the prior art. Obviously, the drawings described below are merely exemplary. For ordinary technicians in this field, other implementation drawings can be derived based on the provided drawings without any creative work.

[0053] Figure 1 This is a structural diagram of a multi-sensory interaction system based on biophilic experience;

[0054] Among them, 1. Interactive experience module; 2. Virtual natural scene module; 3. User feedback module; 4. Multi-sensory stimulation module; 5. UI design and data visualization module; 6. Experience feedback module; 7. Central processing module; 11. User initialization submodule; 12. Device configuration and calibration submodule; 13. Real-time interactive processing submodule; 14. Scenario simulation and triggering submodule; 15. Feedback display and visualization submodule; 16. Personalized options and community interaction submodule; 17. User exit settlement submodule; 21. Visual channel submodule; 22. Falling leaves animation submodule; 23. Sound and visual synchronization submodule; 41. ESP32 touch submodule; 42. Atomization drive module submodule; 43. Skin electrical sensor submodule; 44. Airflow sensor submodule. DETAILED DESCRIPTION

[0055] The accompanying drawings are for illustrative purposes only and are not to be construed as limiting this patent;

[0056] In order to better illustrate this embodiment, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product size;

[0057] It is understandable to those skilled in the art that some well-known structures and descriptions thereof may be omitted in the drawings.

[0058] The technical solution of the present invention is further described below with reference to the accompanying drawings and embodiments.

[0059] Example

[0060] Multi-sensory interactive systems based on biophilic experiences, such as Figure 1 As shown, it includes an interactive experience module 1, a virtual natural scene module 2, a user feedback module 3, a multi-sensory stimulation module 4, a UI design and data visualization module 5, an experience feedback module 6 and a central processing module 7;

[0061] The interactive experience module 1 is used to receive user voice and gesture inputs, and through voice recognition and image recognition technologies, converts the user's intentions into commands that the system can understand, providing an intuitive and natural interaction method, allowing users to easily interact with the system. Combined with the UI design and data visualization module 5, it provides users with a friendly operation interface and feedback;

[0062] The virtual natural scene module 2 uses 3D modeling and real-time rendering technology to create a realistic natural environment, enhance the user's sense of immersion, and dynamically adjust the elements in the scene according to the user's interactive instructions and the system's control signals, including light and shadow changes and wind simulation, to provide the user with an experience similar to or even exceeding the real environment, promoting the user's physical and mental relaxation and pleasure;

[0063] The user feedback module 3 uses biosensor technology to monitor the user's physiological parameters in real time, including heart rate and galvanic skin response, analyze the user's physiological data, assess the user's emotional state and stress level, and transmit the user feedback data to the system for optimizing system output and providing personalized suggestions;

[0064] The multi-sensory stimulation module 4 uses multiple sensory stimulation methods to enhance the user's immersion and sense of reality in the virtual natural environment, providing visual, auditory and tactile multi-sensory stimulation, including stereo images, natural sounds and vibration feedback. According to the changes in the virtual natural scene and user behavior, the type and intensity of sensory stimulation are dynamically adjusted. Through multi-sensory stimulation, the user can better perceive and understand the virtual environment and improve the effect of the interactive experience;

[0065] The UI design and data visualization module 5 designs an intuitive and easy-to-use user interface to facilitate user interaction with the system. It presents user data and system status to users in a visual manner through charts and animations, provides data analysis and mining functions, and helps users better understand their needs and system performance.

[0066] The experience feedback module 6 collects and analyzes users' overall evaluations and suggestions on the multi-sensory interactive system, collects user feedback through questionnaires, interviews, and social media, organizes and analyzes user feedback, identifies system problems and areas for improvement, and converts user feedback into specific improvement measures and product development suggestions, thereby continuously improving the user experience and satisfaction of the system;

[0067] The central processing module 7 is used to receive input data from each module, perform unified processing and analysis, coordinate the work of each module based on user feedback and interactive instructions, ensure the overall smooth and efficient operation of the system, update the system status information in real time, and pass it to the UI design and data visualization module 5 for display;

[0068] The interactive experience module 1 is connected to the central processing module 7, the virtual natural scene module 2 is connected to the central processing module 7, the virtual natural scene module 2 is connected to the multi-sensory stimulation module 4, the user feedback module 3 is connected to the central processing module 7, the multi-sensory stimulation module 4 is connected to the central processing module 7, the UI design and data visualization module 5 is connected to the central processing module 7, and the experience feedback module 6 is connected to the central processing module 7.

[0069] The interactive experience module 1 includes a user initialization submodule 11, a device configuration and calibration submodule 12, a real-time interaction processing submodule 13, a scenario simulation and triggering submodule 14, a feedback display and visualization submodule 15, a personalized option and community interaction submodule 16, and a user exit settlement submodule 17;

[0070] The user initialization submodule 11 is used for initial user guidance and system initialization. When the user enters the healing space, the user is prompted to walk to the designated location through voice or interface guidance. After the user sits down, the large and small screens are activated, and the user is ready to enter the virtual natural scene.

[0071] The device configuration and calibration submodule 12 is used to configure and calibrate the experience device. When the small screen lights up, it prompts the user to wear the experience device and perform necessary calibration operations. It also monitors the sensor connection status and signal quality to ensure data accuracy.

[0072] The real-time interaction processing submodule 13 is used to receive interaction commands input by the user through body movements, touch, and blowing, interpret these commands, convert them into signals that the system can recognize, and transmit them to the central processing module 7 for processing, thereby updating the response in the virtual environment in real time, including the contraction of virtual plants and the flight of birds;

[0073] The scenario simulation and triggering submodule 14 is used to manage the interaction logic and trigger conditions in different scenarios. It triggers corresponding changes in the virtual environment according to the user's interaction instructions and the current scenario. When the user touches the physical plant, the virtual plant shrinks; when the user blows, the flock of birds flies.

[0074] The feedback display and visualization submodule 15 is responsible for displaying the user's emotional state and other related information on the screen. It analyzes the user's emotional fluctuations based on the data collected by the experience device and presents them in a visual manner on the small screen, providing real-time display of data such as experience duration and emotional indicators;

[0075] The personalized options and community interaction submodule 16 provides personalized interaction options and relaxation training, allowing users to choose to continue relaxation, meditation and breathing training functions according to their needs, supports community functions, allows users to share their experiences and relaxation training, and increases the social interactivity of the system;

[0076] The user exit settlement submodule 17 performs necessary exit operations when the user takes off the device and prepares to leave, saves the user's experience data for subsequent analysis and improvement, and settles the user's experience fee.

[0077] The virtual natural scene module 2 includes a visual channel submodule 21, a falling leaf motion effect submodule 22, and a sound effect and visual synchronization submodule 23;

[0078] TouchDesigner is a powerful real-time interactive creation and development platform widely used in visual arts, real-time data visualization, virtual reality and stage performances.

[0079] L-System, or Lindenmayer System, is a formal generative system introduced by Hungarian biologist and botanist Aristid Lindenmayer in 1968. It is a mathematical formalization tool used to describe the growth process of plants, simulate natural forms, and generate graphics. In addition to plant modeling, L-System has also been applied to graphic design, art generation, virtual world modeling and other fields.

[0080] The visual channel submodule 21 uses TouchDesigner software and L-System technology to construct complex natural elements, including plants, animals, and terrain. It uses high-resolution environment maps and lighting effects to enhance the realism of the scene and interactive elements, including a dynamic weather system and day and night changes, to provide rich visual content. The detailed steps are as follows:

[0081] S21.1, L-System builds a virtual plant framework

[0082] Define L-System rules. According to the characteristics of the desired plant, such as branches and leaves, define corresponding L-System rules. The L-System rules will determine the growth pattern and morphology of the plant.

[0083] S21.2. Create an L-System generator. In the TouchDesigner software, use the SOP (Surface Operators) functional element to create the L-System generator. The L-System generator will gradually generate the framework of the virtual plant according to the defined L-System rules.

[0084] In TouchDesigner software, SOP stands for "Surface Operators", which is a functional component used to process and manipulate surface data. SOP components can create complex 3D surfaces and geometries, and are often used in visual effects, animation and simulation fields.

[0085] S21.3. Optimizing the generated virtual plant framework by adjusting parameters of the L-System generator, such as the number of iterations and branch angles;

[0086] S21.4. Environment Mapping Technology Enhances Visual Realism Obtain an environment map. Obtain a high-quality environment map that contains rich lighting information and texture details to provide a realistic environment mapping effect for virtual plants.

[0087] S21.5. Creating an environment mapping material: In the TouchDesigner software, use the MAT (Material) function to create an environment mapping material, using the obtained environment map as input and setting corresponding material parameters, such as reflection intensity and highlights;

[0088] In TouchDesigner, the MAT (Material) function is used to create and edit materials that can be applied to the surface of 3D objects to achieve various visual effects. Environment mapping is a special material technology that uses environment maps to simulate the effect of surfaces reflecting the surrounding environment.

[0089] S21.6. Applying the created environment mapping material to the virtual plant frame, and adjusting the material parameters and lighting conditions so that the virtual plant can realistically reflect the light and shadow information of the surrounding environment, thereby enhancing visual realism;

[0090] S21.7. Optimize the visual effects of the virtual plants as needed, including adding shadow effects and adjusting lighting direction to enhance the realism of the virtual plants;

[0091] S21.8, real-time preview and rendering, adjusting and optimizing the appearance and environmental mapping effect of the virtual plant frame through the real-time preview function of the TouchDesigner software to ensure that the final rendering effect meets expectations;

[0092] The leaf-falling dynamic effect submodule 22 is mainly responsible for simulating the dynamic effect of falling leaves in the wind and responding to the input of the airflow sensor to adjust the motion state of the falling leaves. The detailed steps are as follows:

[0093] S22.1. Initialize the SOP process, create a new SOP network in Houdini, and set the basic working environment and parameters;

[0094] Houdini is a powerful 3D computer graphics software widely used in film, television, games and visual effects production. It was developed by the Canadian company SideFX and is known for its highly flexible node-based workflow and powerful simulation and effects tools.

[0095] S22.2. Collect physical property data of fallen leaves, including mass, shape, and density, create a 3D fallen leaf model library containing fallen leaves of different types and states, select and import a fallen leaf model from the 3D fallen leaf model library, and adjust the size, position, and rotation of the model as needed;

[0096] S22.3. Integrate the airflow sensor, connect the airflow sensor and set up the data interface. Create a custom node or script in the SOP process to receive and process the airflow sensor data.

[0097] S22.4. Set up the leaf-falling dynamics. Add physics engine nodes to the SOP process, including gravity and wind nodes. Connect the airflow sensor data to the corresponding parameters of the wind node. Adjust the wind force based on the real-time data and adjust other physical parameters to simulate the real leaf-falling dynamics.

[0098] S22.5. Create a particle system. Add a particle system node to the SOP process, including a Scatter or POPSNetwork node. Use the falling leaf model as an instance object of the particle system. Set the number, speed, and rotation parameters of the particles as needed.

[0099] The Scatter node is used to randomly distribute particles or instanced objects on the surface or within the volume of a geometry. It can generate particles on the surface or inside the volume of a geometry according to specified parameters. The Scatter node is often used to create the initial particle distribution of a particle system.

[0100] POPS Network (or Particle Operators Network) is a node network used to create and manipulate particle systems. In Houdini, particle systems are defined by POPS Network nodes to define their behavior and appearance. You can add various particle operators (POPs) to the POPS Network, such as gravity, velocity, and lifecycle control, to adjust and control the dynamic behavior of particles.

[0101] S22.6. Add visual effects. Add textures, lighting, and shadows to the fallen leaves to enhance the visual effect. Use Houdini's material system to assign appropriate materials to the fallen leaves. Adjust lighting and camera parameters to achieve the best visual effect.

[0102] The sound and visual synchronization submodule 23 plays corresponding sound effects based on different elements in the visual scene, such as wind, water, and animal calls. It uses 3D sound technology to enable users to perceive the direction and distance of sounds, increasing the sense of immersion. It also includes sound feedback related to user interaction, such as the sound of footsteps when users walk in the scene. The detailed steps are as follows:

[0103] S23.1. Create a river and bird model. Create a river model in Houdini's SOPs network and a bird model using the POPs particle system. Set the particles' initial position, velocity, and lifecycle properties.

[0104] S23.2. Set up the CHOP network. Create a new CHOP network in Houdini and import the position and velocity animation data of the flying bird particles into the CHOP network.

[0105] In Houdini, CHOP stands for "Channel Operators," a type of node used to process and manipulate time-dependent data. CHOP networks allow users to create, edit, and manipulate this temporal data, typically represented as signals or channels. In the context of Houdini, CHOP networks are particularly well-suited for processing animations, particle systems, sounds, motion data, and any other numerical data that varies over time.

[0106] S23.3. Create an event trigger. Create an event trigger in the CHOP network and write logic to detect whether the flying bird particle meets the trigger conditions, such as entering the view or changing speed. When the conditions are met, trigger an event to play the sound effect.

[0107] S23.4. Import and synchronize sound effects. Import the bird chirping sound effect file into Houdini. Use Houdini's audio tools or third-party plug-ins to synchronize the sound effect with events in the CHOP network, ensuring that the sound effect's playback time is consistent with the animation timeline of the visual element.

[0108] The multi-sensory stimulation module 4 includes an ESP32 touch submodule 41, an atomization drive module submodule 42, a skin electrical sensor submodule 43 and an airflow sensor submodule 44;

[0109] The ESP32 touch submodule 41 provides a touch input function, allowing the user to communicate with the system intuitively and conveniently through a touch interactive interface. It can detect the user's touch position, touch force, and touch duration information, provide user input data to the system, and implement menu selection and parameter setting operations in conjunction with the software interface;

[0110] The atomization drive module submodule 42 controls the opening, closing, and adjustment of atomization devices, such as humidifiers and atomizers, and simulates the fog and mist effects in the natural environment according to system instructions or preset conditions to enhance the user's biophilia experience;

[0111] The galvanic skin sensor submodule 43 measures the user's galvanic skin response (GSR), i.e., changes in skin resistance. GSR is a physiological indicator of emotional state and can be used to analyze the user's emotional changes and stress levels.

[0112] The airflow sensor submodule 44 measures airflow velocity, direction, and pressure parameters in the environment, and provides the system with real-time environmental data to adjust simulation effects or trigger corresponding functions.

[0113] Specific implementation method When the user starts the system, the central processing module 7 is responsible for the initialization of the entire system to ensure the normal operation of each module. The UI design and data visualization module 5 is responsible for displaying the system interface and guiding the user to interact. The virtual natural scene module 2 generates a corresponding virtual environment based on the preset or user-selected natural scene. This module uses advanced graphics rendering technology to ensure the realism and immersion of the scene. The interactive experience module 1 is responsible for receiving the user's interactive instructions, such as gestures, voice, touch, etc. The user can interact with the system through gesture recognizers, microphones, touch screens and other devices. The central processing module 7 updates the scene content in the virtual natural scene module 2 in real time according to the user's interactive instructions, such as changing the perspective, moving objects, etc. The multi-sensory stimulation module 4 provides the user with multiple sensory stimulations according to the instructions of the virtual natural scene module 2 and the central processing module 7. Visual stimulation presents the virtual environment through the display screen, auditory stimulation plays natural sounds through the speakers, and tactile stimulation simulates different objects through vibration devices, temperature control and other devices. The sense of touch and olfactory stimulation of the body can simulate the smell in the natural environment through the odor releaser. These sensory stimuli together constitute an immersive biophilic experience. The user feedback module 3 monitors the user's physiological parameters in real time through biosensors and other devices, and transmits the data to the central processing module 7. The central processing module 7 combines the user's behavioral data and virtual environment information to accurately analyze the user's emotional state. These emotional data can provide the system with a deeper understanding of the user in order to optimize the subsequent interactive experience. The UI design and data visualization module 5 is responsible for displaying the user's physiological data, emotional state and other information in the form of charts, graphs, etc. These data visualization results can help system developers better understand user needs and behavior patterns in order to optimize and improve the system. The experience feedback module 6 is responsible for collecting user feedback on the system. These feedback data will be transmitted to the central processing module 7 for processing and analysis. Based on these feedback data, system developers can continuously iterate and optimize the system to improve the accuracy and precision of the user experience.

[0114] The same or similar reference numerals correspond to the same or similar components;

[0115] The terms used in the drawings to describe positional relationships are for illustrative purposes only and should not be construed as limiting this patent;

[0116] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not limitations on the implementation methods of the present invention. For ordinary technicians in the field, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to enumerate all the implementation methods here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the claims of the present invention.

Claims

1. A multi-sensory exchange system based on biophilic experience, characterized by: It includes an interactive experience module (1), a virtual natural scene module (2), a user feedback module (3), a multi-sensory stimulation module (4), a UI design and data visualization module (5), an experience feedback module (6) and a central processing module (7); The interactive experience module (1) is used to receive user voice and gesture inputs, and convert the user's intentions into commands that the system can understand through voice recognition and image recognition technology, providing an intuitive and natural interaction method, allowing users to easily interact with the system, and combining the UI design and data visualization module (5) to provide users with a friendly operation interface and feedback; The virtual natural scene module (2) uses 3D modeling and real-time rendering technology to create a realistic natural environment, enhance the user's sense of immersion, and dynamically adjust the elements in the scene according to the user's interactive instructions and the system's control signals, including light and shadow changes and wind simulation, to provide the user with an experience similar to or exceeding the real environment, promoting the user's physical and mental relaxation and pleasure; The virtual natural scene module (2) includes a visual channel submodule (21), a falling leaf motion effect submodule (22), and a sound effect and visual synchronization submodule (23); The visual channel submodule (21) uses TouchDesigner software and L-System technology to construct complex natural elements, including plants, animals and terrain, and enhances the realism of the scene and interactive elements through high-resolution environment maps and light and shadow effects, including dynamic weather systems and day and night changes, to provide rich visual content. The detailed steps are as follows: S21.1, L-System builds the virtual plant framework; Define L-System rules. According to the characteristics of the desired plants, define corresponding L-System rules. The L-System rules will determine the growth pattern and morphology of the plants. S21.

2. Create an L-System generator. In the TouchDesigner software, use SOP functional elements to create an L-System generator. The L-System generator will gradually generate the framework of the virtual plant according to the defined L-System rules. S21.

3. Optimize the generated virtual plant framework by adjusting parameters of the L-System generator; S21.

4. Environment Mapping Technology Enhances Visual Realism Obtain an environment map. Obtain a high-quality environment map that contains rich lighting information and texture details to provide a realistic environment mapping effect for virtual plants. S21.

5. Creating an environment mapping material: In the TouchDesigner software, use the MAT function to create an environment mapping material, using the obtained environment map as input, and setting corresponding material parameters; S21.

6. Applying the created environment mapping material to the virtual plant frame, and adjusting the material parameters and lighting conditions so that the virtual plant can realistically reflect the light and shadow information of the surrounding environment, thereby enhancing visual realism; S21.

7. Optimize the visual effects of the virtual plants as needed, including adding shadow effects and adjusting lighting direction to enhance the realism of the virtual plants; S21.8, real-time preview and rendering, adjusting and optimizing the appearance and environmental mapping effect of the virtual plant frame through the real-time preview function of the TouchDesigner software to ensure that the final rendering effect meets expectations; The leaf-falling dynamic effect submodule (22) is used to simulate the dynamic effect of falling leaves in the wind and respond to the input of the airflow sensor to adjust the motion state of the falling leaves. The detailed steps are as follows: S22.

1. Initialize the SOP process, create a new SOP network in Houdini, and set the basic working environment and parameters; S22.

2. Collect physical property data of fallen leaves, including mass, shape, and density, create a 3D fallen leaf model library containing fallen leaves of different types and states, select and import a fallen leaf model from the 3D fallen leaf model library, and adjust the size, position, and rotation of the model as needed; S22.

3. Integrate the airflow sensor, connect the airflow sensor and set up the data interface. Create a custom node or script in the SOP process to receive and process the airflow sensor data. S22.

4. Set up the leaf-falling dynamics. Add physics engine nodes to the SOP process, including gravity and wind nodes. Connect the airflow sensor data to the corresponding parameters of the wind node. Adjust the wind force based on the real-time data and adjust other physical parameters to simulate the real leaf-falling dynamics. S22.

5. Create a particle system. Add a particle system node to the SOP process, including a Scatter or POPS Network node. Use the falling leaf model as an instance object of the particle system. Set the number, speed, and rotation parameters of the particles as needed. S22.

6. Add visual effects. Add textures, lighting, and shadows to the fallen leaves to enhance the visual effect. Use Houdini's material system to assign appropriate materials to the fallen leaves. Adjust lighting and camera parameters to achieve the best visual effect. The sound and visual synchronization submodule (23) plays corresponding sound effects according to different elements in the visual scene, and uses 3D sound effect technology to enable users to perceive the direction and distance of the sound, thereby increasing the sense of immersion. It also includes sound effect feedback related to user interaction. The detailed steps are as follows: S23.

1. Create the river and bird models. Create the river model in Houdini's SOPs network and the bird model using the POPs particle system. Set the initial position, velocity, and lifecycle properties of the particles. S23.

2. Set up the CHOP network. Create a new CHOP network in Houdini and import the position and velocity animation data of the flying bird particles into the CHOP network. S23.

3. Create an event trigger. Create an event trigger in the CHOP network and write logic to detect whether the flying bird particle meets the trigger condition. When the condition is met, trigger an event to play the sound effect. S23.

4. Import and synchronize sound effects. Import the bird chirping sound effect file into Houdini. Use Houdini's audio tools or third-party plug-ins to synchronize the sound effect with events in the CHOP network, ensuring that the sound effect's playback time is consistent with the animation timeline of the visual element. The user feedback module (3) utilizes biosensor technology to monitor the user's physiological parameters in real time, including heart rate and skin galvanic response, analyze the user's physiological data, assess the user's emotional state and stress level, and transmit the user feedback data to the system for optimizing system output and providing personalized suggestions; The multi-sensory stimulation module (4) utilizes multiple sensory stimulation means to enhance the user's immersion and sense of reality in the virtual natural environment, provides visual, auditory and tactile multi-sensory stimulation, including stereoscopic images, natural sounds and vibration feedback, and dynamically adjusts the type and intensity of sensory stimulation according to changes in the virtual natural scene and user behavior. Through multi-sensory stimulation, the user is helped to better perceive and understand the virtual environment, thereby improving the effect of the interactive experience; The UI design and data visualization module (5) designs an intuitive and easy-to-use user interface to facilitate user interaction with the system. It presents user data and system status to users in a visual manner through charts and animations, provides data analysis and mining functions, and helps users better understand their needs and system performance. The experience feedback module (6) collects and analyzes users' overall evaluation and suggestions on the multi-sensory exchange system, collects user feedback through questionnaires, interviews and social media, organizes and analyzes user feedback, identifies problems and room for improvement in the system, converts user feedback into specific improvement measures and product development suggestions, and continuously improves the user experience and satisfaction of the system; The central processing module (7) is used to receive input data from each module, perform unified processing and analysis, coordinate the work of each module based on user feedback and interactive instructions, ensure the overall smooth and efficient operation of the system, update the system status information in real time, and pass it to the UI design and data visualization module (5) for display; The interactive experience module (1) is connected to the central processing module (7), the virtual natural scene module (2) is connected to the central processing module (7), the virtual natural scene module (2) is connected to the multi-sensory stimulation module (4), the user feedback module (3) is connected to the central processing module (7), the multi-sensory stimulation module (4) is connected to the central processing module (7), the UI design and data visualization module (5) is connected to the central processing module (7), and the experience feedback module (6) is connected to the central processing module (7).

2. The multi-sensory exchange system based on biophilic experience according to claim 1, characterized in that: The interactive experience module (1) includes a user initialization submodule (11), a device configuration and calibration submodule (12), a real-time interaction processing submodule (13), a situation simulation and triggering submodule (14), a feedback display and visualization submodule (15), a personalized option and community interaction submodule (16), and a user exit settlement submodule (17); The user initialization submodule (11) is used for initial guidance of the user and system initialization. When the user enters the healing space, the user is prompted to walk to the designated location through voice or interface guidance. After the user sits down, the display of the large screen and the small screen is activated, and the user is ready to enter the virtual natural scene. The device configuration and calibration submodule (12) is used for configuration and calibration of the experience device. When the small screen lights up, it prompts the user to wear the experience device and perform necessary calibration operations, monitors the connection status and signal quality of the sensor, and ensures data accuracy. The real-time interaction processing submodule (13) is used to receive interaction instructions input by the user through body movements, touch and blowing, analyze these instructions, and convert them into signals that can be recognized by the system, and transmit them to the central processing module (7) for processing, so as to update the response in the virtual environment in real time, including the contraction of virtual plants and the flight of birds; The scenario simulation and triggering submodule (14) is used to manage the interaction logic and trigger conditions in different scenarios, and trigger corresponding virtual environment changes according to the user's interaction instructions and the current scenario. When the user touches the physical plant, the virtual plant is triggered to shrink; when the user blows air, the flock of birds is triggered to fly; The feedback display and visualization submodule (15) is responsible for displaying the user's emotional state and other related information on the screen, analyzing the user's emotional fluctuations based on the data collected by the experience device, and presenting them in a visual manner on the small screen, providing real-time display of the experience duration and emotional index data; The personalized options and community interaction submodule (16) provides personalized interaction options and relaxation training, allowing users to choose to continue relaxation, meditation and breathing training functions according to their needs, supports community functions, allows users to share experiences and relaxation training, and increases the social interactivity of the system; The user exit settlement submodule (17) performs necessary exit operations when the user takes off the device and prepares to leave, saves the user's experience data for subsequent analysis and improvement, and settles the user's experience fee.

3. The multi-sensory exchange system based on biophilic experience according to claim 2, characterized in that: The multi-sensory stimulation module (4) includes an ESP32 touch submodule (41), an atomization drive module submodule (42), a skin electrical sensor submodule (43) and an airflow sensor submodule (44); The ESP32 touch submodule (41) provides a touch input function, allowing the user to communicate with the system intuitively and conveniently through a touch interaction interface, and can detect the user's touch position, touch strength and touch duration information, provide the system with user input data, and realize menu selection and parameter setting operations in combination with the software interface; The atomization drive module submodule (42) controls the opening, closing and adjustment of the atomization device, and simulates the fog and water mist effects in the natural environment according to system instructions or preset conditions, thereby enhancing the user's biophilia experience; The skin electrical sensor submodule (43) measures the user's skin electrical response, that is, the change in skin resistance; The airflow sensor submodule (44) measures airflow velocity, direction, and pressure parameters in the environment, and provides the system with real-time environmental data so as to adjust simulation effects or trigger corresponding functions.

Images