Multi-sensory interaction system based on biological experience

Abstract

The invention discloses a multi-sensory interaction system based on parental and biological experience. The multi-sensory interaction system comprises an interaction experience module, a virtual natural scene module, a user feedback module, a multi-sensory stimulation module, a UI design and data visualization module, an experience feedback module and a central processing module, according to the invention, the defects in accuracy and precision between tactile feedback simulation and physiological reaction and emotion in the prior art can be overcome, and an immersive, multi-sensory and natural close healing experience is provided for the user.

Description

Technical Field

[0001] The present invention relates to the field of human-computer interaction, and more specifically, to a multi-sensory interaction system based on biophilic experience. Background Art

[0002] With the acceleration of urbanization, urban residents are facing increasing life pressure. In order to alleviate this pressure, people begin to look for various ways to relax and relieve stress. With the accelerated pace of life, the increase of competitive pressure and the advent of the information age, national mental health issues are becoming increasingly prominent. These problems not only affect the quality of life and happiness of individuals, but also bring challenges to social stability and development. However, existing stress relief methods are often ineffective or unable to provide a full range of biophilic experiences.

[0003] The existing technology has not fully achieved realism and accuracy in simulating tactile feedback, and there are deficiencies in the accuracy and precision between physiological reactions and emotions. Summary of the invention

[0004] In order to overcome the above-mentioned deficiencies of the prior art, the present invention designs a multi-sensory interaction system based on biophilic experience, which can effectively overcome the problems of the prior art.

[0005] In order to solve the above technical problems, the technical solution of the present invention is as follows:

[0006] A multi-sensory interactive system based on biophilic experience, including interactive experience module, virtual natural scene module, user feedback module, multi-sensory stimulation module, UI design and data visualization module, experience feedback module and central processing module;

[0007] The interactive experience module is used to receive the user's voice and gesture input, and convert the user's intention into instructions that the system can understand through voice recognition and image recognition technology, providing an intuitive and natural interaction method so that the user can easily interact with the system. Combined with the UI design and data visualization module, it provides users with a friendly operation interface and feedback;

[0008] The virtual natural scene module uses 3D modeling and real-time rendering technology to create a realistic natural environment, enhance the user's sense of immersion, and dynamically adjust the elements in the scene according to the user's interactive instructions and the system's control signals, including light and shadow changes and wind simulation, to provide users with an experience similar to or beyond the real environment, and promote the user's physical and mental relaxation and pleasure;

[0009] The user feedback module uses biosensor technology to monitor the user's physiological parameters in real time, including heart rate and galvanic skin response, analyze the user's physiological data, evaluate the user's emotional state and stress level, and transmit the user feedback data to the system for optimizing system output and providing personalized suggestions;

[0010] The multi-sensory stimulation module uses multiple sensory stimulation methods to enhance the user's immersion and sense of reality in the virtual natural environment, provides visual, auditory and tactile multi-sensory stimulation, including stereoscopic images, natural sounds and vibration feedback, and dynamically adjusts the type and intensity of sensory stimulation according to changes in the virtual natural scene and user behavior. Through multi-sensory stimulation, it helps users better perceive and understand the virtual environment and improves the effect of interactive experience;

[0011] The UI design and data visualization module designs an intuitive and easy-to-use user interface to facilitate user interaction with the system. It presents user data and system status to users in a visual manner through charts and animations, provides data analysis and mining functions, and helps users better understand their needs and system performance.

[0012] The experience feedback module collects and analyzes users' overall evaluation and suggestions on the multi-sensory interaction system, collects user feedback through questionnaires, interviews and social media, organizes and analyzes user feedback, identifies system problems and room for improvement, converts user feedback into specific improvement measures and product development suggestions, and continuously improves the user experience and satisfaction of the system;

[0013] The central processing module is used to receive input data from each module, perform unified processing and analysis, coordinate the work of each module according to user feedback and interactive instructions, ensure the overall smooth and efficient operation of the system, update the system status information in real time, and pass it to the UI design and data visualization module for display;

[0014] The interactive experience module is connected to the central processing module, the virtual natural scene module is connected to the central processing module, the virtual natural scene module is connected to the multi-sensory stimulation module, the user feedback module is connected to the central processing module, the multi-sensory stimulation module is connected to the central processing module, the UI design and data visualization module is connected to the central processing module, and the experience feedback module is connected to the central processing module.

[0015] Preferably, the interactive experience module includes a user initialization submodule, a device configuration and calibration submodule, a real-time interaction processing submodule, a situation simulation and triggering submodule, a feedback display and visualization submodule, a personalized option and community interaction submodule, and a user exit settlement submodule;

[0016] The user initialization submodule is used for initial guidance of the user and system initialization. When the user enters the healing space, the user is prompted to walk to the designated location through voice or interface guidance. After the user sits down, the display of the large and small screens is started to prepare to enter the virtual natural scene.

[0017] The device configuration and calibration submodule is used for configuration and calibration of the experience device. When the small screen lights up, it prompts the user to wear the experience device and perform necessary calibration operations, monitors the connection status and signal quality of the sensor, and ensures data accuracy;

[0018] The real-time interaction processing submodule is used to receive interaction instructions input by the user through body movements, touch, and blowing, analyze these instructions, and convert them into signals recognizable by the system, and transmit them to the central processing module for processing, and update the response in the virtual environment in real time, including the contraction of virtual plants and the flight of birds;

[0019] The scenario simulation and triggering submodule is used to manage the interaction logic and trigger conditions in different scenarios. According to the user's interaction instructions and the current scenario, the corresponding virtual environment changes are triggered. When the user touches the physical plant, the virtual plant shrinks; when the user blows, the flock of birds flies.

[0020] The feedback display and visualization submodule is responsible for displaying the user's emotional state and other related information on the screen. It analyzes the user's emotional fluctuations based on the data collected by the experience device and presents them on the small screen in a visual manner, providing real-time display of data such as experience duration and emotional indicators;

[0021] The personalized options and community interaction submodule provides personalized interaction options and relaxation training, allowing users to choose to continue relaxation, meditation and breathing training functions according to their needs, supports community functions, allows users to share their experiences and relaxation training, and increases the social interactivity of the system;

[0022] The user exit settlement submodule performs necessary exit operations when the user takes off the device and prepares to leave, saves the user's experience data for subsequent analysis and improvement, and settles the user's experience fee.

[0023] Preferably, the virtual natural scene module includes a visual channel submodule, a falling leaf motion effect submodule, and a sound effect and visual synchronization submodule;

[0024] The visual channel submodule uses TouchDesigner software and L-System technology to build complex natural elements, including plants, animals and terrain, and enhances the realism and interactive elements of the scene through high-resolution environment maps and light and shadow effects, including dynamic weather systems and day and night changes, to provide rich visual content. The detailed steps are as follows:

[0025] S21.1, L-System builds a virtual plant framework

[0026] Define L-System rules, according to the characteristics of the desired plants (such as branches, leaves, etc.), define corresponding L-System rules, and the L-System rules will determine the growth pattern and morphology of the plants;

[0027] S21.2, creating an L-System generator. In the TouchDesigner software, use the SOP (Surface Operators) functional element to create the L-System generator. The L-System generator will gradually generate the framework of the virtual plant according to the defined L-System rules;

[0028] S21.3, optimizing the generated virtual plant framework by adjusting the parameters of the L-System generator, such as the number of iterations, branch angles, etc.;

[0029] S21.4. Environmental mapping technology enhances visual realism. Obtain an environmental map. Obtain a high-quality environmental map. The environmental map should contain rich light and shadow information and texture details so that the virtual plant provides a realistic environmental mapping effect.

[0030] S21.5, creating an environment mapping material, in the TouchDesigner software, using the MAT (Material) function element to create an environment mapping material, using the obtained environment map as input, and setting corresponding material parameters, such as reflection intensity, highlight, etc.;

[0031] S21.6, applying the environment mapping material, applying the created environment mapping material to the virtual plant frame, and adjusting the material parameters and lighting conditions so that the virtual plant can realistically reflect the light and shadow information of the surrounding environment to enhance the visual realism;

[0032] S21.7. Optimizing the visual effects of the virtual plants as needed, including adding shadow effects and adjusting the lighting direction, to enhance the realism of the virtual plants;

[0033] S21.8, real-time preview and rendering, through the real-time preview function of the TouchDesigner software, adjust and optimize the appearance and environment mapping effect of the virtual plant framework to ensure that the final rendering effect meets expectations;

[0034] The leaf-falling dynamic effect submodule is used to simulate the dynamic effect of leaves falling in the wind and respond to the input of the airflow sensor to adjust the motion state of the leaves. The detailed steps are as follows:

[0035] S22.1. Initialize the SOP process, create a new SOP network in Houdini, and set the basic working environment and parameters;

[0036] S22.2, collecting physical property data of fallen leaves, including mass, shape and density, creating a 3D fallen leaf model library, including fallen leaves of different types and states, selecting and importing a fallen leaf model from the 3D fallen leaf model library, and adjusting the size, position and rotation of the model as needed;

[0037] S22.3, integrate the airflow sensor, connect the airflow sensor and set up the data interface, and create a custom node or script in the SOP process to receive and process the airflow sensor data;

[0038] S22.4. Set up leaf falling dynamics, add physical engine nodes in the SOP process, including gravity and wind nodes, connect the airflow sensor data to the corresponding parameters of the wind node, adjust the wind force according to the real-time data, and adjust other physical parameters to simulate the real leaf falling dynamics;

[0039] S22.5. Create a particle system. Add a particle system node in the SOP process, including a Scatter or POPSNetwork node. Use the fallen leaf model as an instance object of the particle system. Set the number, speed, and rotation parameters of the particles as needed.

[0040] S22.6. Add visual effects, add texture, lighting and shadow effects to the fallen leaves to enhance the visual effect, use Houdini's material system to assign appropriate materials to the fallen leaves, and adjust the lighting and camera parameters to obtain the best visual effect;

[0041] The sound and visual synchronization submodule plays corresponding sound effects according to different elements in the visual scene, such as wind, water, animal calls, etc., and uses 3D sound technology to enable users to perceive the direction and distance of the sound, increasing the sense of immersion. It also includes sound feedback related to user interaction, such as the footsteps of users walking in the scene. The detailed steps are as follows:

[0042] S23.1. Create river and bird models. Create a river model in Houdini's SOPs network, use the POPs particle system to create a bird model, and set the initial position, speed, and life cycle properties of the particles.

[0043] S23.2. Set up the CHOP network. Create a new CHOP network in Houdini and import the position and velocity animation data of the flying bird particles into the CHOP network.

[0044] S23.3. Create an event trigger. Create an event trigger in the CHOP network and write logic to detect whether the flying bird particle meets the trigger conditions, such as entering the view, speed change, etc. When the conditions are met, trigger an event to play the sound effect;

[0045] S23.4. Import and synchronize sound effects. Import the bird chirping sound effect file into Houdini. Use Houdini's audio tools or third-party plug-ins to synchronize the sound effect with events in the CHOP network to ensure that the playback time of the sound effect is consistent with the animation timeline of the visual element.

[0046] Preferably, the multi-sensory stimulation module includes an ESP32 touch submodule, an atomization drive module submodule, a skin electrical sensor submodule and an airflow sensor submodule;

[0047] The ESP32 touch submodule provides a touch input function, allowing the user to communicate with the system intuitively and conveniently through a touch interaction interface. It can detect the user's touch position, touch strength and touch duration information, provide the system with user input data, and combine with the software interface to realize menu selection and parameter setting operations;

[0048] The atomization drive module submodule controls the opening, closing and adjustment of atomization equipment, such as humidifiers and atomizers, and simulates the fog and water mist effects in the natural environment according to system instructions or preset conditions to enhance the user's biophilic experience;

[0049] The galvanic skin sensor submodule measures the user's galvanic skin response (GSR), that is, the change in skin resistance. The galvanic skin response is a physiological indicator of emotional state and can be used to analyze the user's emotional changes and stress level;

[0050] The airflow sensor submodule measures airflow velocity, direction and pressure parameters in the environment, and provides the system with real-time environmental data to adjust the simulation effect or trigger corresponding functions.

[0051] Compared with the prior art, the beneficial effects of the technical solution of the present invention are as follows: the present invention designs a multi-sensory interaction system based on biophilic experience, and the multi-sensory stimulation module provides users with accurate tactile feedback according to the instructions of the virtual natural scene module. These feedbacks simulate the tactile experience in the real environment, such as the feeling of wind blowing, the temperature of water, etc. By referring to the multi-layer network model in the force tactile perception mechanism, the system transfers and maps tactile stimulation through different channels to ensure the accuracy and completeness of tactile information. The central processing module receives the tactile stimulation information from the multi-sensory stimulation module, and combines the data of the virtual natural scene module to perform intelligent analysis and processing on the tactile stimulation to achieve synchronization and matching with the virtual environment. The central processing module has a fast response capability to ensure that when the user interacts with the system, tactile feedback can be provided in real time, thereby enhancing the realism and accuracy of the user experience, and using biosensors to monitor in real time The system measures the user's physiological parameters, such as heart rate, skin electrical response, etc., to ensure the accuracy and real-time nature of the physiological data. Combined with the user's behavioral data and virtual environment information, the system can accurately analyze the user's emotional state and provide a multi-sensory experience that is more in line with the user's emotional state. The UI design and data visualization module displays the user's physiological data, emotional state and other information in the form of charts, graphs, etc., providing system developers with an intuitive analysis basis. Based on user feedback and data analysis, the UI design and data visualization module can optimize the system's interaction design to make it more in line with the user's physiological reactions and emotional needs. The experience feedback module collects user feedback on the system, such as satisfaction, improvement suggestions, etc., as an important reference for system optimization. Based on user feedback, system developers can continuously iterate and optimize the system to continuously improve the accuracy and precision of the system in tactile feedback simulation and between physiological reactions and emotions. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only exemplary, and for ordinary technicians in this field, other implementation drawings can be derived based on the provided drawings without paying any creative work.

[0053] Figure 1 This is a structural diagram of a multi-sensory interactive system based on biophilic experience;

[0054] Among them, 1. Interactive experience module; 2. Virtual natural scene module; 3. User feedback module; 4. Multi-sensory stimulation module; 5. UI design and data visualization module; 6. Experience feedback module; 7. Central processing module; 11. User initialization submodule; 12. Device configuration and calibration submodule; 13. Real-time interactive processing submodule; 14. Scenario simulation and triggering submodule; 15. Feedback display and visualization submodule; 16. Personalized options and community interaction submodule; 17. User exit settlement submodule; 21. Visual channel submodule; 22. Falling leaves animation submodule; 23. Sound and visual synchronization submodule; 41. ESP32 touch submodule; 42. Atomization drive module submodule; 43. Skin electrical sensor submodule; 44. Airflow sensor submodule. DETAILED DESCRIPTION

[0055] The drawings are for illustrative purposes only and should not be construed as limiting the present patent;

[0056] In order to better illustrate the present embodiment, some parts in the drawings may be omitted, enlarged or reduced, and do not represent the size of the actual product;

[0057] It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0058] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0059] Example

[0060] Multi-sensory interactive systems based on biophilic experiences, such as Figure 1 As shown, it includes an interactive experience module 1, a virtual natural scene module 2, a user feedback module 3, a multi-sensory stimulation module 4, a UI design and data visualization module 5, an experience feedback module 6 and a central processing module 7;

[0061] The interactive experience module 1 is used to receive the user's voice and gesture input, and convert the user's intention into instructions that the system can understand through voice recognition and image recognition technology, providing an intuitive and natural interaction method so that the user can easily interact with the system. Combined with the UI design and data visualization module 5, it provides users with a friendly operation interface and feedback;

[0062] The virtual natural scene module 2 uses 3D modeling and real-time rendering technology to create a realistic natural environment, enhance the user's sense of immersion, and dynamically adjust the elements in the scene according to the user's interactive instructions and the system's control signals, including light and shadow changes and wind simulation, to provide the user with an experience similar to or beyond the real environment, and promote the user's physical and mental relaxation and pleasure;

[0063] The user feedback module 3 uses biosensor technology to monitor the user's physiological parameters in real time, including heart rate and skin galvanic response, analyze the user's physiological data, evaluate the user's emotional state and stress level, and transmit the user feedback data to the system for optimizing system output and providing personalized suggestions;

[0064] The multi-sensory stimulation module 4 uses a variety of sensory stimulation methods to enhance the user's immersion and sense of reality in the virtual natural environment, provides visual, auditory and tactile multi-sensory stimulation, including stereoscopic images, natural sounds and vibration feedback, and dynamically adjusts the type and intensity of sensory stimulation according to changes in the virtual natural scene and user behavior. Through multi-sensory stimulation, it helps users better perceive and understand the virtual environment and improves the effect of interactive experience;

[0065] The UI design and data visualization module 5 designs an intuitive and easy-to-use user interface to facilitate user interaction with the system, presents user data and system status to users in a visual manner through charts and animations, provides data analysis and mining functions, and helps users better understand their needs and system performance;

[0066] The experience feedback module 6 collects and analyzes users' overall evaluation and suggestions on the multi-sensory interactive system, collects user feedback through questionnaires, interviews and social media, organizes and analyzes user feedback, identifies system problems and room for improvement, converts user feedback into specific improvement measures and product development suggestions, and continuously improves the user experience and satisfaction of the system;

[0067] The central processing module 7 is used to receive input data from each module, perform unified processing and analysis, coordinate the work of each module according to user feedback and interactive instructions, ensure the overall smooth and efficient operation of the system, update the system status information in real time, and pass it to the UI design and data visualization module 5 for display;

[0068] The interactive experience module 1 is connected to the central processing module 7, the virtual natural scene module 2 is connected to the central processing module 7, the virtual natural scene module 2 is connected to the multi-sensory stimulation module 4, the user feedback module 3 is connected to the central processing module 7, the multi-sensory stimulation module 4 is connected to the central processing module 7, the UI design and data visualization module 5 is connected to the central processing module 7, and the experience feedback module 6 is connected to the central processing module 7.

[0069] The interactive experience module 1 includes a user initialization submodule 11, a device configuration and calibration submodule 12, a real-time interaction processing submodule 13, a situation simulation and triggering submodule 14, a feedback display and visualization submodule 15, a personalized option and community interaction submodule 16 and a user exit settlement submodule 17;

[0070] The user initialization submodule 11 is used for initial guidance of the user and system initialization. When the user enters the healing space, the user is prompted to walk to the designated location through voice or interface guidance. After the user sits down, the display of the large screen and the small screen is started to prepare to enter the virtual natural scene.

[0071] The device configuration and calibration submodule 12 is used for configuration and calibration of the experience device. When the small screen lights up, it prompts the user to wear the experience device and perform necessary calibration operations, monitors the connection status and signal quality of the sensor, and ensures data accuracy;

[0072] The real-time interaction processing submodule 13 is used to receive interaction instructions input by the user through body movements, touch, and blowing, analyze these instructions, and convert them into signals recognizable by the system, and transmit them to the central processing module 7 for processing, and update the response in the virtual environment in real time, including the contraction of virtual plants and the flight of birds;

[0073] The scenario simulation and triggering submodule 14 is used to manage the interaction logic and trigger conditions in different scenarios, and trigger the corresponding virtual environment changes according to the user's interaction instructions and the current scenario. When the user touches the physical plant, the virtual plant shrinks; when the user blows, the flock of birds flies.

[0074] The feedback display and visualization submodule 15 is responsible for displaying the user's emotional state and other related information on the screen, analyzing the user's emotional fluctuations based on the data collected by the experience device, and presenting them on the small screen in a visual manner, providing real-time display of data such as experience duration and emotional indicators;

[0075] The personalized options and community interaction submodule 16 provides personalized interaction options and relaxation training, allowing users to choose to continue relaxation, meditation and breathing training functions according to their needs, supports community functions, allows users to share their experiences and relaxation training, and increases the social interactivity of the system;

[0076] The user exit settlement submodule 17 performs necessary exit operations when the user takes off the device and prepares to leave, saves the user's experience data for subsequent analysis and improvement, and settles the user's experience fee.

[0077] The virtual natural scene module 2 includes a visual channel submodule 21, a falling leaf dynamic effect submodule 22, and a sound effect and visual synchronization submodule 23;

[0078] TouchDesigner is a powerful real-time interactive creation and development platform, widely used in visual arts, real-time data visualization, virtual reality and stage performance.

[0079] L-System, or Lindenmayer System, is a formalized generative system introduced by Hungarian biologist and botanist Aristid Lindenmayer in 1968. It is a mathematical formalization tool used to describe the growth process of plants, simulate natural forms and generate graphics. In addition to plant modeling, L-System is also used in graphic design, art generation, virtual world modeling and other fields.

[0080] The visual channel submodule 21 uses TouchDesigner software and L-System technology to construct complex natural elements, including plants, animals and terrain, and enhances the realism and interactive elements of the scene through high-resolution environment maps and light and shadow effects, including dynamic weather systems and day and night changes, to provide rich visual content. The detailed steps are as follows:

[0081] S21.1, L-System builds a virtual plant framework

[0082] Define L-System rules, and define corresponding L-System rules according to the characteristics of the desired plants, such as branches, leaves, etc., which will determine the growth pattern and morphology of the plants;

[0083] S21.2, creating an L-System generator. In the TouchDesigner software, use the SOP (Surface Operators) functional element to create the L-System generator. The L-System generator will gradually generate the framework of the virtual plant according to the defined L-System rules;

[0084] In TouchDesigner software, SOP stands for "Surface Operators", which is a functional element used to process and manipulate surface data. SOP elements can create complex 3D surfaces and geometries, and are often used in visual effects, animation and simulation fields;

[0085] S21.3, optimizing the generated virtual plant framework by adjusting the parameters of the L-System generator, such as the number of iterations, branch angles, etc.;

[0086] S21.4. Environmental mapping technology enhances visual realism. Obtain an environmental map. Obtain a high-quality environmental map. The environmental map should contain rich light and shadow information and texture details so that the virtual plant provides a realistic environmental mapping effect.

[0087] S21.5, creating an environment mapping material, in the TouchDesigner software, using the MAT (Material) function element to create an environment mapping material, using the obtained environment map as input, and setting corresponding material parameters, such as reflection intensity, highlight, etc.;

[0088] In TouchDesigner software, the MAT (Material) function element is used to create and edit materials, which can be applied to the surface of 3D objects to achieve various visual effects. Environment mapping is a special material technology that uses environment maps to simulate the effect of surfaces reflecting the surrounding environment;

[0089] S21.6, applying the environment mapping material, applying the created environment mapping material to the virtual plant frame, and adjusting the material parameters and lighting conditions so that the virtual plant can realistically reflect the light and shadow information of the surrounding environment to enhance the visual realism;

[0090] S21.7. Optimizing the visual effects of the virtual plants as needed, including adding shadow effects and adjusting the lighting direction, to enhance the realism of the virtual plants;

[0091] S21.8, real-time preview and rendering, through the real-time preview function of the TouchDesigner software, adjust and optimize the appearance and environment mapping effect of the virtual plant framework to ensure that the final rendering effect meets expectations;

[0092] The leaf falling effect submodule 22 is mainly responsible for simulating the dynamic effect of falling leaves in the wind and responding to the input of the airflow sensor to adjust the motion state of the falling leaves. The detailed steps are as follows:

[0093] S22.1. Initialize the SOP process, create a new SOP network in Houdini, and set the basic working environment and parameters;

[0094] Houdini is a powerful 3D computer graphics software widely used in film, television, games and visual effects production. It was developed by Canadian company SideFX and is known for its highly flexible node-based workflow and powerful simulation and effects tools.

[0095] S22.2, collecting physical property data of fallen leaves, including mass, shape and density, creating a 3D fallen leaf model library, including fallen leaves of different types and states, selecting and importing a fallen leaf model from the 3D fallen leaf model library, and adjusting the size, position and rotation of the model as needed;

[0096] S22.3, integrate the airflow sensor, connect the airflow sensor and set up the data interface, and create a custom node or script in the SOP process to receive and process the airflow sensor data;

[0097] S22.4. Set up leaf falling dynamics, add physical engine nodes in the SOP process, including gravity and wind nodes, connect the airflow sensor data to the corresponding parameters of the wind node, adjust the wind force according to the real-time data, and adjust other physical parameters to simulate the real leaf falling dynamics;

[0098] S22.5. Create a particle system. Add a particle system node in the SOP process, including a Scatter or POPSNetwork node. Use the fallen leaf model as an instance object of the particle system. Set the number, speed, and rotation parameters of the particles as needed.

[0099] The Scatter node is used to randomly distribute particles or instantiated objects on the surface or volume of a geometry. It can generate particles on the surface or inside the volume of a geometry according to the specified parameters. The Scatter node is usually used to create the initial particle distribution of a particle system.

[0100] POPS Network (or Particle Operators Network) is a node network used to create and manipulate particle systems. In Houdini, particle systems are defined by POPS Network nodes to define their behavior and appearance. You can add various particle operators (Particle Operators, referred to as POPs) to POPS Network, such as gravity, speed, life cycle control, etc., to adjust and control the dynamic behavior of particles.

[0101] S22.6. Add visual effects, add texture, lighting and shadow effects to the fallen leaves to enhance the visual effect, use Houdini's material system to assign appropriate materials to the fallen leaves, and adjust the lighting and camera parameters to obtain the best visual effect;

[0102] The sound and visual synchronization submodule 23 plays corresponding sound effects according to different elements in the visual scene, such as wind sound, water sound, animal sound, etc., and uses 3D sound effect technology to enable users to perceive the direction and distance of the sound, thereby increasing the sense of immersion. It also includes sound effect feedback related to user interaction, such as the footsteps of users walking in the scene. The detailed steps are as follows:

[0103] S23.1. Create river and bird models. Create a river model in Houdini's SOPs network, use the POPs particle system to create a bird model, and set the initial position, speed, and life cycle properties of the particles.

[0104] S23.2. Set up the CHOP network. Create a new CHOP network in Houdini and import the position and velocity animation data of the flying bird particles into the CHOP network.

[0105] In Houdini, CHOP stands for "Channel Operators" and is a node type used to process and manipulate time-related data. CHOP networks allow users to create, edit, and process these time data, which are usually represented as signals or channels. In the context of Houdini, CHOP networks are particularly suitable for processing animations, particle systems, sounds, motion data, and any other numerical data that changes over time.

[0106] S23.3. Create an event trigger. Create an event trigger in the CHOP network and write logic to detect whether the flying bird particle meets the trigger conditions, such as entering the view, speed change, etc. When the conditions are met, trigger an event to play the sound effect;

[0107] S23.4. Import and synchronize sound effects. Import the bird chirping sound effect file into Houdini. Use Houdini's audio tools or third-party plug-ins to synchronize the sound effect with the events in the CHOP network to ensure that the playback time of the sound effect is consistent with the animation timeline of the visual element.

[0108] The multi-sensory stimulation module 4 includes an ESP32 touch submodule 41, an atomization drive module submodule 42, a skin electrical sensor submodule 43 and an airflow sensor submodule 44;

[0109] The ESP32 touch submodule 41 provides a touch input function, allowing the user to communicate with the system intuitively and conveniently through a touch interaction interface, and can detect the user's touch position, touch strength and touch duration information, provide the system with user input data, and implement menu selection and parameter setting operations in combination with the software interface;

[0110] The atomization drive module submodule 42 controls the opening, closing and adjustment of atomization equipment, such as humidifiers and atomizers, and simulates the fog and water mist effects in the natural environment according to system instructions or preset conditions to enhance the user's biophilic experience;

[0111] The galvanic skin sensor submodule 43 measures the galvanic skin response (GSR) of the user, that is, the change in skin resistance. The galvanic skin response is a physiological indicator of emotional state and can be used to analyze the user's emotional changes and stress level;

[0112] The airflow sensor submodule 44 measures airflow velocity, direction and pressure parameters in the environment, and provides the system with real-time environmental data so as to adjust the simulation effect or trigger corresponding functions.

[0113] Specific implementation method When the user starts the system, the central processing module 7 is responsible for the initialization of the entire system to ensure the normal operation of each module. The UI design and data visualization module 5 is responsible for displaying the system interface and guiding the user to interact. The virtual natural scene module 2 generates a corresponding virtual environment based on the preset or user-selected natural scene. This module uses advanced graphics rendering technology to ensure the realism and immersion of the scene. The interactive experience module 1 is responsible for receiving the user's interactive instructions, such as gestures, voice, touch, etc. The user can interact with the system through devices such as gesture recognizers, microphones, and touch screens. The central processing module 7 updates the scene content in the virtual natural scene module 2 in real time according to the user's interactive instructions, such as changing the perspective, moving objects, etc. The multi-sensory stimulation module 4 provides the user with multiple sensory stimulations according to the instructions of the virtual natural scene module 2 and the central processing module 7. Visual stimulation presents the virtual environment through the display screen, auditory stimulation plays natural sounds through the speaker, and tactile stimulation simulates different objects through vibration devices, temperature control and other devices. The sense of touch and olfactory stimulation can simulate the smell in the natural environment through the odor releaser. These sensory stimuli together constitute an immersive biophilic experience. The user feedback module 3 monitors the user's physiological parameters in real time through biosensors and other devices, and transmits the data to the central processing module 7. The central processing module 7 combines the user's behavioral data and virtual environment information to accurately analyze the user's emotional state. These emotional data can provide the system with a deeper understanding of the user in order to optimize the subsequent interactive experience. The UI design and data visualization module 5 is responsible for displaying the user's physiological data, emotional state and other information in the form of charts, graphs, etc. These data visualization results can help system developers better understand user needs and behavior patterns so as to optimize and improve the system. The experience feedback module 6 is responsible for collecting user feedback on the system. These feedback data will be transmitted to the central processing module 7 for processing and analysis. Based on these feedback data, the system developer can continuously iterate and optimize the system to improve the accuracy and precision of the user experience.

[0114] The same or similar reference numerals correspond to the same or similar components;

[0115] The terms used in the drawings to describe positional relationships are only used for illustrative purposes and should not be construed as limiting this patent;

[0116] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not limitations on the implementation methods of the present invention. For ordinary technicians in the relevant field, other different forms of changes or modifications can be made on the basis of the above description. It is not necessary and impossible to list all the implementation methods here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. A multi-sensory interactive system based on biophilic experience, It is characterized in that It includes an interactive experience module (1), a virtual natural scene module (2), a user feedback module (3), a multi-sensory stimulation module (4), a UI design and data visualization module (5), an experience feedback module (6) and a central processing module (7); The interactive experience module (1) is used to receive the user's voice and gesture input, and convert the user's intention into instructions that the system can understand through voice recognition and image recognition technology, providing an intuitive and natural interaction method so that the user can easily interact with the system. In combination with the UI design and data visualization module (5), a user-friendly operation interface and feedback are provided; The virtual natural scene module (2) uses 3D modeling and real-time rendering technology to create a realistic natural environment, enhance the user's sense of immersion, and dynamically adjust the elements in the scene according to the user's interactive instructions and the system's control signals, including light and shadow changes and wind simulation, to provide the user with an experience similar to or beyond the real environment, thereby promoting the user's physical and mental relaxation and pleasure; The user feedback module (3) uses biosensor technology to monitor the user's physiological parameters in real time, including heart rate and skin galvanic response, analyze the user's physiological data, evaluate the user's emotional state and stress level, and transmit the user feedback data to the system for optimizing system output and providing personalized suggestions; The multi-sensory stimulation module (4) uses a variety of sensory stimulation methods to enhance the user's sense of immersion and reality in the virtual natural environment, provides visual, auditory and tactile multi-sensory stimulation, including stereoscopic images, natural sounds and vibration feedback, and dynamically adjusts the type and intensity of sensory stimulation according to changes in the virtual natural scene and user behavior. Through multi-sensory stimulation, the user is helped to better perceive and understand the virtual environment, thereby improving the effect of the interactive experience; The UI design and data visualization module (5) designs an intuitive and easy-to-use user interface to facilitate user interaction with the system, presents user data and system status to users in a visual manner through charts and animations, provides data analysis and mining functions, and helps users better understand their needs and system performance; The experience feedback module (6) collects and analyzes users' overall evaluation and suggestions on the multi-sensory interactive system, collects user feedback through questionnaires, interviews and social media channels, organizes and analyzes user feedback, identifies problems and room for improvement in the system, converts user feedback into specific improvement measures and product development suggestions, and continuously improves the user experience and satisfaction of the system; The central processing module (7) is used to receive input data from each module, perform unified processing and analysis, coordinate the work of each module according to user feedback and interactive instructions, ensure the overall smooth and efficient operation of the system, update the system status information in real time, and pass it to the UI design and data visualization module (5) for display; The interactive experience module (1) is connected to the central processing module (7), the virtual natural scene module (2) is connected to the central processing module (7), the virtual natural scene module (2) is connected to the multi-sensory stimulation module (4), the user feedback module (3) is connected to the central processing module (7), the multi-sensory stimulation module (4) is connected to the central processing module (7), the UI design and data visualization module (5) is connected to the central processing module (7), and the experience feedback module (6) is connected to the central processing module (7).

2. According to claim 1, the multi-sensory interactive system based on biophilic experience, It is characterized in that The interactive experience module (1) comprises a user initialization submodule (11), a device configuration and calibration submodule (12), a real-time interaction processing submodule (13), a situation simulation and triggering submodule (14), a feedback display and visualization submodule (15), a personalized option and community interaction submodule (16) and a user exit settlement submodule (17); The user initialization submodule (11) is used for initial user guidance and system initialization. When the user enters the healing space, the user is prompted to walk to a designated location through voice or interface guidance. After the user sits down, the display of the large screen and the small screen is started to prepare for entering the virtual natural scene. The device configuration and calibration submodule (12) is used for configuration and calibration of the experience device. When the small screen lights up, it prompts the user to wear the experience device and perform necessary calibration operations, monitors the connection status and signal quality of the sensor, and ensures data accuracy; The real-time interaction processing submodule (13) is used to receive interaction instructions input by the user through body movements, touch, and blowing, analyze these instructions, and convert them into signals that can be recognized by the system, and transmit them to the central processing module (7) for processing, and update the response in the virtual environment in real time, including the contraction of virtual plants and the flight of birds; The scenario simulation and triggering submodule (14) is used to manage the interaction logic and trigger conditions in different scenarios, and trigger the corresponding virtual environment changes according to the user's interaction instructions and the current scenario. When the user touches the physical plant, the virtual plant is triggered to shrink; when the user blows air, the flock of birds is triggered to fly; The feedback display and visualization submodule (15) is responsible for displaying the user's emotional state and other related information on the screen, analyzing the user's emotional fluctuations based on the data collected by the experience device, and presenting them on the small screen in a visual manner, providing real-time display of data such as experience duration and emotional indicators; The personalized options and community interaction submodule (16) provides personalized interaction options and relaxation training, allowing users to choose to continue relaxation, meditation and breathing training functions according to their own needs, supports community functions, allows users to share their experiences and relaxation training, and increases the social interactivity of the system; The user exit settlement submodule (17) performs necessary exit operations when the user takes off the device and prepares to leave, saves the user's experience data for subsequent analysis and improvement, and settles the user's experience fee.

3. The multi-sensory interactive system based on biophilic experience according to claim 1, It is characterized in that The virtual natural scene module (2) comprises a visual channel submodule (21), a falling leaf dynamic effect submodule (22) and a sound effect and visual synchronization submodule (23); The visual channel submodule (21) uses TouchDesigner software and L-System technology to construct complex natural elements, including plants, animals and terrain, and enhances the realism and interactive elements of the scene through high-resolution environment maps and light and shadow effects, including dynamic weather systems and day and night changes, to provide rich visual content. The detailed steps are as follows: S21.1, L-System builds a virtual plant framework Define L-System rules, and define corresponding L-System rules according to the characteristics of the desired plants, wherein the L-System rules will determine the growth pattern and morphology of the plants; S21.2, creating an L-System generator, in the TouchDesigner software, using SOP functional elements to create an L-System generator, the L-System generator will gradually generate the framework of the virtual plant according to the defined L-System rules; S21.3, optimizing the generated virtual plant framework by adjusting the parameters of the L-System generator; S21.

4. Environmental mapping technology enhances visual realism. Obtain an environmental map. Obtain a high-quality environmental map. The environmental map should contain rich light and shadow information and texture details so that the virtual plant provides a realistic environmental mapping effect. S21.5, creating an environment mapping material, in the TouchDesigner software, using the MAT function element to create an environment mapping material, using the obtained environment map as input, and setting corresponding material parameters; S21.6, applying the environment mapping material, applying the created environment mapping material to the virtual plant frame, and adjusting the material parameters and lighting conditions so that the virtual plant can realistically reflect the light and shadow information of the surrounding environment to enhance the visual realism; S21.

7. Optimizing the visual effects of the virtual plants as needed, including adding shadow effects and adjusting the lighting direction, to enhance the realism of the virtual plants; S21.8, real-time preview and rendering, through the real-time preview function of the TouchDesigner software, adjust and optimize the appearance and environment mapping effect of the virtual plant framework to ensure that the final rendering effect meets expectations; The leaf-falling dynamic effect submodule (22) is used to simulate the dynamic effect of leaves falling in the wind and to respond to the input of the airflow sensor to adjust the motion state of the leaves falling. The detailed steps are as follows: S22.

1. Initialize the SOP process, create a new SOP network in Houdini, and set the basic working environment and parameters; S22.2, collecting physical property data of fallen leaves, including mass, shape and density, creating a 3D fallen leaf model library, including fallen leaves of different types and states, selecting and importing a fallen leaf model from the 3D fallen leaf model library, and adjusting the size, position and rotation of the model as needed; S22.3, integrate the airflow sensor, connect the airflow sensor and set up the data interface, and create a custom node or script in the SOP process to receive and process the airflow sensor data; S22.

4. Set up leaf falling dynamics, add physical engine nodes in the SOP process, including gravity and wind nodes, connect the airflow sensor data to the corresponding parameters of the wind node, adjust the wind force according to the real-time data, and adjust other physical parameters to simulate the real leaf falling dynamics; S22.

5. Create a particle system. Add a particle system node in the SOP process, including a Scatter or POPS Network node. Use the fallen leaf model as an instance object of the particle system. Set the number, speed, and rotation parameters of the particles as needed. S22.

6. Add visual effects, add texture, lighting and shadow effects to the fallen leaves to enhance the visual effect, use Houdini's material system to assign appropriate materials to the fallen leaves, and adjust the lighting and camera parameters to obtain the best visual effect; The sound and visual synchronization submodule (23) plays corresponding sound effects according to different elements in the visual scene, uses 3D sound effect technology to enable users to perceive the direction and distance of the sound, and increases the sense of immersion. It also includes sound effect feedback related to user interaction. The detailed steps are as follows: S23.

1. Create river and bird models. Create a river model in Houdini's SOPs network, use the POPs particle system to create a bird model, and set the initial position, speed, and life cycle properties of the particles. S23.

2. Set up the CHOP network. Create a new CHOP network in Houdini and import the position and velocity animation data of the flying bird particles into the CHOP network. S23.

3. Create an event trigger. Create an event trigger in the CHOP network and write logic to detect whether the flying bird particle meets the trigger condition. When the condition is met, an event is triggered to play the sound effect. S23.

4. Import and synchronize sound effects. Import the bird chirping sound effect file into Houdini. Use Houdini's audio tools or third-party plug-ins to synchronize the sound effect with events in the CHOP network to ensure that the playback time of the sound effect is consistent with the animation timeline of the visual element.

4. The multi-sensory interactive system based on biophilic experience according to claim 1, It is characterized in that The multi-sensory stimulation module (4) comprises an ESP32 touch submodule (41), an atomization drive module submodule (42), a skin electrical sensor submodule (43) and an airflow sensor submodule (44); The ESP32 touch submodule (41) provides a touch input function, allowing the user to communicate with the system intuitively and conveniently through a touch interaction interface, and can detect the user's touch position, touch strength and touch duration information, provide the system with user input data, and realize menu selection and parameter setting operations in combination with the software interface; The atomization drive module submodule (42) controls the opening, closing and adjustment of the atomization device, and simulates the fog and water mist effects in the natural environment according to system instructions or preset conditions, thereby enhancing the user's biophilic experience; The galvanic skin sensor submodule (43) measures the galvanic skin response of the user, that is, the change in skin resistance; The airflow sensor submodule (44) measures airflow velocity, direction and pressure parameters in the environment, and provides the system with real-time environmental data so as to adjust the simulation effect or trigger corresponding functions.

Images