Interactive demonstration control system for game motor home

Abstract

The invention relates to the technical field of interactive systems, and particularly discloses a game motor home interactive demonstration control system which comprises a sensing interaction module, an intelligent scene engine, a multi-mode output module, a cooperative control module and a safety monitoring module. The perception interaction module integrates physiological, action and environment sensors, deeply analyzes explicit instructions and implicit demands (such as inferring emotions through a heart rate variability rate) of a user in combination with a Transform model, and generates a structured interaction intention label; according to the physiological, action and environment multi-dimensional data fusion mechanism, the system can accurately capture the user state, an accurate basis is provided for follow-up scene adjustment, and the fitting degree of interaction response is remarkably improved; the intelligent scene engine constructs a user immersion model through deep reinforcement learning, calculates an immersion index based on parameters such as heart rate, micro-expression and action angular velocity, and dynamically triggers a high immersion, balance or guide mode.

Description

Technical Field

[0001] This invention relates to the field of interactive systems technology, specifically to a control system for interactive demonstrations in a game RV. Background Technology

[0002] With the integration of immersive interactive technology and mobile entertainment scenarios, gaming RVs, as a new type of entertainment carrier combining mobile space and interactive experience, are gradually becoming an important display format for cultural tourism, sports events, and other scenarios. However, existing gaming RV interactive systems have the following technical limitations:

[0003] Single interaction dimension: Traditional systems rely heavily on explicit input methods such as gamepads and touch controls, lacking awareness of users' physiological states (such as emotional fluctuations and fatigue levels) and implicit intentions. This results in a discrepancy between the interaction response and the user's actual needs, making it difficult to achieve personalized experience adaptation.

[0004] Insufficient immersion and spatial adaptability: The integration between the virtual scene and the physical space of the RV is low, the visual, auditory and tactile outputs lack coordination and linkage, and the presentation mode cannot be dynamically adjusted according to the user's position and actions, which easily creates a "virtual-reality disconnect".

[0005] Cross-device collaboration and load balancing deficiencies: When connecting multiple terminals such as AR glasses and smart wearable devices, problems such as data synchronization delay and unreasonable allocation of rendering tasks often occur, resulting in frame rate drop and interaction stuttering in high-load scenarios.

[0006] Lack of safety mechanisms: The lack of real-time monitoring of user physiological load (such as abnormal heart rate caused by excessive excitement) and device operating status may lead to safety risks in high-intensity interactive scenarios, and there is no graded emergency response mechanism.

[0007] Poor scene adaptability: The game's story branches and difficulty adjustments rely on preset scripts, which cannot be dynamically optimized according to the user's immersion level, making it difficult to cater to both the guidance needs of novice users and the in-depth exploration needs of experienced users.

[0008] These issues result in significant shortcomings in existing systems regarding immersive experience, intelligent adaptation, and security and reliability, limiting the application potential of gaming RVs as mobile interactive platforms. Summary of the Invention

[0009] To address the shortcomings of existing technologies, this invention provides a control system for interactive demonstrations in game RVs, which solves the problems mentioned in the background section.

[0010] To achieve the above objectives, the present invention provides the following technical solution: a game RV interactive demonstration control system, comprising: a perception and interaction module, an intelligent scene engine, a multimodal output module, a collaborative control module, and a safety monitoring module;

[0011] The perception and interaction module is used to collect user physiological characteristics, movement trajectories, and environmental parameters in real time, generating a multi-dimensional interaction data set, which includes the user's heart rate variability. Facial micro-expression feature vectors angular velocity of limb movements Interior lighting intensity of the RV Background noise decibel value and the real-time distance between the user and the device ;

[0012] The intelligent scene engine is used to construct a user immersion model based on a multi-dimensional interaction data set and to calculate a user immersion index using a deep reinforcement learning algorithm. The game scene's story branches, character interaction modes, and environmental rendering parameters are dynamically adjusted based on the immersion index.

[0013] The multimodal output module is used to convert the adjustment instructions generated by the intelligent scene engine into holographic visual signals, force feedback tactile signals and spatial audio signals, so as to realize the fusion presentation of virtual game elements and RV physical space;

[0014] The collaborative control module is used to establish a dynamic communication link between the game RV and the cloud metaverse server and other related smart devices, and to realize real-time interaction and state consistency maintenance of cross-device scene data based on the spatiotemporal synchronization protocol.

[0015] The safety monitoring module is used to monitor the user's physiological load, equipment operating status and environmental safety thresholds in real time. When preset safety conditions are triggered, the scenario degradation or emergency interruption mechanism is automatically activated.

[0016] Preferably, the perception and interaction module includes a user state perception unit, an environment perception unit, and an interaction intent parsing unit;

[0017] The user status sensing unit employs an array-type infrared motion capture device, an 8K eye tracker, and a flexible biosensor to collect data on the three-dimensional coordinate changes of the user's limb joints, the coordinates of the gaze focus point, and skin conductance responses. brainwaves wave frequency ;

[0018] The environmental sensing unit collects the temperature gradient inside the RV through distributed fiber optic sensors, acoustic array microphones, and a temperature and humidity monitor. Humidity distribution Spectral distribution of illumination and voiceprint characteristics of user voice commands ;

[0019] The interaction intent parsing unit performs semantic mapping on the multi-dimensional interaction data set based on the Transformer model, extracts explicit user commands and implicit needs, and generates structured interaction intent tags.

[0020] Preferably, the intelligent scene engine includes an immersion calculation unit, a scene adaptation unit, and a plot deduction unit;

[0021] The immersion calculation unit is used to calculate the user immersion index using the following formula. :

[0022] ;

[0023] In the formula, These are weighting coefficients, ranging from 0.2 to 0.3, and their sum is 1. This serves as the baseline vector for emotions, with a default value of [0.3, 0.5, 0.2], corresponding to calm, excitement, and focus.

[0024] The scene adaptation unit is used to adjust the immersion index. Calculate scene adaptation coefficient ,when When high immersion mode is triggered When, the balance mode is triggered; when At that time, the boot mode is triggered.

[0025] Preferably, the multimodal output module includes a holographic projection unit, a force feedback unit, and a spatial audio unit;

[0026] The holographic projection unit uses a combination of a 4K laser holographic film and a motion tracking projector to project game characters onto the interior wall of the RV, dynamically adjusting the projection scaling ratio using the following formula. :

[0027] ;

[0028] In the formula, The preset maximum interaction distance;

[0029] The force feedback unit is integrated into the RV seat, control handle, and floor contact points, and adjusts according to the virtual collision intensity in the game scene. Output corresponding pressure value ,in, ;

[0030] The spatial audio unit employs a 64-channel sound array, based on the user's head orientation angle. Calculate the difference in acoustic delay To achieve three-dimensional spatial positioning of virtual sound sources, the delay difference calculation formula is: .

[0031] Preferably, the collaborative control module includes a cloud synchronization unit, a cross-device adaptation unit, and a dynamic load balancing unit;

[0032] The cloud synchronization unit uses a 5G-MEC edge computing node to synchronize the real-time scene data of the game RV with the Metaverse cloud server at the millisecond level. The synchronized content includes the coordinates of the user's virtual avatar. Scene item status hash value and plot progress timestamps ;

[0033] The cross-device adaptation unit is used to identify the hardware performance parameters of associated smart devices (such as AR glasses or smartwatches worn by the user) and generate a device capability matrix. The rendering tasks are dynamically allocated according to the matrix (e.g., AR glasses are responsible for near-eye display, and RV holographic devices are responsible for distant scene rendering).

[0034] The dynamic load balancing unit calculates the device load factor using the following formula. ,when At that time, 30% of the rendering tasks will be automatically migrated to the cloud:

[0035] .

[0036] Preferably, the safety monitoring module includes a physiological safety unit, an equipment redundancy unit, and an emergency response unit;

[0037] The physiological safety unit is used to set the user's physiological safety threshold range. When any parameter in the multi-dimensional interactive data set exceeds the threshold, a safety warning signal is generated.

[0038] The equipment redundancy unit adopts a dual power supply module and a hot standby computing node. The heartbeat detection mechanism ensures that when the main equipment fails, the standby equipment can take over control within 50ms.

[0039] Upon receiving a safety warning signal, the emergency response unit executes a three-level response strategy: Level 1 response, Level 2 response, and Level 3 response.

[0040] Preferably, the plot deduction unit adjusts the plot branches based on the user immersion index and interaction intent tags according to the following rules:

[0041] like If the interaction intent tag includes "exploration", it will trigger a hidden story branch and unlock hidden interaction points in the RV's physical space;

[0042] like Furthermore, if the interaction intent tag includes "competition", the intensity of the character's confrontation will be increased, and the vibration frequency of the RV's force feedback device will be increased simultaneously.

[0043] like This will automatically simplify the storyline quest chain and increase the frequency of NPC guidance.

[0044] Preferably, the acoustic delay difference of the spatial audio unit needs to meet the following constraint: when the user's head orientation angle changes, the delay difference is dynamically compensated by a coefficient to avoid audio positioning deviation.

[0045] Preferably, the device capability matrix of the cross-device adaptation unit is a 3×3 matrix, where each element represents the first... Class of equipment for the first The processing efficiency of the class rendering task, with a value between 0 and 1, where, Corresponding to AR glasses, RV host, and cloud server, It corresponds to close-up rendering, distant rendering, and physics engine calculation.

[0046] Preferably, the physiological safety unit is also used to calculate the user's physiological load index using the following formula, and when that occurs, directly triggers a secondary response:

[0047] ;

[0048] In the formula, 20, 50, and 15 are the maximum safe thresholds for heart rate variability, skin conductance response, and limb movement angular velocity, respectively.

[0049] This invention provides a control system for interactive demonstrations in game RVs, which has the following beneficial effects:

[0050] 1. The perception and interaction module integrates multiple types of sensors, including physiological, action, and environmental sensors. Combined with the Transformer model, it performs in-depth analysis of the user's explicit commands and implicit needs (such as inferring emotions through heart rate variability) to generate structured interaction intent tags. This multi-dimensional data fusion mechanism of physiological, action, and environmental factors enables the system to accurately capture the user's state, providing a precise basis for subsequent scene adjustments and significantly improving the fit of the interaction response.

[0051] 2. The intelligent scene engine constructs a user immersion model through deep reinforcement learning, and calculates the immersion index based on parameters such as heart rate, micro-expressions, and motion angular velocity. It can dynamically trigger high immersion, balanced, or guided modes; for example, when the user's immersion is low, the plot can be automatically simplified and NPC guidance can be added, and when the user is highly immersive, hidden branches can be unlocked to achieve "personalized" scene adaptation, which can satisfy the exploration needs of experienced users and reduce the operation threshold of novice users.

[0052] 3. The multimodal output module achieves deep integration of virtual elements with the physical space of the RV through the coordinated linkage of holographic projection, force feedback, and spatial audio. For example, the holographic projection dynamically adjusts the scaling ratio according to the user's distance, the force feedback device combines virtual collision intensity with the user's actions to output corresponding tactile sensations, and the 64-channel sound array achieves three-dimensional sound source localization. This multi-sensory collaborative mechanism greatly enhances the sense of immersion and eliminates the "virtual-reality disconnect".

[0053] 4. The collaborative control module achieves millisecond-level cloud synchronization through 5G-MEC edge computing, dynamically allocates rendering tasks based on the device capability matrix (e.g., AR glasses are responsible for near-view rendering, and the RV host handles far-view rendering), and monitors the device status in real time through the load coefficient formula, automatically migrating tasks to the cloud when under high load; this mechanism ensures low latency and high frame rate in multi-device linkage scenarios, avoiding stuttering or synchronization deviation.

[0054] 5. The safety monitoring module sets safety thresholds for heart rate, brain waves, etc. through the physiological safety unit. The equipment redundancy unit adopts dual power supply and hot standby nodes to achieve fault switching within 50ms. The emergency response unit executes a three-level response strategy (reduce stimulation intensity → pause prompt → emergency help). This "monitoring-redundancy-response" full-link safety mechanism effectively avoids physiological risks and equipment failure risks in high-intensity interactions, ensuring the reliability of system operation.

[0055] 6. The plot deduction unit automatically adjusts plot branches based on immersion index and interaction intention tags (such as unlocking hidden interaction points when highly immersive and inclined to "explore"), and simultaneously optimizes the intensity of character confrontation and the frequency of force feedback; this dynamic deduction mechanism allows game content to break free from the constraints of fixed scripts and evolve in real time with user behavior, significantly improving user stickiness and replay rate.

[0056] In summary, this invention comprehensively enhances the immersiveness, intelligence, and reliability of the game RV interactive system through innovative designs such as multi-dimensional perception, dynamic adaptation, multi-modal fusion, and security redundancy, providing an efficient solution for interactive demonstrations in mobile scenarios. Attached Figure Description

[0057] Figure 1 This is a block diagram illustrating the principle of a game RV interactive demonstration control system according to the present invention. Detailed Implementation

[0058] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0059] like Figure 1 As shown, the present invention provides a technical solution: a control system for interactive demonstrations in a game RV, comprising: a perception and interaction module, an intelligent scene engine, a multimodal output module, a collaborative control module, and a safety monitoring module; the perception and interaction module is used to collect user physiological characteristics, movement trajectories, and environmental parameters in real time, generating a multi-dimensional interactive data set, wherein the multi-dimensional interactive data set includes the user's heart rate variability rate. Facial micro-expression feature vectors angular velocity of limb movements Interior lighting intensity of the RV Background noise decibel value and the real-time distance between the user and the device The intelligent scene engine is used to construct a user immersion model based on a multi-dimensional interactive data set and to calculate the user immersion index using a deep reinforcement learning algorithm. The system dynamically adjusts the plot branches, character interaction modes, and environmental rendering parameters of the game scene based on the immersion index. The multimodal output module converts the adjustment instructions generated by the intelligent scene engine into holographic visual signals, force feedback tactile signals, and spatial audio signals to achieve the fusion presentation of virtual game elements and the physical space of the RV. The collaborative control module establishes a dynamic communication link between the game RV and the cloud metaverse server and other related intelligent devices, and realizes real-time interaction and status consistency maintenance of cross-device scene data based on the spatiotemporal synchronization protocol. The safety monitoring module monitors the user's physiological load, device operating status, and environmental safety thresholds in real time, and automatically activates the scene degradation or emergency interruption mechanism when preset safety conditions are triggered.

[0060] More specifically, the perception and interaction module includes a user state perception unit, an environment perception unit, and an interaction intent parsing unit;

[0061] The user status sensing unit employs an array-type infrared motion capture device, an 8K eye tracker, and a flexible biosensor to collect data on the three-dimensional coordinate changes of the user's limb joints, the coordinates of the gaze focus point, and skin conductance responses. brainwaves wave frequency ;

[0062] The environmental sensing unit collects the temperature gradient inside the RV through distributed fiber optic sensors, acoustic array microphones, and a temperature and humidity monitor. Humidity distribution Spectral distribution of illumination and voiceprint characteristics of user voice commands ;

[0063] The interaction intent parsing unit performs semantic mapping on the multi-dimensional interaction data set based on the Transformer model, extracts explicit user instructions (such as voice commands and gestures) and implicit needs (such as emotional fluctuations inferred from heart rate changes), and generates structured interaction intent tags.

[0064] The user status perception unit employs a 16-point array infrared motion capture device (sampling frequency 120Hz), which emits infrared light and receives reflected signals to collect real-time 3D coordinate changes (accuracy ±0.5mm) of 23 key joints of the user's limbs (such as shoulder, elbow, wrist, and hip), generating a limb movement trajectory dataset. It also features an 8K resolution eye tracker (120° field of view) that captures the coordinates of the user's gaze focus point (sampling rate 200Hz) using corneal reflection, identifying the duration and frequency of the user's attention to specific elements in the virtual scene. Furthermore, it deploys flexible biosensors (attached to the user's wrist and forehead) to simultaneously collect skin conductance responses. (Reflects emotional arousal, measured in 0-100 μS) and brain waves wave frequency (Reflects the degree of relaxation, range 8-13Hz).

[0065] Data output: Generates data including angular velocity of limb movements. Area where eyes linger and The user status dataset is sampled at 10ms intervals to ensure real-time performance.

[0066] Environmental sensing unit: Distributed fiber optic sensors (one monitoring point deployed every 50cm) to collect temperature gradients inside the RV. (Measuring range -10~50℃, accuracy ±0.1℃) and humidity distribution (Measuring range 0~100%RH); 32-channel acoustic array microphone (covering the RV's 360° space), to collect background noise decibel values. (Range 30~120dB), on the other hand, the voiceprint features of the user's voice commands are obtained through a voiceprint feature extraction algorithm. (For authentication and command recognition); Illumination spectrum analyzer (sampling frequency 1Hz) to collect the spectral distribution of illumination inside the RV. (Covering the 400~760nm visible light band), providing an adaptation basis for virtual scene lighting rendering.

[0067] Data output: Generates a spatiotemporal distribution matrix of environmental parameters, in which temperature and humidity data are updated every 200ms, and soundprint and illumination data are synchronized in real time.

[0068] Interaction Intent Parsing Unit: Employs a pre-trained Transformer model (containing a 6-layer encoder and a 6-layer decoder) to process the multi-dimensional data set output by the user state perception unit and the environment perception unit (including... Semantic mapping is performed on 12 types of parameters: For explicit instructions (such as the voice command "open the treasure chest" and the gesture "wave"), strong feature signals in the data are located through the model's multi-head attention mechanism to directly generate instruction labels; for implicit requirements (such as heart rate variability)... A sudden increase in anxiety and frequent shifting of gaze indicate decreased interest. Through time-series correlation analysis of multiple parameters using a model (window size 5s), potential user needs can be inferred.

[0069] Output: Generate structured interaction intent tags (such as "explore", "competition", "relax", "help" etc.). When the tag confidence is ≥0.85, trigger the subsequent scene adjustment process to ensure the accuracy of intent parsing.

[0070] More specifically, the intelligent scene engine includes an immersion calculation unit, a scene adaptation unit, and a plot deduction unit;

[0071] The immersion calculation unit is used to calculate the user immersion index using the following formula. :

[0072] ;

[0073] In the formula, These are weighting coefficients, ranging from 0.2 to 0.3, and their sum is 1. This serves as the baseline vector for emotions, with a default value of [0.3, 0.5, 0.2], corresponding to calm, excitement, and focus.

[0074] The scene adaptation unit is used to adjust the immersion index. Calculate scene adaptation coefficient ,when When this occurs, a high-immersion mode is triggered (plot branch complexity increased by 30%, character AI response speed increased by 50%); when When this happens, a balanced mode is triggered (story branches maintain medium complexity, and environmental interaction cues are enabled); when When this happens, the tutorial mode is triggered (simplifying the operation process and adding a new player tutorial role).

[0075] Immersion calculation unit: based on immersion index The threshold range triggers different scene adaptation modes to achieve accurate matching of "user state - scene complexity".

[0076] Mode Triggering Rules: High Immersion Mode: When This feature is activated at specific times and is suitable for high-investment users. Specific adjustments include: a 30% increase in the complexity of story branches (unlocking multiple ending options); a 50% increase in character AI response speed (interaction latency ≤100ms); and improved environmental rendering accuracy (lighting and shadow detail resolution increased to 4K).

[0077] Balanced mode: when This feature is activated at certain times and is suitable for users with moderate investment of time. Specific adjustments include: maintaining a moderate level of complexity in the story branches (main story + 1-2 side quests); enabling environmental interaction prompts (highlighting interactive objects and displaying dynamic guide arrows); and maintaining basic rendering accuracy (2K resolution for lighting and shadow detail).

[0078] Bootstrapping mode: When This feature is activated on demand and is suitable for low-investment or novice users. Specific adjustments include: simplified operation process (merging consecutive operation steps to reduce the number of inputs); addition of a tutorial character (NPC real-time voice prompts, frequency ≥ 1 time / 30 seconds); and reduced difficulty in combat (enemy AI attack frequency reduced by 40%).

[0079] Story progression unit: Combining immersion index and interaction intent tags (such as "explore" and "competition"), the storyline is dynamically adjusted to enhance scene personalization.

[0080] Typical rule example: If And the intention is "exploration": triggering hidden story branches (such as unlocking the star map puzzle projected on the RV ceiling); if And the intention is "competitive": increase the character's combat strength (enemy health increased by 20%), and simultaneously increase the vibration frequency of the force feedback device (from 5Hz to 10Hz); if Automatically simplify the main quest chain (remove side quests), increase NPC tutorial frequency to once every 15 seconds.

[0081] By over-quantifying the immersion index and linking it to scene mode switching, this engine breaks through the limitations of traditional fixed scripts and achieves closed-loop control of "real-time user status - dynamic scene adjustment". This not only meets the needs of experienced users for in-depth experience, but also lowers the operating threshold for novice users, significantly improving the system's adaptability and user stickiness.

[0082] More specifically, the multimodal output module includes a holographic projection unit, a force feedback unit, and a spatial audio unit;

[0083] The holographic projection unit uses a combination of a 4K laser holographic film and a motion tracking projector to project game characters onto the interior wall of the RV, dynamically adjusting the projection scaling ratio using the following formula. :

[0084] ;

[0085] In the formula, The preset maximum interaction distance;

[0086] The force feedback unit is integrated into the RV seat, control handle, and floor contact points, and adjusts according to the virtual collision intensity in the game scene. Output corresponding pressure value ,in, ;

[0087] The spatial audio unit employs a 64-channel sound array, based on the user's head orientation angle. Calculate the difference in acoustic delay To achieve three-dimensional spatial positioning of virtual sound sources, the delay difference calculation formula is: The acoustic delay difference of the spatial audio unit must meet the following constraints: when the user's head orientation angle changes, the delay difference dynamic compensation coefficient is applied to avoid audio positioning deviation.

[0088] Holographic projection unit: Utilizing a combination of 4K laser holographic film (transmittance ≥85%) and a motion tracking projector (projection ratio 1.2:1), the holographic film is adhered to the interior wall of the RV (covering an area ≥8㎡), and the projector is positioned in the center of the RV's roof. An infrared tracking module continuously monitors the real-time distance between the user and the device. (Sampling frequency 50Hz).

[0089] The projection scaling of game characters / scenes is dynamically adjusted based on the user's distance to ensure that virtual elements are adapted to the user's viewpoint; the scaling calculation formula is as follows: In the formula, The maximum interaction distance is preset (default 5m, which can be adjusted according to the space of the RV). This is the scaling factor (ranging from 0.7 to 1.2).

[0090] When the user approaches the device hour, The projection magnifies 1.1 times to avoid visual strain at close range; when the user moves away from the device... hour, The projection is reduced to 0.7 times to ensure a panoramic view.

[0091] Force feedback unit: The force feedback device is integrated at three key interaction points:

[0092] RV seat backrest (built-in 8 linear vibrators, pressure range 0-50N); control handle (built-in piezoelectric ceramic plate, supports 0.1-1N pressure output); floor contacts (4 pressure sensors distributed in the user activity area).

[0093] Based on the virtual collision intensity in the game scene and the user's actions, a matching pressure sensitivity is output; the pressure value calculation formula is as follows: In the formula, This represents the virtual collision intensity (output by the game engine, with a value of 0-10). Angular velocity of user limb movements (unit) (Values ​​range from 0 to 15).

[0094] For example: virtual collision intensity And the user's angular velocity hour, The seat and handle output 6N pressure simultaneously to enhance the realism of the collision.

[0095] Spatial audio unit: Employs a 64-channel sound array (distributed around the RV, spaced 30cm apart), equipped with MEMS microphones and a digital signal processor (DSP), to acquire the user's head orientation angle {Tbx} in real time (obtained via an infrared head tracker, with an accuracy of ±1°). It calculates the sound wave delay difference to achieve three-dimensional spatial positioning of the virtual sound source, and a dynamic compensation mechanism ensures audio and visual synchronization.

[0096] Delay difference calculation formula: In the formula, 340 represents the head-facing angle (range -90° to 90°); 340 represents the speed of sound in air (m / s). The delay difference between the left and right channels (in seconds) is used to simulate the location of the sound source.

[0097] Constraint: When the head orientation angle changes at a certain rate When, activate dynamic compensation coefficient By outputting the audio signal 0.2ms in advance, positioning deviations are avoided when the head rotates rapidly.

[0098] Through multimodal collaboration involving visual scaling adaptation, tactile intensity linkage, and auditory location synchronization, this module eliminates the "separation" between virtual scenes and physical spaces, enabling users to obtain a consistent immersive experience across visual, tactile, and auditory dimensions, significantly enhancing the realism of interaction.

[0099] More specifically, the collaborative control module includes a cloud synchronization unit, a cross-device adaptation unit, and a dynamic load balancing unit;

[0100] The cloud synchronization unit uses a 5G-MEC edge computing node to synchronize the real-time scene data of the game RV with the Metaverse cloud server at the millisecond level. The synchronized content includes the coordinates of the user's virtual avatar. Scene item status hash value and plot progress timestamps ;

[0101] The cross-device adaptation unit is used to identify the hardware performance parameters of associated smart devices (such as AR glasses or smartwatches worn by the user) and generate a device capability matrix. The rendering tasks are dynamically allocated according to the matrix (e.g., AR glasses are responsible for near-eye display, and RV holographic devices are responsible for distant scene rendering); the device capability matrix of the cross-device adaptation unit is a 3×3 matrix, and the matrix elements represent the first... Class of equipment for the first The processing efficiency of the class rendering task, with a value between 0 and 1, where, Corresponding to AR glasses, RV host, and cloud server, Corresponds to close-up rendering, distant rendering, and physics engine calculations;

[0102] The dynamic load balancing unit calculates the device load factor using the following formula. ,when At that time, 30% of the rendering tasks will be automatically migrated to the cloud:

[0103] .

[0104] Cloud synchronization unit: 5G-MEC (edge ​​computing) nodes are deployed in the RV communication module, supporting the 3GPP standard ultra-low latency communication protocol (air interface latency ≤20ms), and establishing a connection with the Metaverse cloud server through a dedicated data channel.

[0105] Real-time synchronization of three core data types: user virtual avatar coordinates (3D coordinates, accuracy ±1cm, sampling frequency 60Hz); Scene item state hash value (32-bit MD5 hash value, used to verify item status consistency); Story progress timestamp (Millisecond-level precision, recording the completion time of key plot nodes).

[0106] The synchronization strategy adopts an "incremental synchronization + full verification" mode. Under normal conditions, only changed data (such as virtual image position shift) is transmitted, and a full data verification is performed every 30 seconds to ensure the consistency of local data in the RV and cloud data.

[0107] The cross-device adapter unit automatically identifies associated smart devices (AR glasses, smartwatches, etc.) via Bluetooth / BLE protocol and collects their hardware performance parameters (such as CPU frequency, video memory capacity, and display resolution).

[0108] Generate a 3×3 device capability matrix Matrix elements Indicates the first Class of equipment for the first Processing efficiency of class rendering tasks (values ​​from 0 to 1, higher values ​​indicate greater efficiency): row index Corresponding to AR glasses, RV main unit, and cloud server; column index. This corresponds to close-up rendering (such as character facial details), distant rendering (such as background buildings), and physics engine calculations (such as collision detection).

[0109] Tasks are dynamically assigned based on matrix element values, for example: if (AR glasses have high efficiency in close-up rendering), so the AR glasses are responsible for rendering the details of the character displayed near the eye; if (If the RV host has high efficiency in rendering distant scenes), then the RV holographic equipment will handle the background scene rendering; if (If the cloud server's physics engine has high computational efficiency), then the cloud will undertake complex physics simulation tasks.

[0110] The dynamic load balancing unit monitors three core metrics of the RV's local devices in real time: current CPU utilization (%), memory utilization (%), and network latency (ms).

[0111] Calculate the load factor using the formula: ; ( The value ranges from 0 to 1, with higher values ​​indicating heavier equipment loads.

[0112] Task migration mechanism: When When the load balancing strategy is triggered, 30% of the rendering tasks (prioritizing light interactive tasks such as distant scene rendering and non-real-time physics calculation) are automatically migrated to the cloud server. The migration process is achieved through the "breakpoint resume" mechanism, with a task switching delay of ≤50ms, ensuring that users are unaware of any interruption.

[0113] More specifically, the safety monitoring module includes a physiological safety unit, an equipment redundancy unit, and an emergency response unit;

[0114] The physiological safety unit is used to set the user's physiological safety threshold range (such as heart rate variability). alpha wave frequency in electroencephalogram (EEG) When any parameter in the multi-dimensional interactive data set exceeds the threshold, a security warning signal is generated;

[0115] The redundant unit of the equipment adopts a dual power supply module and a hot standby computing node. The heartbeat detection mechanism (3 status interactions per second) ensures that when the main equipment fails, the standby equipment can take over the control authority within 50ms.

[0116] Upon receiving a safety warning signal, the emergency response unit executes a three-level response strategy: Level 1 response (reducing the intensity of scene stimulation), Level 2 response (pausing the game and displaying physiological soothing prompts), and Level 3 response (activating the RV's emergency lighting and sending help requests to preset contacts).

[0117] The physiological safety unit is based on a multi-dimensional interactive data set, focusing on key physiological indicators and pre-setting safety threshold ranges: heart rate variability. Normal range ≤15% (exceeding this range suggests excessive emotional fluctuation); EEG wave frequency Normal range ≥8Hz (below indicates excessive fatigue); skin conductance response Normal range ≤30μS (exceeding this range indicates overexcitation); angular velocity of limb movements The normal range is ≤10 rad / s (if it exceeds this range, it indicates that the movement amplitude is too large).

[0118] Early warning triggering mechanism: Real-time comparison of collected data with thresholds. When any parameter exceeds the range for 3 consecutive sampling cycles (100ms per cycle), a safety early warning signal (including parameter type, exceeding value, and duration) is automatically generated and sent to the emergency response unit simultaneously.

[0119] The redundant unit of the equipment adopts dual independent power supply modules (main power supply + backup battery), supports seamless switching (switching time ≤10ms), and ensures that the system continues to run for ≥10 minutes when power is lost; deploy main / backup dual computing nodes (configured with the same CPU, memory and interface), the main node runs the core program of the system, and the backup node synchronizes the status of the main node in real time.

[0120] Heartbeat detection and switching mechanism: The primary / backup node performs a heartbeat interaction every 333ms (3 times per second) to transmit a status verification code; if the primary node does not receive a heartbeat signal for 2 consecutive times (determined as a fault), the backup node immediately starts the takeover process and obtains full control within 50ms to ensure uninterrupted service.

[0121] The three-level response rules for emergency response units:

[0122] Level 1 response, receiving a minor warning: reduce the intensity of scene stimulation (reduce the frequency of light and shadow flickering by 50%, and reduce the volume of sound effects by 30%); maintain the game progress and simultaneously push out the text prompt "Take a break".

[0123] Level 2 response, receiving moderate warning: pause the game process, freeze the current scene screen; display physiological soothing prompts (such as breathing guidance animation, relaxation music); the RV seat automatically switches to massage mode (low intensity vibration).

[0124] Level 3 Response: Triggered when receiving a severe warning (such as continuous equipment failure or user physiological parameters exceeding the standard for ≥10 seconds): Forcefully interrupt the game, activate the RV emergency lighting (full cabin white light, brightness ≥300 lux); push help information to preset contacts via cellular network (including location coordinates and warning parameters); activate the RV voice call function (looping the "Need assistance" prompt tone).

[0125] More specifically, the plot deduction unit adjusts the plot branches based on the user immersion index and interaction intent tags, according to the following rules:

[0126] like If the interaction intent tag includes "exploration," a hidden story branch will be triggered, unlocking hidden interaction points within the RV's physical space; if Furthermore, if the interaction intent tag includes "competition," then the intensity of the character's confrontation will be increased, and the vibration frequency of the RV's force feedback device will be increased simultaneously; if This will automatically simplify the storyline quest chain and increase the frequency of NPC guidance.

[0127] Immersion Index Values ​​range from 0 to 1, reflecting the user's level of engagement with the current scene (output by the immersion calculation unit).

[0128] Interaction Intent Labels: Structured labels generated by the Perception Interaction Module (such as “Explore”, “Competitive”, “Leisure”, etc., with a confidence level ≥ 0.85).

[0129] Establish a "by setting a pre-defined rule base" The mapping relationship between "range + intent tag" and plot adjustment strategy, the rule triggering conditions and execution actions are clear and quantifiable.

[0130] Adjustments to plot branching rules and implementation details; highly immersive scene exploration (with the intention of "exploration" included).

[0131] The trigger condition indicates that the user is highly immersed and tends to actively explore (such as frequently focusing their gaze on the corner of the RV and exhibiting physical actions such as touching / observing the environment).

[0132] Story adjustments: Unlock hidden story branches (such as triggering the "RV History Archives" side quest, which plays the background story through holographic projection); activate hidden interactive points in the RV's physical space (such as projecting a keypad on a specific area of ​​the floor, or popping up virtual props from the armrests of the seats); the hint intensity of hidden interactive points dynamically increases with the duration of exploration (initially there is no hint, and if not found within 30 seconds, a faint light and shadow guidance will be displayed).

[0133] Medium-immersion competitive scenarios (with the intent to include "competition"); the triggering condition is that the user's immersion is moderate and they tend to engage in competitive interaction (such as voice commands containing "attack" or "challenge", or movement angular velocity). ).

[0134] Story adjustments: Increased character combat strength (enemy health increased by 20%, attack frequency increased by 30%); Simultaneously increased vibration frequency of force feedback devices (seat vibration increased from 5Hz to 10Hz, controller vibration amplitude increased by 50%); Unlocked competitive exclusive items (such as virtual weapon damage bonuses), and the acquisition path is highlighted via holographic projection.

[0135] Low-immersion guided scenarios are triggered by low user immersion (such as wandering gaze, slow movements, and high heart rate variability). This will trigger regardless of the intent tag.

[0136] Story adjustments: Simplify the story quest chain (remove side quests, reduce the main quest steps from 5 to 3); increase the frequency of NPC guidance (from the default 1 time / 60s to 1 time / 15s, using both voice and animation prompts); reduce quest difficulty (such as shortening the target search range and extending the effective interaction time).

[0137] Rule enforcement and feedback mechanism:

[0138] Real-time performance guarantee: The delay in generating plot adjustment commands is ≤200ms to ensure the continuity between user behavior and plot response; Reversibility design: When the user's state changes (e.g., When necessary, it automatically reverts to the corresponding story branch to avoid interruptions in the experience; Data recording: Synchronously records story adjustment logs (including... Values, intent tags, and adjustment content are used to optimize rule base parameters.

[0139] More specifically, the physiological safety unit is also used to calculate the user's physiological load index using the following formula, and when that occurs, directly triggers a secondary response:

[0140] ;

[0141] In the formula, 20, 50, and 15 are the maximum safe thresholds for heart rate variability, skin conductance response, and limb movement angular velocity, respectively.

[0142] Parameter definition and threshold: The maximum safe threshold for the user's heart rate variability (in %, reflecting the degree of emotional fluctuation) is 20 (i.e., (within a safe range); The skin conductance response (unit: μS, reflecting sympathetic nerve activity) has a maximum safe threshold of 50 (i.e., ...). (within a safe range); The maximum safe threshold for angular velocity of limb movements (unit: rad / s, reflecting the intensity of the movement) is 15 (i.e. (within a safe range);

[0143] Weighting coefficients: Heart rate variability accounts for 50% (the most critical indicator), skin conductance accounts for 30%, and limb movement angular velocity accounts for 20%, reflecting the different contributions of different physiological parameters to load assessment.

[0144] Calculation result range: The value range is 0-1 (when all parameters reach the maximum safety threshold). .

[0145] Level 2 response triggering conditions and execution strategy:

[0146] Trigger threshold: when When the time is right, a secondary response is triggered directly (no other parameters are needed for judgment).

[0147] For example: if (Approaching the maximum threshold of 20) (Approaching the maximum threshold of 50) (Approaching the maximum threshold of 15), then This triggers a level 2 response.

[0148] Level 2 response execution content: immediately pause the current game process and freeze the virtual scene screen; display physiological soothing prompts in the holographic projection area (such as a dynamic guiding animation of "Please take a deep breath for 30 seconds" accompanied by 40dB white noise background sound); automatically switch the RV seat to low-intensity massage mode (vibration frequency 3Hz, lasting for 1 minute); and simultaneously record the physiological parameters at the trigger time as a basis for subsequent user status analysis.

[0149] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A control system for interactive demonstrations in a game RV, characterized in that, include: The system includes a perception and interaction module, an intelligent scene engine, a multimodal output module, a collaborative control module, and a safety monitoring module. The perception and interaction module is used to collect user physiological characteristics, movement trajectories, and environmental parameters in real time, generating a multi-dimensional interaction data set, which includes the user's heart rate variability. Facial micro-expression feature vectors angular velocity of limb movements Interior lighting intensity of the RV Background noise decibel value and the real-time distance between the user and the device ; The intelligent scene engine is used to construct a user immersion model based on a multi-dimensional interaction data set and to calculate a user immersion index using a deep reinforcement learning algorithm. The game scene's story branches, character interaction modes, and environmental rendering parameters are dynamically adjusted based on the immersion index. The multimodal output module is used to convert the adjustment instructions generated by the intelligent scene engine into holographic visual signals, force feedback tactile signals and spatial audio signals, so as to realize the fusion presentation of virtual game elements and the physical space of the RV. The collaborative control module is used to establish a dynamic communication link between the game RV and the cloud metaverse server and other related smart devices, and to realize real-time interaction and state consistency maintenance of cross-device scene data based on the spatiotemporal synchronization protocol. The safety monitoring module is used to monitor the user's physiological load, equipment operating status and environmental safety thresholds in real time. When preset safety conditions are triggered, the scenario degradation or emergency interruption mechanism is automatically activated.

2. The control system for interactive demonstrations in a game RV according to claim 1, characterized in that, The perception and interaction module includes a user state perception unit, an environment perception unit, and an interaction intent parsing unit. The user status sensing unit employs an array-type infrared motion capture device, an 8K eye tracker, and a flexible biosensor to collect data on the three-dimensional coordinate changes of the user's limb joints, the coordinates of the gaze focus point, and skin conductance responses. brainwaves wave frequency ; The environmental sensing unit collects the temperature gradient inside the RV through distributed fiber optic sensors, acoustic array microphones, and a temperature and humidity monitor. Humidity distribution Spectral distribution of illumination and voiceprint characteristics of user voice commands ; The interaction intent parsing unit performs semantic mapping on the multi-dimensional interaction data set based on the Transformer model, extracts explicit user commands and implicit needs, and generates structured interaction intent tags.

3. The control system for interactive demonstrations in a game RV according to claim 2, characterized in that, The intelligent scene engine includes an immersion calculation unit, a scene adaptation unit, and a plot deduction unit. The immersion calculation unit is used to calculate the user immersion index using the following formula. : ； In the formula, These are weighting coefficients, ranging from 0.2 to 0.3, and their sum is 1. This serves as the baseline vector for emotions, with a default value of [0.3, 0.5, 0.2], corresponding to calm, excitement, and focus. The scene adaptation unit is used to adjust the immersion index. Calculate scene adaptation coefficient ,when When high immersion mode is triggered When, the balance mode is triggered; when At that time, the boot mode is triggered.

4. The interactive demonstration control system for a game RV according to claim 3, characterized in that, The multimodal output module includes a holographic projection unit, a force feedback unit, and a spatial audio unit; The holographic projection unit uses a combination of a 4K laser holographic film and a motion tracking projector to project game characters onto the interior wall of the RV, dynamically adjusting the projection scaling ratio using the following formula. : ； In the formula, The preset maximum interaction distance; The force feedback unit is integrated into the RV seat, control handle, and floor contact points, and adjusts according to the virtual collision intensity in the game scene. Output corresponding pressure value ,in, ; The spatial audio unit employs a 64-channel sound array, based on the user's head orientation angle. Calculate the difference in acoustic delay To achieve three-dimensional spatial positioning of virtual sound sources, the delay difference calculation formula is: .

5. A control system for interactive demonstrations in a game RV according to claim 4, characterized in that, The collaborative control module includes a cloud synchronization unit, a cross-device adaptation unit, and a dynamic load balancing unit. The cloud synchronization unit uses a 5G-MEC edge computing node to synchronize the real-time scene data of the game RV with the Metaverse cloud server at the millisecond level. The synchronized content includes the coordinates of the user's virtual avatar. Scene item status hash value and plot progress timestamps ; The cross-device adaptation unit is used to identify the hardware performance parameters of associated smart devices (such as AR glasses or smartwatches worn by the user) and generate a device capability matrix. The rendering tasks are dynamically allocated according to the matrix (e.g., AR glasses are responsible for near-eye display, and RV holographic devices are responsible for distant scene rendering). The dynamic load balancing unit calculates the device load factor using the following formula. ,when At that time, 30% of the rendering tasks will be automatically migrated to the cloud: 。 6. A control system for interactive demonstrations in a game RV according to claim 5, characterized in that, The safety monitoring module includes a physiological safety unit, an equipment redundancy unit, and an emergency response unit. The physiological safety unit is used to set the user's physiological safety threshold range. When any parameter in the multi-dimensional interactive data set exceeds the threshold, a safety warning signal is generated. The equipment redundancy unit adopts a dual power supply module and a hot standby computing node. The heartbeat detection mechanism ensures that when the main equipment fails, the standby equipment can take over control within 50ms. Upon receiving a safety warning signal, the emergency response unit executes a three-level response strategy: Level 1 response, Level 2 response, and Level 3 response.

7. A control system for interactive demonstrations in a game RV according to claim 6, characterized in that, The plot deduction unit adjusts plot branches based on user immersion index and interaction intent tags, using the following rules: like If the interaction intent tag includes "exploration", it will trigger a hidden story branch and unlock hidden interaction points in the RV's physical space; like Furthermore, if the interaction intent tag includes "competition", the intensity of the character's confrontation will be increased, and the vibration frequency of the RV's force feedback device will be increased simultaneously. like This will automatically simplify the storyline quest chain and increase the frequency of NPC guidance.

8. A control system for interactive demonstrations in a game RV according to claim 7, characterized in that, The acoustic delay difference of the spatial audio unit must meet the following constraints: when the user's head orientation angle changes, the delay difference is dynamically compensated by a coefficient to avoid audio positioning deviation.

9. A control system for interactive demonstrations in a game RV according to claim 8, characterized in that, The device capability matrix of the cross-device adaptation unit is a 3×3 matrix, and the matrix elements represent the first... Class of equipment for the first The processing efficiency of the class rendering task, taking a value between 0 and 1, where, Corresponding to AR glasses, RV host, and cloud server, It corresponds to close-up rendering, distant rendering, and physics engine calculation.

10. A control system for interactive demonstrations in a game RV according to claim 9, characterized in that, The physiological safety unit is also used to calculate the user's physiological load index using the following formula, at which point a level two response is directly triggered: ； In the formula, 20, 50, and 15 are the maximum safe thresholds for heart rate variability, skin conductance response, and limb movement angular velocity, respectively.

Images