An interactive method and system based on CAVE immersive space

Abstract

The application discloses an interactive method and system based on a CAVE immersive space, relates to the field of CAVE virtual interaction technology, and comprises the following steps: collecting original data, preprocessing, generating original vectors, performing category label marking, calculating prior probability and conditional probability under each category label, constructing joint distribution probability and conditional probability distribution for weighting, and generating a mixed log likelihood function; based on the mixed log likelihood function, a loss function is generated after conversion, a gradient is calculated, and after descending verification and iteration and updating based on the gradient, an optimal weight vector is generated; the optimal weight vector is fed back, a trained prediction model is obtained and deployed, the original vector is reacquired as input, and a prediction category is output; according to the prediction category, a space state vector is constructed, a rotation gain value is calculated, and adjustment and display are performed; and the application can reduce user dizziness, improve interaction naturalness and immersion.

Description

Technical Field

[0001] This invention relates to the field of CAVE virtual interactive technology, and in particular to an interactive method and system based on CAVE immersive space. Background Technology

[0002] With the development of virtual reality technology, immersive interactive environments have gradually become an important research direction in academia and industry. CAVE (Covered Ambient Air Array) is a typical immersive spatial system that creates a highly immersive virtual environment through multi-faceted stereoscopic projection and a stereoscopic sound system, allowing users to experience spatial perception similar to real-world scenes. Unlike traditional head-mounted displays, CAVE systems allow users to maintain natural visual, auditory, and spatial balance while moving freely, thus finding wide application in virtual laboratories, engineering simulations, military training, and medical visualization. In such systems, interaction methods are crucial; users need to interact with the virtual environment in real time through multimodal inputs such as body position, head direction, and gestures, driving the system to generate corresponding visual feedback and spatial adjustments. Currently, common interaction technologies are mostly based on methods such as camera capture, inertial measurement units, deep learning gesture recognition models, and probabilistic statistical modeling to detect and classify user behavior.

[0003] Existing technologies often only analyze user gestures or positions individually, lacking the ability to jointly model "category labels + spatial states," resulting in unnatural interactive responses. Furthermore, they often employ purely generative models (such as Gaussian mixture models) or purely discriminative models (such as deep classification networks), making it difficult to balance the interpretability of generative models with the accuracy of discriminative models, thus limiting the robustness of interactive systems. Moreover, they rely heavily on traditional gradient descent or sampling inference, leading to reduced real-time performance of CAVE systems. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides an interactive method and system based on CAVE immersive space, which solves the problems of unnatural interactive response, limited robustness of interactive system and reduced real-time performance of CAVE system in the prior art.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0007] In a first aspect, the present invention provides an interactive method based on a CAVE immersive space, comprising,

[0008] The raw data is collected and preprocessed to generate raw vectors. Then, category labels are labeled and the prior probability and conditional probability under each category label are calculated. The joint probability distribution and conditional probability distribution are constructed and weighted to generate a mixed log-likelihood function.

[0009] The transformation is performed based on the mixed log-likelihood function. After generating the loss function, the gradient is calculated. Then, descent verification and iterative updates are performed based on the gradient to generate the optimal weight vector. Feedback is given based on the optimal weight vector to obtain the trained prediction model for deployment. The original vector is then retrieved again as input to output the predicted category.

[0010] Based on the predicted category, a spatial state vector is constructed, and the rotational gain value is calculated, then adjusted and displayed.

[0011] As a preferred embodiment of the interactive method based on CAVE immersive space described in this invention, the following steps are performed: Category labeling is performed, and the prior probability and conditional probability under each category label are calculated. A joint distribution probability and a weighted conditional probability distribution are constructed to generate a mixed log-likelihood function. This involves manually labeling each feature in the original vector with category labels, including movement, turning, waving, and grabbing. The mean and variance formulas are used to obtain the feature mean and variance under each category label. A Gaussian distribution fitting formula is used to obtain the conditional probability of each feature under the category label. The number of features under each category label is counted, and a ratio formula is used to calculate the ratio between the number of features and the total number of features, which serves as the prior probability of each category label. Based on the prior probability and the original vector, a joint distribution model is performed, and the joint distribution probability is calculated.

[0012] Furthermore, based on the original vector and category label, a log-linear method is used to model the conditional probability of the category label and the original vector. After calculating the conditional probability distribution of the category label under a given feature vector, the joint probability distribution is weighted with the conditional probability distribution and defined as a mixture log-likelihood function. The mixture log-likelihood function is then maximized.

[0013] As a preferred embodiment of the interactive method based on CAVE immersive space described in this invention, the following steps are taken: after transforming based on the mixed log-likelihood function to generate a loss function, the gradient is calculated, and then descent verification and iterative-update are performed based on the gradient to generate the optimal weight vector. This refers to constructing a prediction model through the architecture of Naive Bayes and Conditional Random Field, and based on the value of the mixed log-likelihood function, taking the negative value, defining it as the loss function of minimizing the problem as the loss function of the prediction model, and then taking the derivative to obtain the gradient vector.

[0014] The initial step size is set by using the backtracking search method, and the descent condition formula is constructed by combining the Armijo condition with the loss function value and gradient vector.

[0015] When the result in the descent condition formula is less than or equal to, it means that the descent amplitude is sufficient and the current step size is accepted; otherwise, it means that the descent amplitude is insufficient. After setting the reduction factor using the rule of thumb, the step size is reduced, and then the verification is performed again based on the reduced step size.

[0016] After successful verification, the initial acceleration parameters are set, the overall step size is calculated, and the weight vector is iteratively updated in combination with the gradient vector.

[0017] During each iteration-update, the difference between the loss function values ​​between adjacent iterations is calculated using subtraction. The initial acceleration parameters are then corrected based on the difference. After generating the acceleration parameters required for each iteration, the results are fed back to the comprehensive step size formula to calculate the new comprehensive step size for iterative-update of the loss function values.

[0018] After each iteration-update, obtain the gradient vector of the loss function value at the current iteration, and use the Euclidean norm to obtain the length of the current gradient vector;

[0019] Set a convergence threshold. When the length of the current gradient vector is less than or equal to the convergence threshold, it means that the current loss function value is in the optimal state, stop the iteration, and output the optimal weight vector; otherwise, continue the iteration.

[0020] As a preferred embodiment of the interactive method based on CAVE immersive space described in this invention, the following steps are taken: the optimal weight vector is fed back to obtain a trained prediction model for deployment, and the original vector is re-acquired as input, and the predicted category value is output. The optimal weight vector is fed back to the conditional probability distribution formula to generate a trained prediction model, which is then deployed to the interactive system of CAVE immersive space. The original vector is then re-acquired as input to the trained prediction model, and the predicted category is output.

[0021] As a preferred embodiment of the interactive method based on CAVE immersive space described in this invention, the step of constructing a spatial state vector and calculating the rotation gain value according to the predicted category refers to constructing a spatial state vector based on the predicted category and the user's three-dimensional spatial position and angular velocity, and then using empirical rules to set adjustment factors for different predicted categories and calculating the adjustment constant of the CAVE immersive space.

[0022] The API interface is used to read the CAVE model of the CAVE immersive space from the CAD file, and the bounding box coordinates of the CAVE model are obtained using the geometry API. The center point coordinates of the CAVE are obtained using the center calculation formula of the bounding box.

[0023] Extract the user's three-dimensional spatial position from the spatial state vector, use Euclidean distance to obtain the distance from the user's three-dimensional spatial position to the coordinates of the CAVE center point, and then calculate the spatial attenuation factor of the user's three-dimensional spatial position.

[0024] Extract the angular velocity from the spatial state vector, set the rotation sensitivity, and calculate the rotation suppression factor based on the angular velocity;

[0025] Based on the adjustment constant, spatial attenuation factor, and rotational suppression factor, the product is calculated using multiplication operations and defined as the rotational gain value.

[0026] As a preferred embodiment of the interactive method based on CAVE immersive space described in this invention, the step of further adjustment and display refers to using the rotation gain value as the input to the rendering engine in the CAVE immersive space to generate an adjusted rotation speed that is displayed in real time in the CAVE projection.

[0027] As a preferred embodiment of the interactive method based on CAVE immersive space described in this invention, the step of collecting raw data and preprocessing it to generate raw vectors refers to using a multi-camera array and an inertial measurement unit to collect raw data of the user in the CAVE immersive space, including the user's three-dimensional spatial position, angular velocity and acceleration data, and gesture sequence.

[0028] The collected raw data is aligned and normalized;

[0029] Based on angular velocity and acceleration data, an extended Kalman filter algorithm is used for recursive fusion to generate the head orientation Euler angle. A 3DResNet model pre-trained on Kinetics is used, taking the gesture sequence as input and outputting the gesture vector. The user's 3D spatial position, head orientation Euler angle, and gesture vector are concatenated as features to form the original vector.

[0030] Secondly, the present invention provides an interactive system based on a CAVE immersive space, comprising,

[0031] The data acquisition and generation module is used to collect raw data, preprocess it, generate raw vectors, label the data with category labels, calculate the prior probability and conditional probability under each category label, construct the joint probability distribution and conditional probability distribution, weight them, and generate a mixed log-likelihood function.

[0032] The optimized output module is used to transform the mixed log-likelihood function, generate the loss function, calculate the gradient, perform descent verification and iterative update based on the gradient, generate the optimal weight vector, provide feedback based on the optimal weight vector, obtain the trained prediction model for deployment, and re-acquire the original vector as input to output the predicted category.

[0033] The construction and display module is used to construct a spatial state vector based on the predicted category, calculate the rotation gain value, and then adjust and display it.

[0034] Thirdly, the present invention provides a computer device including a memory and a processor, the memory storing a computer program, wherein: when the computer program is executed by the processor, it implements any step of the interactive method based on CAVE immersive space as described in the first aspect of the present invention.

[0035] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, it implements any step of the interactive method based on CAVE immersive space as described in the first aspect of the present invention.

[0036] The beneficial effects of this invention are as follows: This invention uses a hybrid log-likelihood function to weight the joint probability distribution and the conditional probability distribution, and maximizes the weighted result for training the prediction model. This effectively solves the problem of separation between action recognition and spatial state in the prior art, and realizes unified modeling of user action categories and multimodal information such as position and angular velocity. Furthermore, by using gradient descent and acceleration parameter correction, the convergence efficiency of the prediction model is improved, avoiding the slow iteration of traditional optimization methods, and meeting the high real-time requirements of the CAVE system. Secondly, by constructing a spatial state vector and combining it with adjusting constants, spatial decay factors, and rotation suppression factors, dynamic rotation gain is calculated and input into the rendering engine, thereby reducing user dizziness and improving the naturalness and immersion of interaction. Attached Figure Description

[0037] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1 This is a flowchart of the interactive method based on CAVE immersive space in Example 1.

[0039] Figure 2 This is a structural diagram of the interactive system based on the CAVE immersive space in Example 1.

[0040] Figure 3 This is a flowchart of generating the trained prediction model in Example 1. Detailed Implementation

[0041] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0042] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0043] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0044] Example 1, referring to Figures 1-3 This is the first embodiment of the present invention, which provides an interactive method based on a CAVE immersive space, including the following steps:

[0045] S1. Collect raw data and preprocess it to generate raw vectors. Then, label the data with category labels and calculate the prior probability and conditional probability under each category label. Construct the joint probability distribution and conditional probability distribution, weight them, and generate a mixed log-likelihood function.

[0046] Specifically, collecting raw data and preprocessing it to generate raw vectors refers to using a multi-camera array and inertial measurement unit to collect raw data of the user in the CAVE immersive space, including the user's three-dimensional spatial position, angular velocity and acceleration data, and gesture sequence.

[0047] The collected raw data is aligned and normalized;

[0048] Based on angular velocity and acceleration data, an extended Kalman filter algorithm is used for recursive fusion to generate the head orientation Euler angle. A 3DResNet model pre-trained on Kinetics is used, taking the gesture sequence as input and outputting the gesture vector. The user's 3D spatial position, head orientation Euler angle, and gesture vector are concatenated as features to form the original vector.

[0049] By jointly acquiring data from multiple cameras and an inertial measurement unit (IMU), the multimodal integrity of user data was ensured. Alignment and normalization operations enabled data from different sources to be processed under a unified standard, improving modeling accuracy. Furthermore, the use of extended Kalman filtering reduced pose estimation noise and ensured the smooth continuity of head orientation. Secondly, gesture motion vectors were extracted using 3DResNet, giving gesture recognition strong robustness and generalization ability. Finally, feature concatenation constructed a comprehensive raw vector containing spatial location, head orientation, and motion features, thus providing a solid data foundation for subsequent processing.

[0050] Further, category labels are labeled, and the prior and conditional probabilities under each category label are calculated. The joint probability distribution and the conditional probability distribution are weighted to generate a mixed log-likelihood function. Based on the original vector, the category labels of each feature in the original vector are manually labeled, including movement, turning, waving, and grabbing. The mean and variance of the features under each category label are obtained using the mean and variance formulas. The conditional probability of each feature under the category label is obtained by combining the Gaussian distribution fitting formula. The number of features under each category label is counted, and the ratio between the number of features and the total number of features is calculated using the ratio formula as the prior probability of each category label. Based on the prior probability and the original vector, joint distribution modeling is performed, and the joint probability distribution (i.e., the probability that the original vector and the category label appear together) is calculated.

[0051] The conditional probability of each feature under the category label is obtained by combining the Gaussian distribution fitting formula, which is:

[0052]

[0053] In the formula, Indicates the category label Under the conditions The conditional probability of each feature occurring. Indicates category label Next The variance of each feature, Represents an exponential function. Indicates the first One characteristic, Indicates category label Next The mean of each feature;

[0054] The formula for calculating the joint probability distribution is as follows:

[0055]

[0056] In the formula, Represents the original vector With category labels The joint probability distribution, Indicates category label The prior probability, Indicates the total number of features;

[0057] Further, based on the original vector and category label, the log-linear method is used to model the conditional probability of the category label and the original vector. After calculating the conditional probability distribution of the category label under a given feature vector, the joint probability distribution is weighted with the conditional probability distribution and defined as a mixture log-likelihood function. The mixture log-likelihood function is maximized.

[0058] The formula for calculating the conditional probability distribution of the category label under a given feature vector is as follows:

[0059]

[0060] In the formula, Indicates category label Given a feature vector The conditional probability distribution under the following conditions Indicates an index for a specific category. Represents the set of all possible category labels. Represents the weight vector. This indicates the transpose operation. Represents the characteristic function;

[0061] The characteristic function is defined as follows:

[0062]

[0063] In the formula, Indicates the indicator function (i.e., when) When the indicator function is set to 1, it is set to 0 otherwise. hour, ,when hour, );

[0064] The weighting of the joint probability distribution and the conditional probability distribution is given by the following formula:

[0065]

[0066] In the formula, This represents the mixture log-likelihood function value of the weight vector. The weighting coefficients can be represented by cross-validation. It represents the natural logarithm.

[0067] The original vectors are manually labeled with category labels, and the distribution characteristics of each category feature are calculated by combining the mean and variance. This provides an interpretable statistical basis for the subsequent prediction model, avoiding the drawbacks of black-box models that cannot explain interactive behavior. It also ensures that this invention can establish statistically based distinctions between different interactive actions (such as movement and turning). Modeling the conditional probability using a Gaussian distribution fitting formula smooths the impact of noise and outliers on probability estimation, improving the stability and generalization ability of the prediction. This not only avoids the overfitting risk of simple frequency statistics but also allows the invention to maintain a high recognition rate in real-world interactive scenarios. The prior probability of each category label is obtained through a proportional formula and combined with the conditional probability to obtain the joint distribution probability. This step provides global statistical consistency, enabling the simulation of the complete generation mechanism of interactive data, facilitating subsequent weighted integration. Secondly, the conditional probability distribution is calculated using a log-linear method, and the joint distribution and conditional distribution are weighted and combined by defining a mixed log-likelihood function. This step overcomes the limitation of traditional methods that require either generative or discriminative methods, achieving the complementary advantages of both. It significantly improves the accuracy of user action prediction and provides a more natural feedback mechanism for CAVE immersive interaction. By maximizing the mixture log-likelihood function, the optimal weight vector can be obtained, thereby training a prediction model that combines interpretability and accuracy.

[0068] S2. Based on the mixed log-likelihood function, the transformation is performed to generate the loss function and then the gradient is calculated. After descent verification and iterative update based on the gradient, the optimal weight vector is generated. Feedback is given based on the optimal weight vector to obtain the trained prediction model for deployment. The original vector is then re-acquired as input to output the predicted category.

[0069] Specifically, the loss function is generated by transforming the mixture log-likelihood function, and then the gradient is calculated. After descent verification and iterative update based on the gradient, the optimal weight vector is generated. The prediction model is constructed using the architecture of Naive Bayes and Conditional Random Field. The negative value of the mixture log-likelihood function is taken as the loss function of the minimization problem and used as the loss function of the prediction model. The derivative is then calculated to obtain the gradient vector.

[0070] The initial step size is set using the backtracking search method (e.g.) And using the Armijo condition combined with the loss function value and gradient vector, a descent condition formula is constructed, which is:

[0071]

[0072] In the formula, Represents the loss function. Indicates the first The weight vector at the next iteration Indicates the initial step size. Indicates the first The gradient vector at the next iteration Represents constant factors (i.e.) ), Denotes the Euclidean norm;

[0073] When the result in the descent condition formula is less than or equal to, it means that the descent amplitude is sufficient and the current step size is accepted; otherwise, it means that the descent amplitude is insufficient. After setting the reduction factor using the rule of thumb, the step size is reduced, and then the verification is performed again based on the reduced step size.

[0074] The formula for reducing the step size is:

[0075]

[0076] In the formula, Indicates the first Step size in the next iteration This represents the reduction factor (which can be set through experiments and relevant empirical knowledge, or by taking an empirical value of 0.5).

[0077] After successful verification, initial acceleration parameters (e.g., based on relevant experience and experiments) are set. After that, calculate the overall step size, and perform iterative updates of the weight vector in conjunction with the gradient vector;

[0078] The formula for calculating the comprehensive step size is as follows:

[0079]

[0080] In the formula, Indicates the first The overall step size at the next iteration Indicates the initial acceleration parameters;

[0081] The iterative update of the weight vector, which is performed in conjunction with the gradient vector, is formulated as follows:

[0082]

[0083] In the formula, Indicates the first The new weight vector in the next iteration Indicates the first The weight vector at the next iteration Indicates the mixing factor (which can be set through experiments and book knowledge);

[0084] During each iteration-update, the difference between the loss function values ​​between adjacent iterations is calculated using subtraction. The initial acceleration parameters are then corrected based on the difference. After generating the acceleration parameters required for each iteration, the results are fed back to the comprehensive step size formula to calculate the new comprehensive step size for iterative-update of the loss function values.

[0085] The initial acceleration parameters are corrected based on the difference, using the following formula:

[0086]

[0087] In the formula, Indicates the first The acceleration parameters required for the next iteration Indicates the first and The difference between the values ​​of the loss function in each iteration;

[0088] After each iteration-update, obtain the gradient vector of the loss function value at the current iteration, and use the Euclidean norm to obtain the length of the current gradient vector;

[0089] Set a convergence threshold based on the required accuracy and common standards in the field of numerical optimization. When the length of the current gradient vector is less than or equal to the convergence threshold, it indicates that the current loss function value is in the optimal state, the iteration stops, and the optimal weight vector is output; otherwise, the iteration continues.

[0090] By using a loss function weighted by joint and conditional distributions, the global statistical advantage of generative methods and the classification accuracy of discriminative methods can be unified in CAVE interactive modeling. This operation provides a robust loss function for the prediction model, ensuring that the prediction model can more accurately identify user actions in CAVE immersive interactive scenarios. The combination of backtracking search and Armijo conditions allows the invention to adaptively adjust the step size, avoiding excessively fast oscillations or slow convergence problems that may occur with fixed step sizes. The Armijo condition ensures that the loss value decreases with each update, thereby improving the stability of the iteration process. Furthermore, when the step size does not meet the descent condition, the reduction factor can quickly converge to a reasonable range, avoiding redundant iterations. By setting the reduction ratio, the optimization process becomes more robust. Secondly, the introduction of acceleration parameters improves the linear convergence speed of traditional gradient descent, giving the optimization process the fast convergence characteristics of quasi-second-order methods. The combined step size ensures stability while dynamically adjusting the iteration amplitude. This step can significantly reduce model training time in the CAVE interactive system, enabling the prediction model to be deployed and used quickly. Furthermore, by balancing historical and current iteration information through a hybrid factor, the parameter update process is ensured to be smooth and less prone to getting trapped in local optima. The introduction of this hybrid factor can improve the generalization ability of the prediction model under different users and scenarios, ensuring the robustness of action recognition. Meanwhile, accelerating parameters based on loss function difference correction avoids underfitting or overfitting problems caused by fixed parameters, making the optimization process adaptive to different stages. This not only enhances stability over long periods of operation but also makes the CAVE interactive experience consistently natural. Moreover, rigorous convergence is determined through Euclidean norm, ensuring that the final output weight vector is an approximate global optimum, thereby guaranteeing the highest reliability of the deployed prediction model and avoiding delays or recognition errors during interaction.

[0091] Furthermore, the optimal weight vector is fed back to obtain the trained prediction model for deployment. The original vector is then retrieved again as input, and the predicted class value is output. The optimal weight vector is fed back to the conditional probability distribution formula to generate the trained prediction model, which is then deployed to the interactive system of the CAVE immersive space. The original vector is then retrieved again as input to the trained prediction model, and the predicted class is output.

[0092] The optimal weight vector is obtained by training the prediction model and then fed back into the conditional probability distribution formula to form the trained prediction model. This avoids the problems of unstable weight parameters and fluctuating prediction accuracy in traditional interactive models, enabling the prediction model to maintain high-precision class discrimination ability in a CAVE environment. The output of the predicted class provides more stable data support for subsequent interaction-driven processes.

[0093] S3. Based on the predicted category, construct the spatial state vector, calculate the rotation gain value, and then adjust and display it.

[0094] Specifically, based on the predicted category, a spatial state vector is constructed, and the rotational gain value is calculated. After constructing the spatial state vector based on the predicted category and the user's 3D spatial position and angular velocity, an adjustment factor is set for different predicted categories using empirical rules. The adjustment constant for the CAVE immersive space is calculated using the following formula:

[0095]

[0096] In the formula, This represents the adjustment constant for the CAVE immersive space. Represents a reference constant (set through experiments and knowledge of related fields). Indicates the first Adjustment factors for each predicted category;

[0097] The API interface is used to read the CAVE model of the CAVE immersive space from the CAD file, and the bounding box coordinates of the CAVE model are obtained using the geometry API. The center point coordinates of the CAVE are obtained using the center calculation formula of the bounding box.

[0098] After extracting the user's 3D spatial position from the spatial state vector, and using Euclidean distance to obtain the distance from the user's 3D spatial position to the CAVE center point coordinates, the spatial attenuation factor of the user's 3D spatial position is calculated. The formula is as follows:

[0099]

[0100] In the formula, The spatial attenuation factor representing the user's three-dimensional spatial location. The base of the natural logarithm. This represents the distance from the user's 3D spatial location to the center point of the CAVE. Represents a constant (which can be set through experiments and knowledge of related fields). This indicates the CAVE boundary threshold (which can be set based on personal experience or by taking 70–80% of the CAVE boundary size).

[0101] The angular velocity is extracted from the spatial state vector. Rotation sensitivity is set using domain knowledge and experimental parameters. Combined with the angular velocity, the rotation suppression factor is calculated using the following formula:

[0102]

[0103] In the formula, Indicates rotational repression factor, Indicates rotational sensitivity. The magnitude representing the angular velocity;

[0104] Based on the adjustment constant, spatial attenuation factor, and rotational suppression factor, the product is calculated using multiplication operations and defined as the rotational gain value.

[0105] By unifying the predicted category, user position, and angular velocity into a single state vector, it's possible to simultaneously capture interaction intent and physical state, avoiding the disconnect between "action classification" and "spatial awareness" in traditional methods. Furthermore, introducing an adjustment constant makes gain calculations for different interaction categories more aligned with human perception characteristics. The introduction of a spatial attenuation factor ensures that gain adjustment remains almost constant in the CAVE center region, while sharply increasing near the boundaries, preventing abrupt experiences for users due to exceeding boundaries. This significantly improves the smoothness and safety of spatial interactions and reduces discomfort caused by spatial constraints. Secondly, the introduction of a rotation suppression factor dynamically adjusts the correspondence between the screen rotation speed and the user's actual rotation speed, reducing over-rendering at high angular velocities. This step effectively alleviates dizziness caused by rapid rotation, enhancing the comfort of the immersive experience.

[0106] Further adjustments and demonstrations are then performed by using the rotation gain value as input to the rendering engine in the CAVE immersive space, generating an adjusted rotation speed that is then displayed in real time in the CAVE projection.

[0107] By introducing a rotation gain value and using it as input to the rendering engine, the rotation speed of the virtual screen is dynamically adjusted in real time, thereby generating visual feedback in the CAVE immersive space that highly matches the user's actions. This step not only ensures the naturalness of the interaction and reduces user dizziness, but also improves the adaptability of the rendering engine.

[0108] This embodiment also provides an interactive system based on a CAVE immersive space, including:

[0109] The data acquisition and generation module is used to collect raw data, preprocess it, generate raw vectors, label the data with category labels, calculate the prior probability and conditional probability under each category label, construct the joint probability distribution and conditional probability distribution, weight them, and generate a mixed log-likelihood function.

[0110] The optimized output module is used to transform the mixed log-likelihood function, generate the loss function, calculate the gradient, perform descent verification and iterative update based on the gradient, generate the optimal weight vector, provide feedback based on the optimal weight vector, obtain the trained prediction model for deployment, and re-acquire the original vector as input to output the predicted category.

[0111] The construction and display module is used to construct a spatial state vector based on the predicted category, calculate the rotation gain value, and then adjust and display it.

[0112] This embodiment also provides a computer device applicable to the interactive method based on CAVE immersive space, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the interactive method based on CAVE immersive space as proposed in the above embodiment.

[0113] The computer device can be a terminal, comprising a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0114] This embodiment also provides a storage medium storing a computer program that, when executed by a processor, implements the interactive method for implementing a CAVE-based immersive space as proposed in the above embodiments. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0115] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. An interactive method based on CAVE immersive space, characterized in that: include, The raw data is collected and preprocessed to generate raw vectors. Then, category labels are labeled and the prior probability and conditional probability under each category label are calculated. The joint probability distribution and conditional probability distribution are constructed and weighted to generate a mixed log-likelihood function. The transformation is performed based on the mixed log-likelihood function. After generating the loss function, the gradient is calculated. Then, descent verification and iterative updates are performed based on the gradient to generate the optimal weight vector. Feedback is given based on the optimal weight vector to obtain the trained prediction model for deployment. The original vector is then retrieved again as input to output the predicted category. Based on the predicted category, construct the spatial state vector, calculate the rotation gain value, and then adjust and display it. The next step involves labeling category tags and calculating the prior and conditional probabilities for each category tag. A joint probability distribution and a weighted conditional probability distribution are then constructed to generate a mixed log-likelihood function. This is based on the original vector, where each feature in the original vector is manually labeled with a category tag, including movement, turning, waving, and grabbing. The mean and variance formulas are used to obtain the mean and variance of each feature under each category tag. A Gaussian distribution fitting formula is used to obtain the conditional probability of each feature under the category tag. The number of features under each category tag is counted, and a ratio formula is used to calculate the ratio between the number of features and the total number of features, which serves as the prior probability for each category tag. Based on the prior probability and the original vector, a joint distribution model is performed, and the joint probability distribution is calculated. Further, based on the original vector and category label, the log-linear method is used to model the conditional probability of the category label and the original vector. After calculating the conditional probability distribution of the category label under a given feature vector, the joint probability distribution is weighted with the conditional probability distribution and defined as a mixture log-likelihood function. The mixture log-likelihood function is maximized. The process involves transforming the mixture log-likelihood function, generating a loss function, calculating the gradient, and then performing descent verification and iterative updates based on the gradient to generate the optimal weight vector. This process involves constructing a prediction model using a Naive Bayes and Conditional Random Field architecture, and using the negative value of the mixture log-likelihood function as the loss function for minimizing the problem. The gradient vector is then obtained by taking the derivative. The initial step size is set using the backtracking search method, and the descent condition formula is constructed by combining the Armijo condition with the loss function value and gradient vector. The formula is as follows: In the formula, Represents the loss function. Indicates the first The weight vector at the next iteration Indicates the initial step size. Indicates the first The gradient vector at the next iteration Indicates a constant factor. Denotes the Euclidean norm; When the result in the descent condition formula is less than or equal to, it means that the descent amplitude is sufficient and the current step size is accepted; otherwise, it means that the descent amplitude is insufficient. After setting the reduction factor using the rule of thumb, the step size is reduced, and then the verification is performed again based on the reduced step size. The formula for reducing the step size is: In the formula, Indicates the first Step size in the next iteration Indicates the reduction factor; After successful verification, the initial acceleration parameters are set, the overall step size is calculated, and the weight vector is iteratively updated in combination with the gradient vector. The formula for calculating the comprehensive step size is as follows: In the formula, Indicates the first The overall step size at the next iteration Indicates the initial acceleration parameters; The iterative update of the weight vector, which is performed in conjunction with the gradient vector, is formulated as follows: In the formula, Indicates the first The new weight vector in the next iteration Indicates the first The weight vector at the next iteration Indicates the mixing factor; During each iteration-update, the difference between the loss function values ​​between adjacent iterations is calculated using subtraction. The initial acceleration parameters are then corrected based on the difference. After generating the acceleration parameters required for each iteration, the results are fed back to the comprehensive step size formula to calculate the new comprehensive step size for iterative-update of the loss function values. After each iteration-update, obtain the gradient vector of the loss function value at the current iteration, and use the Euclidean norm to obtain the length of the current gradient vector; Set a convergence threshold. When the length of the current gradient vector is less than or equal to the convergence threshold, it means that the current loss function value is in the optimal state, stop the iteration, and output the optimal weight vector; otherwise, continue the iteration. The process of constructing a spatial state vector based on the predicted category and calculating the rotational gain value refers to the process of constructing a spatial state vector based on the predicted category and the user's three-dimensional spatial position and angular velocity, and then using empirical rules to set adjustment factors for different predicted categories to calculate the adjustment constant of the CAVE immersive space. The API interface is used to read the CAVE model of the CAVE immersive space from the CAD file, and the bounding box coordinates of the CAVE model are obtained using the geometry API. The center point coordinates of the CAVE are obtained using the center calculation formula of the bounding box. After extracting the user's 3D spatial position from the spatial state vector, and using Euclidean distance to obtain the distance from the user's 3D spatial position to the CAVE center point coordinates, the spatial attenuation factor of the user's 3D spatial position is calculated. The formula is as follows: In the formula, The spatial attenuation factor representing the user's three-dimensional spatial location. The base of the natural logarithm. This represents the distance from the user's 3D spatial location to the center point of the CAVE. Represents a constant. Indicates the CAVE boundary threshold; Extract the angular velocity from the spatial state vector, set the rotation sensitivity, and calculate the rotation suppression factor based on the angular velocity. The formula is as follows: In the formula, Indicates rotational repression factor, Indicates rotational sensitivity. The magnitude representing the angular velocity; Based on the adjustment constant, spatial attenuation factor, and rotational suppression factor, the product is calculated using multiplication operations and defined as the rotational gain value.

2. The interactive method based on CAVE immersive space as described in claim 1, characterized in that: The optimal weight vector is fed back to obtain a trained prediction model for deployment. The original vector is then retrieved again as input, and the predicted category value is output. The optimal weight vector is fed back to the conditional probability distribution formula to generate a trained prediction model, which is then deployed to the interactive system of the CAVE immersive space. The original vector is then retrieved again as input to the trained prediction model, and the predicted category is output.

3. The interactive method based on CAVE immersive space as described in claim 2, characterized in that: The further adjustment and display refers to using the rotation gain value as input to the rendering engine in the CAVE immersive space to generate an adjusted rotation speed that is displayed in real time in the CAVE projection.

4. The interactive method based on CAVE immersive space as described in claim 3, characterized in that: The process of preprocessing the collected raw data to generate raw vectors refers to using a multi-camera array and inertial measurement unit to collect raw data of the user in the CAVE immersive space, including the user's three-dimensional spatial position, angular velocity and acceleration data, and gesture sequence. The collected raw data is aligned and normalized; Based on angular velocity and acceleration data, an extended Kalman filter algorithm is used for recursive fusion to generate the head orientation Euler angle. A 3DResNet model pre-trained on Kinetics is used, taking the gesture sequence as input and outputting the gesture vector. The user's 3D spatial position, head orientation Euler angle, and gesture vector are concatenated as features to form the original vector.

5. An interactive system based on CAVE immersive space, based on the interactive method based on CAVE immersive space according to any one of claims 1 to 4, characterized in that: include, The data acquisition and generation module is used to collect raw data, preprocess it, generate raw vectors, label the data with category labels, calculate the prior probability and conditional probability under each category label, construct the joint probability distribution and conditional probability distribution, weight them, and generate a mixed log-likelihood function. The optimized output module is used to transform the mixed log-likelihood function, generate the loss function, calculate the gradient, perform descent verification and iterative update based on the gradient, generate the optimal weight vector, provide feedback based on the optimal weight vector, obtain the trained prediction model for deployment, and re-acquire the original vector as input to output the predicted category. The construction and display module is used to construct a spatial state vector based on the predicted category, calculate the rotation gain value, and then adjust and display it.

6. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, it implements the steps of the interactive method based on CAVE immersive space as described in any one of claims 1 to 4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the interactive method based on CAVE immersive space as described in any one of claims 1 to 4.

Images