Three-dimensional live-action sand table bidirectional intelligent interaction system and method

By using logistic regression models and body data monitoring in the three-dimensional real-life sand table system, the VR device parameters are automatically adjusted, which solves the problem of user discomfort in the VR environment and improves the personalization and comfort of the user experience.

CN118897623APending Publication Date: 2024-11-05SHAANXI XINGCHEN TIMES TECH DEV CO LTD

Patent Information

Application Number
CN202410972641.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-19
Publication Date
2024-11-05

AI Technical Summary

Technical Problem

The existing three-dimensional real-life sand table two-way intelligent interactive system has failed to effectively solve the discomforts such as visual fatigue, motion sickness, etc. that users may experience during long-term immersion in VR environments, and lack systematic symptom monitoring and equipment parameter optimization mechanisms, resulting in users Limited experience.

Method used

The user's physical data is collected through VR equipment and body data monitoring devices, the body state characteristic parameters are extracted, and the logistic regression model is used to determine whether the user is in a discomfort state, calculate the correlation, and automatically adjust the VR equipment parameters to relieve discomfort symptoms until the safety threshold is reached. .

Benefits of technology

It has achieved adaptation to individual differences of users, dynamically adjusted the VR environment, significantly improved the user's comfort and interactive experience, reduced discomfort, and improved the personalization and sustainability of the user experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118897623A_ABST
    Figure CN118897623A_ABST
Patent Text Reader

Abstract

The invention discloses a two-way intelligent interaction system and method for a three-dimensional live-action sand table, and relates to the technical field of three-dimensional sand tables, and the technical key points of the scheme are as follows: collecting body data of a user at a preset frequency through VR equipment and a body data monitoring device, extracting body state characteristic parameters of the user from the body data, and determining the body state characteristic parameters of the user; taking the body state characteristic parameters of the user as input variables, training the logistic regression model, deploying the trained model to a central controller, judging whether the user is in a physical discomfort state according to a model prediction result, calculating the relevancy between each body state characteristic parameter and the VR equipment parameters, and determining whether the user is in the physical discomfort state. And if the judgment result is that the user has uncomfortable symptoms, according to the correlation analysis result, sending an adjustment instruction to automatically adjust the parameters of the VR equipment until the uncomfortable symptoms of the user disappear or reach a preset safety threshold value, thereby realizing adaptation to individual differences of the user, and promoting more personalized and accurate interactive experience.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of three-dimensional sand tables, and in particular to a two-way intelligent interactive system and method for a three-dimensional real-scene sand table. Background Art

[0002] With the rapid development of science and technology, the three-dimensional real-life sand table two-way intelligent interactive system has been widely used in many fields as an innovative technical means. The three-dimensional real-life sand table two-way intelligent interactive system is developed on the basis of the traditional sand table model by integrating multimedia display, interactive functions, and advanced computer technology. The system not only retains the intuitiveness and physicality of the traditional sand table model, but also through the combination of sound, light, electricity, images, three-dimensional animation and other technologies, the sand table has a higher information display capability and interactivity.

[0003] In the Chinese invention application with application publication number CN116630584A, a digital sandbox system and method based on VR panorama and AR interaction are disclosed. The embodiment of the present invention constructs a VR panoramic scene of a digital sandbox; receives the visiting requirements of the digital sandbox, plans the VR visiting route; marks multiple key scene locations; generates the first-person scene in real time according to the VR panoramic scene during the digital visit; and interactively guides the user's first-person scene based on AR technology at multiple key scene locations. It is possible to plan the VR visiting route according to the visiting requirements, generate the first-person scene in real time, conduct VR visiting displays, and interactively guide at multiple key scene locations based on AR technology.

[0004] In view of the above invention, the prior art has the following deficiencies:

[0005] 1. Existing technologies often ignore the discomfort that users may experience when immersed in VR environments or interactive operations for a long time, such as visual fatigue and motion sickness. This discomfort is not only due to the limitations of the device itself, but also due to the physiological differences and tolerance of users. Each user has different sensitivities to parameters such as light intensity, color saturation, and movement speed. Most existing systems use unified or limited setting options and fail to provide comfort adjustment solutions based on individual user differences, thus limiting the continuous optimization and personalized customization of user experience;

[0006] 2. There may be potential correlations and mutual influences between the discomfort symptoms experienced by users and the various parameter settings of VR devices. For example, although excessively high resolution or frame rate settings can improve image quality, they may also increase visual burden; inappropriate viewing angle movement speed may induce motion sickness. However, existing technologies have not yet explored this area in depth, and lack a systematic symptom monitoring and device parameter optimization mechanism. As a result, when faced with user feedback, the system often finds it difficult to quickly and accurately identify the root cause of the problem and give effective adjustment suggestions, thereby limiting the efficiency of problem solving and further improvement of user experience. Summary of the invention

[0007] 1. Technical issues to be resolved

[0008] In view of the deficiencies in the prior art, the present invention provides a three-dimensional real-scene sandbox two-way intelligent interactive system and method, which collects user's body data at a preset frequency through VR equipment and a body data monitoring device, extracts user's body state characteristic parameters from the body data, uses the user's body state characteristic parameters as input variables, trains a logistic regression model, deploys the trained model to a central controller, determines whether the user is in a state of physical discomfort based on the model prediction results, calculates the correlation between each body state characteristic parameter and the VR device parameter, and if the judgment result is that the user has discomfort symptoms, sends an adjustment instruction to automatically adjust the VR device parameters based on the correlation analysis results until the user's discomfort symptoms disappear or reach a preset safety threshold, thereby solving the problems mentioned in the background technology.

[0009] (II) Technical solution

[0010] To achieve the above objectives, the present invention is implemented through the following technical solutions: a three-dimensional real-scene sand table two-way intelligent interaction method, comprising the following steps:

[0011] Use a 3D scanner to scan the target scene, obtain three-dimensional data, import the three-dimensional data into the modeling software, build and optimize the model, import the three-dimensional real-scene sandbox model into the VR system, and build and debug the environment;

[0012] The user's body data is collected at a preset frequency through VR equipment and a body data monitoring device, and characteristic parameters of the user's body state are extracted from the body data, including heart rate characteristic parameters, blood pressure characteristic parameters, and eye movement characteristic parameters;

[0013] Initialize the logistic regression model, take the user's physical state characteristic parameters as input variables, train the logistic regression model, deploy the trained model to the central controller, and determine whether the user is in a state of physical discomfort based on the model prediction results;

[0014] Calculate the correlation between each body state characteristic parameter and VR device parameter. If the judgment result is that the user has discomfort symptoms, send adjustment instructions to automatically adjust the VR device parameters according to the correlation analysis results until the user's discomfort symptoms disappear or reach the preset safety threshold.

[0015] Furthermore, the VR device includes a head-mounted display, a sensor array and an eye tracking device, the sensor array includes a gyroscope, an accelerometer and a magnetometer, the body data monitoring device includes a heart rate monitor, a blood pressure monitor and a physiological signal sensor, and the physiological signal sensor includes a skin conductivity sensor and a breathing rate sensor.

[0016] Furthermore, the user's heart rate, blood pressure, and eye movement data are collected at a preset frequency through a body data monitoring device, and transmitted to a central controller via Bluetooth or WiFi to extract characteristic parameters related to the user's body state, including heart rate characteristic parameters, blood pressure characteristic parameters, and eye movement characteristic parameters.

[0017] Furthermore, the heart rate characteristic parameters include the average heart rate and the heart rate standard deviation; the blood pressure characteristic parameters include the average systolic / diastolic pressure and the blood pressure fluctuation range; the eye movement characteristic parameters include the eye movement speed and the abnormal movement frequency, wherein the abnormal movement frequency is obtained by counting the number of abnormal movement patterns.

[0018] Furthermore, the specific process of counting the number of abnormal movement patterns is as follows:

[0019] Define abnormal movement patterns, use eye tracking equipment to collect the user's eye movement data, including eye position, velocity, and acceleration information, extract features related to eye movement, including saccade parameters, gaze parameters, and nystagmus parameters, compare the eye movement features at each time point with the set threshold, determine whether there is an abnormality, and count the number of abnormal movements detected.

[0020] Furthermore, the user's physical state characteristic parameters are used as input variables, the weights and biases of the logistic regression model are initialized as small random numbers, and the loss function is used as the training target. The formula of the loss function is:

[0021]

[0022] Among them, L() represents the loss function, N is the number of samples, represents the abnormal prediction value of the i-th sample, y i represents the actual value, σ() represents the Sigmoid function, x i represents the input variable, ω represents the weight vector, and b represents the bias term.

[0023] Furthermore, gradient descent is used to update weights and biases to minimize the loss function. The gradient calculation formula is:

[0024]

[0025]

[0026] Among them, L represents the loss function.

[0027] Furthermore, the user's body state characteristic parameters are obtained, including heart rate characteristic parameters, blood pressure characteristic parameters, eye movement characteristic parameters, etc., and VR device parameters are obtained, including the display parameters and motion tracking sensitivity of the VR device. The correlation between each body state characteristic parameter and the VR device parameter is calculated. The calculation formula is as follows:

[0028]

[0029] Among them, R represents the correlation, X j , Y j are the observed values ​​of the two sets of data, are the means of two groups of data respectively, j represents the label of the data, j=1,2,…,M, M represents the number of data.

[0030] Furthermore, if the judgment result is that the user has discomfort symptoms, an adjustment instruction is sent according to the correlation analysis result, and the VR device parameters including VR device display parameters and motion tracking sensitivity are automatically adjusted according to the adjustment instruction, the user's body data is continuously monitored, and the VR device parameters are adjusted in real time until the user's discomfort symptoms disappear or reach the preset safety threshold.

[0031] A three-dimensional real-scene sand table two-way intelligent interactive system, comprising:

[0032] The 3D real scene construction module uses a 3D scanner to scan the target scene, obtain 3D data, import the 3D data into the modeling software, build and optimize the model, and import the 3D real scene sandbox model into the VR system to build and debug the environment;

[0033] The data collection and feature extraction module collects the user's body data at a preset frequency through VR equipment and body data monitoring devices, and extracts characteristic parameters related to the user's body state from the body data, including heart rate characteristic parameters, blood pressure characteristic parameters, and eye movement characteristic parameters;

[0034] The central controller initializes the logistic regression model, takes the user's physical state characteristic parameters as input variables, trains the logistic regression model, deploys the trained model to the central controller, and determines whether the user is in a physical discomfort state based on the model prediction results;

[0035] The automatic adjustment module performs correlation analysis by calculating the correlation between each body state characteristic parameter and the VR device parameter. If the judgment result is that the user has discomfort symptoms, the module sends an adjustment instruction to automatically adjust the VR device parameters according to the correlation analysis results until the user's discomfort symptoms disappear or reach the preset safety threshold.

[0036] (III) Beneficial effects

[0037] The present invention provides a three-dimensional real-scene sand table two-way intelligent interactive system and method, which has the following beneficial effects:

[0038] (1) The three-dimensional data obtained by the 3D scanner can capture the shape, size, texture and other details of the target scene with high precision. After the three-dimensional real-scene sandbox model is imported into the VR system, the constructed three-dimensional real-scene sandbox model is visually almost the same as the real world. Users can freely roam, observe and even interact with the model in the virtual environment, which can significantly improve user participation and depth of understanding.

[0039] (2) By real-time monitoring of the user's physiological reactions (such as heart rate, blood pressure) and eye movements, it is possible to more accurately understand the user's emotional state, concentration level and even physical condition, thereby dynamically adjusting the interactive content, difficulty or stimulation intensity in the VR environment, making the experience more personalized and in line with the user's immediate state.

[0040] (3) Based on individual differences and physical conditions of users, the logistic regression model can quickly identify abnormal physical conditions by collecting and analyzing the user's physical condition characteristic parameters in real time. When discomfort symptoms are detected in the user, the early warning mechanism is triggered in time to reduce or eliminate the user's discomfort.

[0041] (4) By monitoring the user's body data in real time and automatically adjusting the parameters of the VR device accordingly, it can quickly respond to and reduce the user's discomfort caused by long-term use or improper parameter settings, such as dizziness and nausea, thereby significantly improving the user's comfort and overall experience during the three-dimensional real-life sandbox interaction process. Each user has different physiological reactions and tolerance. Through continuous monitoring and automatic adjustment, it can adapt to the individual differences of users, thereby promoting a more personalized and accurate interactive experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 A schematic diagram of the steps of the three-dimensional real-scene sandbox two-way intelligent interactive method of the present invention;

[0043] Figure 2 It is a structural schematic diagram of the three-dimensional real-scene sand table two-way intelligent interactive system of the present invention. DETAILED DESCRIPTION

[0044] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] See also Figure 1 The present invention provides a three-dimensional real-scene sand table two-way intelligent interaction method, comprising the following steps:

[0046] Step 1: Use a 3D scanner to scan the target scene, obtain 3D data, import the 3D data into the modeling software, build and optimize the model, import the 3D real-scene sandbox model into the VR system, and build and debug the environment;

[0047] The step 1 includes the following contents:

[0048] Step 101: Use a high-precision 3D scanner or camera to perform all-round scanning or photography of objects or scenes in the real world to obtain accurate three-dimensional data, and pre-process the collected data, including denoising, hole filling, optimization, etc., to improve the accuracy and quality of the three-dimensional model;

[0049] Denoising: Remove noise from the model surface to make the model surface smoother and reduce jaggedness and graininess during rendering. Use the denoising tool or plug-in in the 3D model processing software to adjust the parameters of the denoising algorithm, such as intensity, radius, etc., according to the specific situation and needs of the model to achieve the best denoising effect;

[0050] Hole filling: Repair holes or missing parts on the model surface to ensure the integrity and accuracy of the model. Use the software's hole detection tool or manually check the model to identify the holes that need to be repaired. According to the size and location of the hole, choose the appropriate repair method, such as using the fill tool, copying adjacent faces, or remodeling.

[0051] Optimization: Reduce the number of polygons of the model, optimize textures and materials, improve rendering efficiency, etc. to ensure that the model can run smoothly in the VR environment. Use the software's simplification tools or optimization plug-ins to reduce the number of polygons of the model. While reducing polygons, pay attention to maintaining the shape and details of the model. Check and optimize the texture and material settings of the model to ensure that the texture resolution is appropriate and the material properties are set correctly. Try to use compression technology to reduce the size of the texture file. If necessary, bake the model for lighting and shadows, and pre-calculate and store the lighting and shadow information in the texture to reduce the burden of real-time rendering.

[0052] Step 102: Import the pre-processed data into the 3D modeling software, use polygon modeling, surface modeling or NURBS modeling technology to build a 3D model, and build a complete scene, including terrain, buildings, vegetation and other elements to ensure the authenticity and interactivity of the scene, optimize the details and process the texture mapping of the model to make it more realistic and in line with the actual scene;

[0053] Step 103: In the VR platform software (such as Unity, Unreal Engine, etc.), load the 3D real-life sandbox model into the project through the import function. Depending on the software, you may need to select the appropriate file format (such as FBX, OBJ, etc.) and import settings;

[0054] Step 104: Create a virtual environment for the 3D real-life sandbox model in the VR project, including setting effects such as lighting, shadows, and reflections, and adding necessary interactive elements (such as buttons, triggers, etc.);

[0055] Step 105: Start the VR system and perform detailed debugging and optimization to ensure that the 3D real-scene sandbox model presents realistic visual effects in the VR environment and that all interactive elements can work normally. At the same time, performance needs to be optimized to ensure the smooth operation of the VR system.

[0056] When using, combine the contents of steps 101 to 105:

[0057] The three-dimensional data obtained by the 3D scanner can capture the shape, size, texture and other details of the target scene with high precision. After the three-dimensional real-scene sandbox model is imported into the VR system, the constructed three-dimensional real-scene sandbox model is visually almost the same as the real world. Users can freely roam, observe and even interact with the model in the virtual environment, which can significantly improve user participation and depth of understanding.

[0058] Step 2: Collect the user's body data at a preset frequency through the VR device and the body data monitoring device, and extract the user's body state characteristic parameters from the body data, including heart rate characteristic parameters, blood pressure characteristic parameters, and eye movement characteristic parameters;

[0059] The step 2 includes the following contents:

[0060] Step 201: The user wears the VR device and the body data monitoring device, verifies the user's role and authority, and assigns the user corresponding operation authority and resource access authority according to the user's role and authority. The user can only perform interactive operations and data access within the scope of his authority;

[0061] VR devices include head-mounted displays, sensor arrays, and eye-tracking devices. The head-mounted display has a high-resolution display and supports a low-latency refresh rate to reduce visual delays. The sensor array includes a gyroscope, accelerometer, and magnetometer to accurately track the movement of the user's head and body. The eye-tracking device is integrated into the head-mounted display to track the user's eye movement in real time and analyze the user's visual attention focus.

[0062] The body data monitoring device includes a heart rate monitor, a blood pressure monitor and a physiological signal sensor. The heart rate monitor uses a non-invasive photoelectric sensor to monitor the heart rate through the change of light reflected from the skin; the blood pressure monitor uses a cuff-type or wrist-type blood pressure monitor to automatically measure the user's blood pressure at regular intervals; the physiological signal sensor is optional, such as a skin conductivity sensor for monitoring the user's emotional changes, and a respiratory rate sensor for evaluating the user's relaxation state;

[0063] Step 202: Collect the user's heart rate, blood pressure, eye movement and other data at a preset frequency (e.g., every second) through the body data monitoring device, and transmit it to the central controller via Bluetooth or WiFi;

[0064] Step 203: extracting features related to the user's physical state from the collected physical data, including heart rate feature parameters, blood pressure feature parameters, and eye movement feature parameters; wherein the heart rate feature parameters include average heart rate and heart rate standard deviation, the blood pressure feature parameters include average systolic / diastolic pressure, blood pressure fluctuation range (difference between maximum and minimum values), and the eye movement feature parameters include eye movement speed (number of pixels or angles moved per second) and abnormal movement frequency (number of times abnormal movement patterns are counted);

[0065] The specific process of counting the number of abnormal movement patterns is as follows:

[0066] Step 2031: define abnormal movement patterns, including irregular saccades, prolonged fixations, unstable fixations, and abnormal nystagmus;

[0067] Irregular saccades: saccades that are too fast or too slow, or saccades that change direction suddenly, etc.; excessive fixation time: staying on a single fixation point for too long, beyond the normal range; unstable fixation: small but frequent eye tremors during fixation; abnormal nystagmus: involuntary rapid back-and-forth movement of the eyeballs;

[0068] Step 2032: using an eye tracking device (such as an eye tracker) to collect the user's eye movement data, including information such as the position, velocity, and acceleration of the eye;

[0069] Step 2033: extracting features related to eye movement from the eye movement data, including saccade parameters, fixation parameters, and nystagmus parameters, wherein saccade parameters include duration, amplitude (i.e., the distance between the start and end points of the saccade), speed, etc., fixation parameters include duration of fixation, stability of fixation points (such as standard deviation), etc., and nystagmus parameters include frequency and amplitude of nystagmus, etc.;

[0070] Step 2034: pre-set a threshold for each feature, compare the eye movement feature at each time point with the set threshold, determine whether there is an abnormality, and count the number of abnormal movements detected within a period of time (such as one minute, five minutes, etc.);

[0071] When used, combine the contents of step 201 to step 203:

[0072] By monitoring the user's physiological reactions (such as heart rate, blood pressure) and eye movements in real time, we can more accurately understand the user's emotional state, concentration level and even physical condition, and thus dynamically adjust the interactive content, difficulty or stimulation intensity in the VR environment to make the experience more personalized and in line with the user's immediate state.

[0073] Step 3: Initialize the logistic regression model, take the user's physical state characteristic parameters as input variables, train the logistic regression model, deploy the trained model to the central controller, and determine whether the user is in a state of physical discomfort based on the model prediction results;

[0074] The step three includes the following contents:

[0075] Step 301: Use a large amount of labeled user data, take the user's physical state characteristic parameters as input variables, initialize the weights and biases of the logistic regression model to small random numbers (for example, sampled from the normal distribution N(0,0.01)), and use the loss function as the training target. The formula of the loss function is:

[0076]

[0077] Where N is the number of samples, represents the abnormal prediction value of the i-th sample, that is, the probability of predicting physical discomfort, y i represents the actual value, where σ() is the Sigmoid function, x i is the input variable, i.e., the characteristic parameter of the body state, ω is the weight vector, and b is the bias term;

[0078] It should be noted that the Sigmoid function can compress the input real value to between 0 and 1, and the output value is continuous between 0 and 1, but does not include 0 and 1. It can map any real value to the interval (0,1). The mathematical expression of the Sigmoid function is:

[0079]

[0080] Among them, z is the input of the function, e is the base of the natural logarithm, σ(z) is the output of the function, and its value range is (0,1);

[0081] Step 302: Use gradient descent (or more advanced optimization algorithms such as Adam) to update weights and biases to minimize the loss function. The gradient calculation formula is:

[0082]

[0083]

[0084] Update weights and biases according to the learning rate:

[0085]

[0086] It should be noted that the learning rate (η) is a hyperparameter that is pre-set before training the neural network to control the step size of weight and bias updates. The setting of the learning rate has an important impact on the training effect of the neural network and needs to be selected and adjusted according to the actual situation. In practical applications, the learning rate decay strategy or adaptive learning rate algorithm can be used to further improve the training effect of the model.

[0087] Step 303: Use the test data set to evaluate the model performance (such as accuracy, recall rate, F1 score, etc.), and optimize the model based on the evaluation results. Deploy the trained model to the central controller, receive and process the sensor data in real time, and determine whether the user has discomfort symptoms based on the model prediction results.

[0088] It should be noted that due to individual differences and physical conditions of users, this is mainly because VR technology simulates a three-dimensional environment to make users feel immersive, but this experience poses new challenges to the user's vision, hearing and even body balance system, which may cause nausea, vomiting, dizziness, sweating, pale complexion, etc. Therefore, real-time monitoring of individual differences and physical conditions of users is required;

[0089] When used, combine the contents of step 301 to step 303:

[0090] Based on the individual differences and physical conditions of users, the logistic regression model can quickly identify abnormal physical conditions by collecting and analyzing the user's physical condition characteristic parameters in real time. When discomfort symptoms are detected in the user, the early warning mechanism is triggered in time to reduce or eliminate the user's discomfort.

[0091] Step 4: Perform correlation analysis by calculating the correlation between each body state characteristic parameter and the VR device parameter. If the judgment result is that the user has discomfort symptoms, send adjustment instructions to automatically adjust the VR device parameters according to the correlation analysis results until the user's discomfort symptoms disappear or reach the preset safety threshold.

[0092] The step 4 includes the following contents:

[0093] Step 401: obtaining the user's physical state characteristic parameters, including heart rate characteristic parameters, blood pressure characteristic parameters, eye movement characteristic parameters, etc., and obtaining VR device parameters, including display parameters and motion tracking sensitivity of the VR device;

[0094] Step 402: performing correlation analysis by calculating the correlation between each body state characteristic parameter and the VR device parameter. The correlation calculation formula is as follows:

[0095]

[0096] Among them, R represents the correlation, X j , Y j are the observed values ​​of two sets of data, X and Y are the means of the two sets of data, j represents the label of the data, j = 1, 2, ..., M, M represents the number of data;

[0097] It should be noted that if the R value is close to 1 or -1, it means that there is a strong correlation between the body state characteristic parameter and the VR device parameter. According to the positive or negative value of R, it can be determined whether the VR device parameters need to be adjusted to reduce or increase the corresponding body state characteristic parameter, thereby reducing the user's discomfort;

[0098] Step 403: If the judgment result is that the user has discomfort symptoms, an adjustment instruction is sent according to the correlation analysis result, and the display parameters (such as screen brightness, contrast, color saturation, etc.) and motion tracking sensitivity of the VR device are automatically adjusted according to the instruction to reduce the conflict between the user's vision and motion perception and alleviate the discomfort symptoms;

[0099] Step 404: Continuously monitor the user's physical data and adjust the VR device parameters in real time until the user's discomfort symptoms disappear or reach a preset safety threshold;

[0100] When used, combine the contents of step 401 to step 404:

[0101] By monitoring the user's physical data in real time and automatically adjusting the parameters of the VR device accordingly, it can quickly respond to and reduce the user's discomfort caused by long-term use or improper parameter settings, such as dizziness and nausea, thereby significantly improving the user's comfort and overall experience during the three-dimensional real-life sandbox interaction process. Each user has different physiological reactions and tolerance. Through continuous monitoring and automatic adjustment, it is possible to adapt to individual differences in users, thereby promoting a more personalized and accurate interactive experience.

[0102] See also Figure 2 The present invention also provides a three-dimensional real-scene sandbox two-way intelligent interactive system, including: a three-dimensional real-scene modeling module, a data acquisition and feature extraction module, a central controller and an automatic adjustment module; wherein,

[0103] The 3D real scene construction module uses a 3D scanner to scan the target scene, obtain 3D data, import the 3D data into the modeling software, build and optimize the model, and import the 3D real scene sandbox model into the VR system to build and debug the environment;

[0104] The data collection and feature extraction module collects the user's body data at a preset frequency through VR equipment and body data monitoring devices, and extracts characteristic parameters related to the user's body state from the body data, including heart rate characteristic parameters, blood pressure characteristic parameters, and eye movement characteristic parameters;

[0105] The central controller initializes the logistic regression model, takes the user's physical state characteristic parameters as input variables, trains the logistic regression model, deploys the trained model to the central controller, and determines whether the user is in a physical discomfort state based on the model prediction results;

[0106] The automatic adjustment module performs correlation analysis by calculating the correlation between each body state characteristic parameter and the VR device parameter. If the judgment result is that the user has discomfort symptoms, the module sends an adjustment instruction to automatically adjust the VR device parameters according to the correlation analysis results until the user's discomfort symptoms disappear or reach the preset safety threshold.

[0107] In the application, the several formulas involved are all calculated by removing dimensions and taking their numerical values. The formula is a formula of the most recent real situation obtained by collecting a large amount of data and performing software simulation. The coefficients in the formula are set by technicians in this field according to actual conditions.

[0108] The above embodiments may be implemented in whole or in part by software, hardware, firmware or any other combination thereof. When implemented by software, the above embodiments may be implemented in whole or in part in the form of a computer program product. A person of ordinary skill in the art may appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein may be implemented in combination with electronic hardware, computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution.

[0109] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, and may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0110] The above description is only a specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. A two-way intelligent interactive method for a three-dimensional real-scene sand table. Features: The following steps are involved: Use a 3D scanner to scan the target scene, obtain three-dimensional data, import the three-dimensional data into the modeling software, build and optimize the model, import the three-dimensional real-scene sandbox model into the VR system, and build and debug the environment; The user's body data is collected at a preset frequency through VR equipment and a body data monitoring device, and characteristic parameters of the user's body state are extracted from the body data, including heart rate characteristic parameters, blood pressure characteristic parameters, and eye movement characteristic parameters; Initialize the logistic regression model, take the user's physical state characteristic parameters as input variables, train the logistic regression model, deploy the trained model to the central controller, and determine whether the user is in a state of physical discomfort based on the model prediction results; Calculate the correlation between each body state characteristic parameter and VR device parameter. If the judgment result is that the user has discomfort symptoms, send adjustment instructions to automatically adjust the VR device parameters according to the correlation analysis results until the user's discomfort symptoms disappear or reach the preset safety threshold.

2. A three-dimensional real-scene sandbox two-way intelligent interactive method according to claim 1, Features: The VR device includes a head-mounted display, a sensor array and an eye tracking device. The sensor array includes a gyroscope, an accelerometer and a magnetometer. The body data monitoring device includes a heart rate monitor, a blood pressure monitor and a physiological signal sensor. The physiological signal sensor includes a skin conductivity sensor and a breathing rate sensor.

3. A three-dimensional real-scene sandbox two-way intelligent interactive method according to claim 1, Features: Through the body data monitoring device, the user's heart rate, blood pressure, and eye movement data are collected at a preset frequency and transmitted to the central controller via Bluetooth or WiFi to extract characteristic parameters related to the user's body state, including heart rate characteristic parameters, blood pressure characteristic parameters, and eye movement characteristic parameters.

4. A three-dimensional real-scene sandbox two-way intelligent interactive method according to claim 3, Features: Heart rate characteristic parameters include average heart rate and heart rate standard deviation; blood pressure characteristic parameters include average systolic / diastolic pressure and blood pressure fluctuation range; eye movement characteristic parameters include eye movement speed and abnormal movement frequency, among which the abnormal movement frequency is obtained by counting the number of abnormal movement patterns.

5. A three-dimensional real-scene sand table two-way intelligent interactive method according to claim 4, Features: The specific process of counting the number of abnormal movement patterns is as follows: Define abnormal movement patterns, use eye tracking equipment to collect the user's eye movement data, including eye position, velocity, and acceleration information, extract features related to eye movement, including saccade parameters, gaze parameters, and nystagmus parameters, compare the eye movement features at each time point with the set threshold, determine whether there is an abnormality, and count the number of abnormal movements detected.

6. A two-way intelligent interactive method for a three-dimensional real-scene sand table according to claim 1, Features: The user's physical state characteristic parameters are used as input variables, the weights and biases of the logistic regression model are initialized as small random numbers, and the loss function is used as the training target. The formula of the loss function is: Among them, L() represents the loss function, N is the number of samples, represents the abnormal prediction value of the i-th sample, y i represents the actual value, σ() represents the Sigmoid function, x i represents the input variable, ω represents the weight vector, and b represents the bias term.

7. A three-dimensional real-scene sand table two-way intelligent interactive method according to claim 6, Features: Use gradient descent to update weights and biases to minimize the loss function. The gradient calculation formula is: Among them, L represents the loss function.

8. A three-dimensional real-scene sand table two-way intelligent interactive method according to claim 1, Features: Obtain the user's body state characteristic parameters, including heart rate characteristic parameters, blood pressure characteristic parameters, eye movement characteristic parameters, etc., obtain VR device parameters, including the display parameters and motion tracking sensitivity of the VR device, and calculate the correlation between each body state characteristic parameter and VR device parameter. The calculation formula is as follows: Among them, R represents the correlation, X j , Y j are the observed values ​​of the two sets of data, are the means of two groups of data respectively, j represents the label of the data, j=1,2,…,M, M represents the number of data.

9. A three-dimensional real-scene sand table two-way intelligent interactive method according to claim 8, Features: If the judgment result is that the user has discomfort symptoms, an adjustment instruction is sent according to the correlation analysis results, and the VR device parameters including VR device display parameters and motion tracking sensitivity are automatically adjusted according to the adjustment instructions. The user's body data is continuously monitored, and the VR device parameters are adjusted in real time until the user's discomfort symptoms disappear or the preset safety threshold is reached.

10. A three-dimensional real-scene sandbox two-way intelligent interactive system, used to implement the method described in any one of claims 1 to 9, Features: include: The 3D real scene construction module uses a 3D scanner to scan the target scene, obtain 3D data, import the 3D data into the modeling software, build and optimize the model, and import the 3D real scene sandbox model into the VR system to build and debug the environment; The data collection and feature extraction module collects the user's body data at a preset frequency through VR equipment and body data monitoring devices, and extracts characteristic parameters related to the user's body state from the body data, including heart rate characteristic parameters, blood pressure characteristic parameters, and eye movement characteristic parameters; The central controller initializes the logistic regression model, takes the user's physical state characteristic parameters as input variables, trains the logistic regression model, deploys the trained model to the central controller, and determines whether the user is in a physical discomfort state based on the model prediction results; The automatic adjustment module performs correlation analysis by calculating the correlation between each body state characteristic parameter and the VR device parameter. If the judgment result is that the user has discomfort symptoms, the module sends an adjustment instruction to automatically adjust the VR device parameters according to the correlation analysis results until the user's discomfort symptoms disappear or reach the preset safety threshold.

Citation Information

Patent Citations

  • Interactive teaching method for intelligent education

    CN114999237A

  • Critical patient early activity safety training system based on virtual reality technology

    CN115547451A

  • Method and system for constructing VR virtual world based on real world, VR glasses and storage medium

    CN115641408A

Cited By

  • Abnormity detection method and device for digital psychological sand table

    CN119622612A

  • Cross-domain education interaction behavior analysis method and system based on semantic alignment and tendency matching

    CN122133679A