Virtual reality dynamic field angle adjusting system and adjusting method

Abstract

The invention discloses a virtual reality dynamic field angle adjusting system and method, and relates to the technical field of virtual reality, and the system comprises a multi-source signal collection module which is used for collecting eye movement tracking data and head movement data of a user in a virtual reality environment in real time, and the head movement data comprises head angular velocity and angular acceleration information; the nystagmus signal extraction module is connected with the multi-source signal acquisition module; the vestibular mismatch index calculation engine is connected with the nystagmus signal extraction module and is used for calculating the nystagmus strength index; the self-adaptive field angle controller is connected with the vestibular mismatch index calculation engine; and the rendering pipeline adaptation module is connected with the adaptive field angle controller. By mining neglected nystagmus physiological signals in eye movement tracking data and combining multi-dimensional information fusion and personalized parameter adaptation, dynamic adjustment of a field angle is realized, and the problem of dizziness and individual difference adaptation caused by a fixed field angle in an existing virtual reality system are solved.

Description

Technical Field

[0001] This application relates to the field of virtual reality technology, specifically to a virtual reality dynamic field of view adjustment system and adjustment method. Background Technology

[0002] Virtual reality technology provides users with an immersive visual experience through head-mounted display devices. Field of view (FOV) is one of the key parameters for measuring the immersion of VR devices; it is defined as the angle of view that a user can observe through the display device. Currently, most mainstream VR headsets adopt a fixed field of view design, typically between 90 and 120 degrees.

[0003] However, the fixed field of view design has the following technical problems: First, there's the issue of dizziness caused by a mismatch between visual and vestibular perception. The vestibular system, located in the inner ear, is responsible for sensing head movement and spatial position. In a VR environment, when the motion information presented in the visual image doesn't match the actual motion information perceived by the vestibular system, a sensory conflict occurs, triggering symptoms such as dizziness and nausea. This phenomenon is known as "motion sickness" or "VR vertigo." A fixed, large field of view exacerbates this mismatch in fast-moving scenes because peripheral vision is more sensitive to motion.

[0004] Second, it cannot adapt to individual user differences. Different users have significantly different sensitivities to visual stimuli. Some users (such as patients with vestibular dysfunction, elderly users, and first-time VR users) have a low tolerance threshold for visual motion, and a fixed field of view cannot be adjusted according to individual differences.

[0005] Third, there is a lack of real-time physiological monitoring. Existing technologies mainly rely on subjective user feedback or simple motion parameters to determine comfort, which cannot objectively assess the user's vestibular system status in real time, leading to lag or inaccuracy in regulation. Summary of the Invention

[0006] The purpose of this application is to provide a virtual reality dynamic field of view adjustment system and adjustment method to solve the problems mentioned in the background art.

[0007] To achieve the above objectives, this application provides the following technical solution: a virtual reality dynamic field of view adjustment system, comprising: A multi-source signal acquisition module is used to acquire eye-tracking data and head motion data of the user in the virtual reality environment in real time. The eye-tracking data includes the gaze angle of both eyes and the corresponding timestamp, and the head motion data includes head angular velocity and angular acceleration information. The nystagmus signal extraction module is connected to the multi-source signal acquisition module and is used to extract the fast phase and slow phase features of eye movement from the eye tracking data, identify the vestibular nystagmus pattern, and calculate the nystagmus intensity index based on the slow phase amplitude and nystagmus frequency. The vestibular mismatch index calculation engine is connected to the nystagmus signal extraction module and is used to weight and fuse the nystagmus intensity index, the cumulative motion load index calculated based on head movement, the scene complexity index calculated based on scene optical flow, and the historical vertigo cumulative index calculated based on historical data to obtain the vestibular mismatch index. An adaptive field of view controller, connected to the vestibular mismatch index calculation engine, is used to calculate the target field of view value through nonlinear mapping based on the vestibular mismatch index and user-personalized parameters, and generate a field of view adjustment command through smooth interpolation; The rendering pipeline adaptation module is connected to the adaptive field of view controller and is used to adjust the view frustum parameters of the virtual camera according to the field of view adjustment command, and generate an edge gradient mask when the field of view shrinks.

[0008] Preferably, the nystagmus signal extraction module includes: The eye velocity calculation unit is used to perform time differentiation on eye tracking data to obtain an eye angular velocity sequence; The fast-phase and slow-phase separation unit is used to separate eye movements into fast-phase segments and slow-phase segments according to a velocity threshold, wherein the velocity threshold ranges from 30 to 50 degrees per second. The nystagmosis mode detection unit is used to analyze the relationship between the slow phase drift direction and the fast phase jump direction within a sliding time window. When the angle between the two is greater than 120 degrees, it is determined that vestibular nystagmosis exists. The nystagmus intensity quantification unit is used to calculate the nystagmus intensity index based on the slow phase amplitude and nystagmus frequency of the nystagmus period.

[0009] Preferably, the vestibular mismatch index calculation engine includes: The cumulative motion load calculation unit is used to perform integral calculations on the head angular velocity with a time decay factor to obtain the cumulative motion load index. The scene complexity calculation unit is used to use the ratio of the average amplitude of scene optical flow to the head angular velocity as a scene complexity index. The historical vertigo accumulation calculation unit is used to perform a weighted summation of the vestibular mismatch index of historical frames with a decay factor to obtain the historical vertigo accumulation index. The weighted fusion calculation unit is used to sum the above four indicators according to the weight coefficients corresponding to the user profile to obtain the normalized vestibular mismatch index.

[0010] Preferably, the adaptive field-of-view controller includes: The user profile management unit is used to store users' personalized parameters, including weight coefficient groups, upper and lower limits of comfort thresholds and historical decay coefficients, and classifies users into high tolerance and fast recovery type, medium tolerance and medium recovery type, low tolerance and slow recovery type or fluctuating adaptation type according to their physiological tolerance characteristics. The field of view mapping unit is used to calculate the target field of view value using a piecewise mapping strategy. When the vestibular mismatch index is lower than the lower limit of the comfort threshold, it outputs the maximum field of view; when it is higher than the upper limit of the comfort threshold, it outputs the minimum field of view; when it is in between, it uses a smooth interpolation function to calculate. The smooth gradient unit is used to limit the rate of change of the field of view to no more than 10 to 20 degrees per second, so as to avoid abrupt changes in the field of view.

[0011] Preferably, the rendering pipeline adaptation module includes: The view frustum reconstruction unit is used to update the projection matrix of the virtual camera according to the view angle adjustment command; The edge masking generation unit is used to generate a ring mask that gradually transitions outward from the edge of the effective field of view to complete occlusion, and a cosine function is used to achieve a smooth transition. The gaze point rendering optimization unit is used to dynamically allocate rendering resources based on the user's gaze point position, allocating the resources saved in the masked area to the gaze center area to improve image quality.

[0012] This application also discloses a method for adjusting the dynamic field of view in virtual reality, including: Step S1: Collect the user's eye-tracking data and head movement data in real time; Step S2: Extract nystagmus features from eye-tracking data, identify vestibular nystagmus by analyzing fast and slow phase motion patterns, and calculate nystagmus intensity index. Step S3: The nystagmus intensity index, cumulative motion load index, scene complexity index, and historical vertigo cumulative index are weighted and fused to obtain the vestibular mismatch index. Step S4: Calculate the target field of view value based on the vestibular mismatch index and user-personalized parameters, and generate a field of view adjustment sequence through smooth interpolation; Step S5: Update the view frustum parameters according to the view angle adjustment sequence, generate an edge gradient mask, and optimize the rendering resource allocation of the foveation point region.

[0013] Preferably, step S2 includes: Preprocess the eye-tracking data to remove blinking interference and high-frequency noise; Calculate the eyeball angular velocity sequence and separate fast-phase and slow-phase segments based on velocity thresholds; Within the sliding time window, the average drift direction of the slow phase and the jump direction of the fast phase are calculated. When the angle between the two is greater than the reverse judgment threshold, the presence of vestibular nystagmus is determined. The nystagmus intensity index is calculated by multiplying the slow phase amplitude of the nystagmus period by the cumulative sum of the nystagmus frequency.

[0014] Preferably, step S3 includes: The cumulative motion load index is obtained by performing an integral calculation on the head angular velocity with time decay, and the decay rate is determined according to the user profile type. The scene complexity index is obtained by dividing the average optical flow amplitude of the scene by the head angular velocity. The historical vertigo cumulative index is obtained by performing a weighted summation with attenuation on the vestibular mismatch index of historical frames. The attenuation coefficient is determined based on the user's recovery time constant. The four indicators are weighted, summed, and normalized according to the weight coefficients corresponding to the user profiles to obtain the vestibular mismatch index, which has a value range of zero to one.

[0015] Preferably, a user calibration step is included before step S1: During the baseline acquisition phase, eye movement baseline data of the user in a resting state are collected for 20 to 40 seconds. During the progressive stimulation phase, the scene was rotated at an increasing speed, and the rotation speed at which the nystagmus intensity first significantly increased was recorded as the sensitivity threshold estimate. During the recovery observation phase, the scene motion was stopped, and the time required for the vestibular mismatch index to recover to the baseline was recorded as the recovery time constant. Users are categorized into corresponding profile types based on sensitivity thresholds and recovery time constants, and a set of personalized parameters is determined.

[0016] Preferably, step S5 includes: Calculate the difference between the target field of view and the current field of view. If the absolute value of the difference exceeds the maximum change in a single frame, adjust it gradually at a limited rate. Update the virtual camera's frustum parameters and projection matrix based on the actual field of view angle values; Generate a ring-shaped gradient mask with an inner boundary corresponding to the effective field of view and an outer boundary corresponding to the maximum field of view, and use a cosine curve to smoothly transition the transparency; Detect the user's gaze point position, and when the gaze point is close to the occlusion area, locally expand the effective field of view to avoid obscuring the gazed content; The rendering resources saved in the masked area are redistributed to the gaze center area, improving the image quality in the core area.

[0017] Compared with the prior art, the beneficial effects of this application are: This application incorporates nystagmus signals into field-of-view control, enabling direct perception of the vestibular system's state. Nystagmus is the direct output of the vestibulo-ocular reflex, reflecting abnormal states of the vestibular system earlier and more accurately than head movement or heart rate data. Field-of-view adjustment based on nystagmus signals allows for preventative intervention before the user subjectively perceives dizziness, providing early protection rather than a delayed response.

[0018] This application establishes user profiles through a calibration process, providing differentiated parameter configurations and adjustment strategies based on the physiological characteristics of different users. High-tolerance users can obtain a wider field of view and a stronger sense of immersion, while low-tolerance users can obtain more adequate protection and better comfort.

[0019] This application employs a smooth, gradual field-of-view adjustment method and edge-gradient masking technology. The rate of change in the field of view is strictly limited within the human eye's perception threshold, making it difficult for the user to perceive any change in the field of view. The edge masking uses a smooth cosine curve to achieve a transparency transition, eliminating noticeable occlusion boundaries. A gaze point protection mechanism ensures that the area the user is looking at remains visible at all times. This makes the field-of-view adjustment process completely transparent to the user, preventing any secondary discomfort. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of the overall process structure of this application; Figure 2 This is a schematic diagram of the full field of view of this application; Figure 3 This is a schematic diagram of the contracted field of view state in this application; Figure 4 This is a schematic diagram of the gaze protection mechanism in this application. Detailed Implementation

[0021] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0022] Please see Figures 1-4 The virtual reality dynamic field-of-view adjustment system provided in this application includes a multi-source signal acquisition module, an nystagmus signal extraction module, a vestibular mismatch index calculation engine, an adaptive field-of-view controller, and a rendering pipeline adaptation module. It can sense the user's physiological state in real time and dynamically adjust the field-of-view parameters of the virtual reality display device accordingly.

[0023] A multi-source signal acquisition module is used to acquire eye-tracking data and head motion data of the user in the virtual reality environment in real time. The eye-tracking data includes the gaze angle of both eyes and the corresponding timestamp, and the head motion data includes head angular velocity and angular acceleration information. The nystagmus signal extraction module is connected to the multi-source signal acquisition module and is used to extract the fast phase and slow phase features of eye movement from the eye tracking data, identify the vestibular nystagmus pattern, and calculate the nystagmus intensity index based on the slow phase amplitude and nystagmus frequency. The vestibular mismatch index calculation engine is connected to the nystagmus signal extraction module and is used to weight and fuse the nystagmus intensity index, the cumulative motion load index calculated based on head movement, the scene complexity index calculated based on scene optical flow, and the historical vertigo cumulative index calculated based on historical data to obtain the vestibular mismatch index. An adaptive field of view controller, connected to the vestibular mismatch index calculation engine, is used to calculate the target field of view value through nonlinear mapping based on the vestibular mismatch index and user-personalized parameters, and generate a field of view adjustment command through smooth interpolation; The rendering pipeline adaptation module is connected to the adaptive field of view controller and is used to adjust the view frustum parameters of the virtual camera according to the field of view adjustment command, and generate an edge gradient mask when the field of view shrinks.

[0024] The multi-source signal acquisition module is the data entry point for the entire system. In a specific embodiment of this application, the multi-source signal acquisition module includes two sub-units: an eye-tracking data acquisition unit and a head motion data acquisition unit.

[0025] The eye-tracking data acquisition unit establishes a data communication interface with the eye-tracking sensors built into the virtual reality headset. The eye-tracking data acquisition unit continuously acquires eye-tracking data at a sampling frequency of no less than 120 times per second by calling the software development interface provided by the device manufacturer.

[0026] Each eye-tracking data sampling point contains the following specific information: each eye-tracking sampling point includes binocular gaze direction angle information and a corresponding timestamp. The timestamp accuracy should be at the millisecond level or higher to ensure the accuracy of subsequent time-related calculations. Some eye-tracking sensors can also provide additional auxiliary information, such as pupil diameter, eyelid opening and closing degree, gaze point confidence, etc., which can be used for data quality assessment and preprocessing.

[0027] The head motion data acquisition unit establishes a data communication interface with the inertial measurement unit (IMU) built into the virtual reality headset. The IMU is a standard feature in all virtual reality headsets and is used to implement head posture tracking. The head motion data acquisition unit acquires raw data output from the IMU at a sampling frequency of at least one thousand times per second. This high sampling rate is crucial for capturing the detailed features of rapid head movements.

[0028] Each head motion data sampling point contains the following specific information: Head motion data includes triaxial angular velocity and angular acceleration information. The unit of angular velocity is usually degrees per second or radians per second, and the unit of angular acceleration is usually degrees per square second or radians per square second.

[0029] The multi-source signal acquisition module also includes a data synchronization unit, which is used to time-align data from different sensors to ensure that eye-tracking data and head motion data can be obtained simultaneously at each moment in subsequent processing.

[0030] The multi-source signal acquisition module also includes a data buffer unit, which is responsible for maintaining a cache of historical data within a sliding time window. The data buffer unit is used to maintain historical data within the sliding time window.

[0031] The nystagmus signal extraction module of this application includes four sub-units: eye velocity calculation unit, fast phase-slow phase separation unit, nystagmus mode detection unit, and nystagmus intensity quantification unit.

[0032] The eye velocity calculation unit is the first processing stage in the nystagmus signal extraction module, responsible for preprocessing and velocity calculation of the raw eye-tracking data. The preprocessing process includes the following steps: The first step is blink detection and rejection. The eye speed calculation unit identifies the time intervals in which blinks occur by monitoring the continuity of eye movement data and eyelid opening and closing information, and marks the data within these intervals as invalid. In subsequent calculations, invalid data intervals will be skipped or filled in using interpolation methods.

[0033] Secondly, high-frequency noise is filtered out. The raw output signal of the eye-tracking sensor inevitably contains measurement noise and quantization noise. The eye speed calculation unit uses a low-pass filter to smooth the raw data and filter out noise components that exceed the upper limit of the physiological eye movement frequency.

[0034] Next is binocular data fusion. The eye velocity calculation unit averages and fuses the fixation angle data of the left and right eyes to obtain fused data representing the overall fixation direction of both eyes.

[0035] After preprocessing, the eye velocity calculation unit performs time differentiation on the eye fixation angle time series to calculate the eye angular velocity at each sampling moment. The eye velocity calculation unit further calculates the magnitude of the angular velocity vector, which is obtained by adding the squares of the horizontal and vertical components and taking the square root to obtain a scalar value representing the magnitude of the eye movement velocity.

[0036] The fast-phase / slow-phase separation unit receives eye angular velocity sequence data and separates eye movements into fast-phase and slow-phase movements based on preset velocity thresholds. Fast-phase movements are characterized by rapid, jumping motions with large angular velocity amplitudes and short durations; slow-phase movements are characterized by small angular velocity amplitudes and long durations, resulting in slow, smooth motions. Fast phases are high-speed jumps, while slow phases are low-speed drifts.

[0037] The fast-phase / slow-phase separation unit sets a velocity classification threshold. The separation method is as follows: when the amplitude of the eyeball angular velocity at a certain moment is greater than the velocity classification threshold, the eyeball motion state at that moment is marked as fast-phase motion; when the amplitude of the eyeball angular velocity at a certain moment is less than or equal to the velocity classification threshold, the eyeball motion state at that moment is marked as slow-phase motion.

[0038] The fast-phase / slow-phase separation unit is also responsible for aggregating consecutive fast-phase state moments into fast-phase motion segments and consecutive slow-phase state moments into slow-phase motion segments. Each motion segment records attribute information such as its start time, end time, duration, start position, and end position.

[0039] The nystagmus pattern detection unit is connected to the fast-phase / slow-phase separation unit and receives sequence data of fast-phase and slow-phase segments. The function of this unit is to analyze the pattern characteristics of fast-phase and slow-phase motions within a preset sliding time window to determine whether vestibular nystagmus is present.

[0040] The length of the sliding time window ranges from 300 milliseconds to 800 milliseconds. A window length that is too short will prevent the complete nystagmus cycle from being captured, while a window length that is too long will increase detection latency. In a preferred embodiment of this application, the length of the sliding time window is set to 500 milliseconds.

[0041] Vestibular nystagmus has a unique movement pattern that can be distinguished from normal eye movement patterns. Normal eye movement patterns are characterized by: relatively stable eye movement during fixation; slow eye movement following a target during smooth tracking; and rapid eye movement jumping to a new fixation point and then stabilizing again during saccades. In normal patterns, there is no fixed opposite relationship between the direction of fast-phase movement (saccades) and the direction of slow-phase movement (tracking).

[0042] The movement pattern of vestibular nystagmus is as follows: the eyeball first undergoes an involuntary slow drift in the direction of movement, which is caused by abnormal vestibular-ocular reflex; then the central nervous system detects that the eyeball has deviated from the desired position and triggers a rapid jump in the opposite direction to pull the eyeball back; then the eyeball begins to drift slowly in the same direction again, and so on, forming a periodic nystagmus pattern.

[0043] The detection process of the nystagmus mode detection unit is as follows: The first step is to extract all slow-phase motion segments within the current sliding time window and calculate the displacement vector for each slow-phase segment. The displacement vector is defined as the eye position at the end of the slow-phase segment minus the eye position at the beginning of the segment. Then, the average displacement vectors of all slow-phase segments within the window are averaged to obtain the average drift direction vector of the slow-phase motion within that window.

[0044] The second step is to extract all fast-phase motion segments within the current sliding time window and calculate the jump direction vector for each fast-phase segment. The jump direction vector is defined as the eye position at the end of the fast-phase segment minus the eye position at the beginning of the fast-phase segment.

[0045] The third step is to calculate the angle between the slow-phase average drift direction vector and the jump direction vectors of each fast phase. The angle can be calculated using the ratio of the vector dot product to the vector magnitude.

[0046] The fourth step is to set a reverse judgment threshold, which typically ranges from 120 to 180 degrees. When there are at least two complete fast-phase-slow-phase alternation cycles within the window, and the average angle between the slow-phase drift direction and the fast-phase jump direction is greater than the reverse judgment threshold, a vestibular nystagmus pattern is determined to exist within that sliding time window.

[0047] The nystagmus detection unit outputs a Boolean result indicating whether vestibular nystagmus exists in the current window. This result is then passed to the nystagmus intensity quantization unit.

[0048] The nystagmus intensity quantification unit is connected to the nystagmus pattern detection unit. When a vestibular nystagmus pattern is detected, a quantitative index of nystagmus intensity is further calculated.

[0049] The nystagmus intensity quantization unit first identifies each complete nystagmus cycle within the current sliding time window. A complete nystagmus cycle consists of a slow-phase motion segment followed by a fast-phase motion segment. For each nystagmus cycle, the nystagmus intensity quantization unit calculates the following two characteristic parameters: Slow-phase amplitude: defined as the total amplitude of the change in eye angle during the slow-phase motion segment of the nystagmus cycle. The larger the slow-phase amplitude, the more severe the eye drift caused by abnormal vestibulo-ocular reflex, reflecting a higher degree of vestibular stimulation.

[0050] Nystagmus frequency: defined as the reciprocal of the duration of the nystagmus cycle, representing the number of nystagmus events per unit time.

[0051] The nystagmus intensity quantification unit multiplies the slow-phase amplitude of each nystagmus cycle by its corresponding nystagmus frequency to obtain the intensity contribution value for that cycle. Then, the intensity contribution values ​​of all nystagmus cycles within the window are summed and divided by the length of the sliding time window to obtain a normalized nystagmus intensity index value. This index considers both the amplitude and frequency characteristics of nystagmus, comprehensively reflecting the vestibular stimulation level at the current moment.

[0052] When the nystagmus detection unit outputs a detection result indicating the absence of vestibular nystagmus, the nystagmus intensity quantification unit sets the nystagmus intensity index to zero or a baseline value close to zero.

[0053] The vestibular mismatch index calculation engine reflects the degree of mismatch in the user's current visual vestibular system, providing a basis for subsequent field-of-view adjustment decisions.

[0054] The vestibular mismatch index calculation engine consists of four sub-units: motion cumulative load calculation unit, scene complexity calculation unit, historical vertigo cumulative calculation unit, and weighted fusion calculation unit.

[0055] The motion cumulative load calculation unit is connected to the multi-source signal acquisition module to receive head angular velocity data. It calculates the cumulative effect of the user's head motion intensity over a period of time.

[0056] The vestibular system exhibits cumulative and time-decay characteristics in response to motor stimuli; therefore, this application incorporates a time-decay mechanism in the calculation process.

[0057] The motion cumulative load calculation unit uses a cumulative calculation method with a time decay factor to calculate the motion cumulative load index. Specifically, this unit maintains a cumulative variable and updates it at each sampling time. The update process includes two steps: First, the cumulative variable is multiplied by a decay factor, the value of which is less than one, so that the contribution of historical motion in the cumulative variable decays exponentially; second, the head angular velocity amplitude at the current time is added to the cumulative variable, representing the new contribution of the current motion to the cumulative load.

[0058] The specific value of the attenuation factor is related to the attenuation rate parameter. A larger attenuation rate parameter results in a smaller attenuation factor, and the faster the impact of historical motion fades; conversely, a smaller attenuation rate parameter results in a larger attenuation factor, and the longer the impact of historical motion lasts. In this application, the attenuation rate parameter is read from a parameter library based on the user profile type. High-tolerance users use a larger attenuation rate parameter because their vestibular system has a stronger recovery ability, and the impact of historical motion fades quickly; low-tolerance users use a smaller attenuation rate parameter because their vestibular system has a weaker recovery ability, and the impact of historical motion lasts for a longer period.

[0059] The scene complexity calculation unit establishes a communication connection with the virtual reality rendering engine to obtain image and motion information of the current rendered frame. This unit is used to calculate the degree of matching between the current visual scene and the user's head movement.

[0060] Visual-vestibular conflict is the root cause of motion sickness in virtual reality. Scene complexity metrics are used to quantify the degree of this visual-vestibular inconsistency.

[0061] The scene complexity calculation unit first calculates the optical flow field between the current rendering frame and the previous rendering frame. The optical flow field can be calculated using classic optical flow estimation algorithms, or it can directly utilize the motion vector buffer data in the rendering engine.

[0062] After calculating the optical flow field, the scene complexity calculation unit calculates the average amplitude of the optical flow vectors of all pixels in the optical flow field to obtain the average optical flow amplitude of the current frame. This average value reflects the overall intensity of motion in the visual image.

[0063] Then, the scene complexity calculation unit obtains the head angular velocity amplitude at the current moment from the multi-source signal acquisition module. The average optical flow amplitude is divided by the head angular velocity amplitude to obtain the scene complexity index.

[0064] The historical vertigo accumulation calculation unit maintains a historical data cache queue, storing the vestibular mismatch index calculation results from the past several frames. The function of this unit is to calculate the ongoing impact of historical vertigo states on the current state.

[0065] The human vestibular system exhibits a delayed and cumulative response to stimuli, leading to the introduction of a historical cumulative term. During recovery, the user's tolerance threshold to new stimuli decreases. Therefore, the historical vertigo cumulative index is used to reflect this delayed effect.

[0066] The historical vertigo accumulation calculation unit reads the vestibular mismatch index stored values ​​from the historical data cache queue for several past frames. The number of historical frames typically ranges from thirty to one hundred and twenty frames, with the specific value determined based on the rendering frame rate and the desired historical backtracking time. For example, when the rendering frame rate is ninety frames per second and a one-second historical backtracking is desired, the number of historical frames is set to ninety frames.

[0067] The historical vertigo accumulation calculation unit performs a weighted summation of the vestibular mismatch index from each historical frame, with a time decay factor. Historical data further removed from the current frame has a smaller weight exponentially, while historical data closer to the current frame has a larger weight. The decay coefficient of the time decay factor is calculated based on the recovery time constant measured during the user calibration phase. A longer recovery time constant indicates a slower recovery of the user's vestibular system, a smaller decay coefficient, and a longer duration of the impact of historical vertigo on the current state.

[0068] The weighted fusion calculation unit is connected to the above three calculation units and simultaneously receives nystagmus intensity indicators from the nystagmus signal extraction module.

[0069] The weighted fusion calculation unit reads the current user's weight coefficient group from the user profile database. The weight coefficient group contains four weight coefficients, corresponding to the nystagmus intensity index, cumulative motion load index, scene complexity index, and historical vertigo cumulative index, respectively. The sum of the four weight coefficients equals one to ensure the scale consistency of the fusion result.

[0070] The weighted fusion calculation unit multiplies the nystagmus intensity index, cumulative motion load index, scene complexity index, and historical vertigo cumulative index by their respective weighting coefficients, and then adds the four products together to obtain the original calculation result of the vestibular mismatch index.

[0071] Because the original scales of various indicators may differ, the weighted fusion calculation unit also needs to normalize the original calculation results. Normalization ensures that the final vestibular mismatch index ranges from 0 to 1. When the calculation result is less than zero, it is truncated to 0; when the calculation result is greater than 1, it is truncated to 1. The closer the normalized vestibular mismatch index value is to 0, the more comfortable the user's current state; the closer the value is to 1, the more the user's current state is approaching or has reached the vertigo threshold.

[0072] After completing the calculation of the current frame, the weighted fusion calculation unit stores the vestibular mismatch index into the historical data cache queue for use in the calculation of historical vertigo cumulative index in subsequent frames.

[0073] The adaptive field-of-view controller is a key module that translates the vestibular mismatch index into specific field-of-view adjustment actions. This controller comprises three sub-units: a user profile management unit, a field-of-view mapping unit, and a smooth transition unit.

[0074] The user profile management unit is used to store and manage users' personalized parameter sets. Each user's parameter set includes the following: The weighting coefficient group includes the weighting coefficients of the nystagmus intensity index, cumulative motion load index, scene complexity index, and historical vertigo cumulative index, which determine the relative importance of each component in the calculation of the vestibular mismatch index.

[0075] Lower limit of comfort threshold: When the vestibular mismatch index is below this threshold, the user is considered to be in a completely comfortable state and can use the maximum field of view to obtain the best sense of immersion.

[0076] Comfort threshold upper limit: When the vestibular mismatch index is higher than this threshold, it is considered that the user is close to or has reached the vertigo threshold, and the minimum field of view needs to be used to provide protection.

[0077] Minimum field of view: The minimum protective field of view allowed by the system, typically set between 50 and 70 degrees. The field of view will not fall below this value to ensure that basic visual functions are not affected.

[0078] Maximum field of view: The maximum immersive field of view allowed by the system, usually set between 100 and 120 degrees, depending on the optical design capabilities of the hardware.

[0079] Historical decay coefficient: determines the rate of time decay in the historical cumulative vertigo index and is related to the user's vestibular system recovery ability.

[0080] The user profile management unit is also responsible for classifying users based on their physiological tolerance characteristics exhibited during the calibration phase. This application defines four user profile types: high tolerance and rapid recovery, moderate tolerance and moderate recovery, low tolerance and slow recovery, and fluctuating adaptation.

[0081] The field-of-view mapping unit is connected to the user profile management unit and the vestibular mismatch index calculation engine. It is used to calculate the target field-of-view value that should be used at the current moment based on the current vestibular mismatch index and the user's personalized parameters.

[0082] The field-of-view mapping unit employs a piecewise nonlinear mapping strategy. The mapping curve is divided into three segments: The first segment, the comfort zone: When the vestibular mismatch index is less than or equal to the lower limit of the comfort threshold, the user is in a completely comfortable state. At this time, the target field of view is set to the maximum field of view value to provide the best immersive experience.

[0083] The second segment, the danger zone: When the vestibular mismatch index is greater than or equal to the upper limit of the comfort threshold, the user is approaching or has reached the dizziness threshold. At this time, the target field of view is set to the minimum field of view value to provide maximum protection.

[0084] The third segment, the transition zone: When the vestibular mismatch index is between the lower and upper limits of the comfort threshold, the user is in a transitional state between comfort and discomfort. At this time, the target field of view is calculated by smooth interpolation based on the relative position of the vestibular mismatch index.

[0085] The interpolation calculation in the transition zone uses a cubic polynomial smooth interpolation function, which has the following characteristics: the function value is equal to 1 and the derivative is equal to 0 at the lower limit of the comfort threshold; the function value is equal to 0 and the derivative is equal to 0 at the upper limit of the comfort threshold. This ensures a smooth transition of the mapping curve at the two threshold boundaries, avoiding any abrupt changes in slope that could cause a jarring adjustment.

[0086] The target field-of-view value output by the field-of-view mapping unit represents the ideal field-of-view value corresponding to the current vestibular mismatch index. However, this value may differ significantly from the actual field-of-view value of the previous frame. Directly applying the target field-of-view value could lead to sudden changes in the field-of-view, causing new visual discomfort. Therefore, a smoothing unit is needed for further processing.

[0087] The smooth transition unit is connected to the field of view mapping unit and receives the target field of view value. The function of this unit is to limit the rate of change of the field of view, ensuring that the change process is smooth and the rate of change does not exceed the perception threshold of the human eye.

[0088] The smooth gradation unit sets an upper threshold for the rate of change of the field of view. When the rate of change of the field of view is below this threshold, the user will hardly perceive the change in the field of view. Therefore, the smooth gradation unit limits the rate of change of the field of view to this range.

[0089] The processing procedure for smooth gradient units is as follows: First, read the actual field of view value at the end of the previous frame from the frame state buffer.

[0090] Then, calculate the difference between the current target field of view value and the actual field of view value of the previous frame.

[0091] Next, the maximum allowable change in field of view per frame is calculated. This value is equal to the perceptual threshold divided by the rendering frame rate. For example, when the perceptual threshold is set to 15 degrees per second and the rendering frame rate is 90 frames per second, the maximum change per frame is approximately 0.17 degrees.

[0092] Finally, compare the absolute value of the target difference with the maximum change in a single frame. If the absolute value of the target difference exceeds the maximum change in a single frame, it indicates that the target field of view differs significantly from the current field of view, making adjustment impossible within one frame. In this case, set the actual field of view of the current frame to the actual field of view of the previous frame plus (or minus) the maximum change in a single frame, achieving a gradual adjustment with speed limits. If the absolute value of the target difference does not exceed the maximum change in a single frame, it indicates that the target field of view differs slightly from the current field of view, allowing adjustment to be completed within one frame. In this case, directly set the actual field of view of the current frame to the target field of view value.

[0093] Through this rate limiting mechanism, even if the vestibular mismatch index changes drastically, the field of view will adjust smoothly and gradually at a speed imperceptible to the user, avoiding secondary discomfort caused by sudden changes in the field of view.

[0094] The rendering pipeline adaptation module is the execution module used to translate view angle adjustment commands into actual rendering effects. This module includes three sub-units: the view frustum reconstruction unit, the edge mask generation unit, and the foveated rendering optimization unit.

[0095] The frustum reconstruction unit is connected to the adaptive field-of-view controller and receives the actual field-of-view value of the current frame. This unit updates the virtual camera's frustum geometry and perspective projection matrix in real time based on the field-of-view value.

[0096] The view frustum is the geometric representation of the visible area of ​​a virtual camera, and it is shaped like a truncated pyramid. The main parameters of the view frustum include the horizontal field of view, the vertical field of view, the near clipping plane distance, and the far clipping plane distance. When the field of view changes, the shape of the view frustum changes accordingly, and the element values ​​of the projection matrix need to be recalculated.

[0097] The frustum reconstruction unit first sets the actual field of view value of the current frame as the horizontal field of view parameter. Then, it calculates the corresponding vertical field of view parameter based on the aspect ratio of the display screen. The aspect ratio is defined as the ratio of the width to the height of the display screen, and the vertical field of view is equal to the horizontal field of view divided by the aspect ratio.

[0098] The frustum reconstruction unit then calculates the values ​​of each element of the perspective projection matrix based on the horizontal field of view, vertical field of view, near clipping plane distance, and far clipping plane distance. The projection matrix is ​​a 4x4 transformation matrix used to transform points in the 3D scene to the 2D screen coordinate system.

[0099] The frustum reconstruction unit submits the calculated projection matrix to the rendering engine's rendering pipeline. The rendering engine uses the updated projection matrix when performing vertex transformations, thus achieving the actual change in the field of view.

[0100] The edge masking generation unit generates a gradient edge masking effect when the field of view shrinks. When the current actual field of view is smaller than the maximum field of view, the edges of the field of view need to be occluded to guide the user's visual attention to the central effective area.

[0101] The edge masking generation unit first calculates the geometric parameters of the mask. The normalized radius of the inner boundary is defined as the radial position of the edge of the current effective field of view in the normalized screen coordinate system, and the normalized radius of the outer boundary is defined as the radial position of the edge of the maximum field of view in the normalized screen coordinate system. The region between the inner and outer boundaries is a gradient transition region.

[0102] The edge mask generation unit generates a mask texture that matches the rendering resolution. For each pixel in the texture, its normalized radial distance to the screen center is calculated. Based on the relationship between the radial distance and the inner and outer boundaries, the transparency value of that pixel is determined. When the radial distance of a pixel is less than the inner boundary radius, the pixel is located within the effective field of view. The transparency is set to 0, which means it is completely transparent and does not obscure the original rendered image.

[0103] When the radial distance of a pixel is greater than the outer boundary radius, the pixel is located outside the maximum field of view. The transparency is set to 1, which means it is completely opaque and completely occludes the original rendered image.

[0104] When a pixel's radial distance lies between the inner and outer boundary radii, the pixel is within the gradient transition region. Opacity is calculated using a cosine-smooth curve based on the relative position of the radial distance. The cosine curve ensures a smooth transition of opacity from completely transparent at the inner boundary to completely opaque at the outer boundary, without noticeable boundary lines or step-like changes.

[0105] The edge mask generation unit passes the generated mask texture to the post-processing stage of the rendering pipeline. After each frame is rendered, the post-processing stage blends the mask texture with the original rendered image, overlaying a gradient dark mask effect on the edge areas. The mask color is typically set to black or dark gray.

[0106] The gaze-point rendering optimization unit connects to the eye-tracking component in the multi-source signal acquisition module to obtain the user's gaze position information for the current frame. The function of this unit is to optimize the allocation strategy of rendering resources based on the gaze position.

[0107] The human eye's visual resolution is highest in the central area of ​​focus and gradually decreases towards the periphery. Leveraging this physiological characteristic, limited rendering resources can be concentrated on the area the user is looking at, reducing the overall computational load while maintaining subjective image quality. When the field of view shrinks, causing edge areas to be covered by occlusion, rendering resources in these areas can be further conserved and reallocated.

[0108] The gaze-point rendering optimization unit first obtains the normalized coordinates of the user's gaze point on the screen in the current frame from the eye-tracking data. The normalized coordinates have the screen center as the origin and range from -1 to +1.

[0109] The foveated rendering optimization unit then performs foveated protection detection. It calculates the normalized radial distance from the foveated point to the screen center and determines whether the foveated point is approaching or entering the transition region of the edge mask. A safety margin parameter is set; when the radial distance of the foveated point is greater than the inner boundary radius minus the safety margin, the foveated point is considered to be approaching the masked region.

[0110] When a gaze point is detected approaching the masked area, the gaze point rendering optimization unit locally expands the inner boundary radius along the radial direction of the gaze point. The expansion amount is proportional to the degree to which the gaze point exceeds the safety margin. This local expansion mechanism ensures that the content area that the user is actively looking at is not obscured by the mask, avoiding the unpleasant experience of lost visual information.

[0111] The foveated rendering optimization unit is also responsible for implementing hierarchical rendering resource allocation strategies.

[0112] When the field of view shrinks significantly, the area of ​​the edge region increases, resulting in a corresponding increase in saved rendering resources. The foveated rendering optimization unit reallocates these saved resources to the core region, further improving the rendering quality of the core region. For example, the supersampling rate of the core region can be increased from two times to four times, or a higher-quality global illumination algorithm can be enabled. This dynamic resource reallocation mechanism ensures that field of view shrinkage not only does not degrade image quality, but may even improve the image quality of the user's gaze area.

[0113] The user calibration process is a parameter measurement procedure performed when a user first uses the system or actively triggers calibration. Through the calibration process, the system can acquire the user's personalized physiological parameters, providing a basis for subsequent dynamic field-of-view adjustment.

[0114] The calibration process consists of five stages: baseline acquisition, progressive stimulation, recovery observation, parameter calculation, and parameter storage and verification.

[0115] The purpose of the baseline acquisition phase is to acquire eye movement baseline data of the user in a completely resting state. This phase lasts from twenty to forty seconds.

[0116] At the start of the baseline acquisition phase, the system presents the user with a simple virtual scene. A static gaze point marker is placed in the center of the scene, against a low-contrast monochrome or gradient background, containing no moving elements or content that could distract visual attention. The system guides the user, via voice or text prompts, to keep their head still and continuously gaze at the gaze point marker in the center of the scene.

[0117] While the user remains at rest, the system continuously collects eye-tracking data. Data collected during the baseline acquisition phase includes: the frequency distribution characteristics of resting-state eye tremors, reflecting the baseline level of the user's fixation stability; the amplitude distribution characteristics of resting-state eye tremors, reflecting the baseline level of the user's eye movement amplitude; the baseline value of the pupil diameter, reflecting the user's pupil size in a relaxed state; the stability index of the fixation point position, reflecting the baseline level of the user's fixation accuracy; and the natural blink frequency, used for parameter calibration of the subsequent blink detection algorithm.

[0118] Data acquired during the baseline acquisition phase will serve as a normal reference for subsequent nystagmus detection and abnormality assessment. For example, vestibular nystagmus is only diagnosed when eye movement characteristics significantly deviate from the baseline level.

[0119] The purpose of the progressive stimulation phase is to determine the user's vestibular sensitivity threshold. This phase lasts for sixty to ninety seconds.

[0120] During the progressive stimulation phase, the system controls the virtual scene to rotate horizontally around the user, simulating the user's visual experience in a rotating environment. The rotation speed increases progressively according to a preset gradient sequence, starting from a low initial speed and gradually increasing to a higher final speed.

[0121] During the progressive stimulation phase, the system continuously monitors the changing trends of each component of the nystagmus intensity index and the vestibular mismatch index. The system records the moment when the nystagmus intensity index first shows a significant increase. The criterion for a significant increase is that the nystagmus intensity index exceeds the mean value measured during the baseline acquisition phase plus three times the standard deviation. The scene rotation speed corresponding to this moment is recorded as an estimate of the user's individual vestibular stimulation sensitivity threshold.

[0122] The sensitivity threshold estimate reflects the critical stimulus intensity at which the user's vestibular system begins to produce an abnormal response. A higher sensitivity threshold indicates a stronger tolerance to visual-motor stimuli, while a lower sensitivity threshold indicates a weaker tolerance.

[0123] The purpose of the recovery observation phase is to determine the recovery time constant of the user's vestibular system. This phase lasts for twenty to forty seconds.

[0124] At the end of the progressive stimulation phase, the scene rotation speed has reached its maximum, and the user's vestibular mismatch index has typically risen to a high level. At the start of the recovery observation phase, the system immediately stops the scene rotation, restoring the scene to a completely static state.

[0125] During the static phase, the system continuously monitors the process of the vestibular mismatch index gradually recovering from its peak at the end of the stimulation phase to the baseline level. The system records the time elapsed until the vestibular mismatch index drops to half the difference between the peak and the baseline, and defines this time as the user's recovery time constant.

[0126] The recovery time constant reflects how quickly a user's vestibular system recovers from a stimulated state to a normal state. A shorter recovery time constant indicates a stronger recovery ability of the user's vestibular system and a faster decline in the influence of historical stimuli on the current state; a longer recovery time constant indicates a weaker recovery ability of the user's vestibular system and a longer duration of the influence of historical stimuli on the current state.

[0127] The purpose of the parameter calculation phase is to calculate the user's personalized parameter set based on the calibration measurement results.

[0128] The parameter calculation phase first categorizes users into one of four profile types based on the sensitivity threshold determined in the progressive stimulation phase and the recovery time constant determined in the recovery observation phase. The classification rules are as follows: When the rotational speed corresponding to the sensitivity threshold is higher than 70 degrees per second and the recovery time constant is less than three seconds, the user is classified as a high-tolerance, fast-recovery type.

[0129] When the rotational speed corresponding to the sensitivity threshold is between 40 and 70 degrees per second, and the recovery time constant is between 3 and 8 seconds, the user is classified as medium tolerance and medium recovery type.

[0130] When the rotational speed corresponding to the sensitivity threshold is less than 40 degrees per second, or the recovery time constant is longer than eight seconds, the user is classified as low-tolerance slow-recovery type.

[0131] When the fluctuation range of various indicators during the gradual stimulation phase exceeds 30% of the average, the user is classified as a fluctuation-adaptive type.

[0132] After the profile classification is completed, the parameter calculation stage reads the default parameters for the corresponding profile type from a pre-set parameter template library. The parameter template library pre-stores the recommended weight coefficient set and comfort threshold parameters for each profile type.

[0133] The parameter calculation phase also calculates the attenuation coefficient of the historical cumulative vertigo index based on the recovery time constant. The calculation method is to divide the value of the natural logarithm of 2 by the recovery time constant.

[0134] The purpose of the parameter storage and verification phase is to persistently store the calculated parameter set and optionally perform verification.

[0135] During the parameter storage and verification phase, the user's personalized parameter set is stored in the user profile database. A unique identifier for each user is established in the database, linking it to their parameter set, allowing for quick loading of personal configurations the next time the user uses the service.

[0136] After storage is complete, the system may optionally invite the user to participate in a short verification experience. The verification experience lasts from thirty to sixty seconds, allowing the user to move freely in a typical virtual reality scene and experience the actual effect of dynamic field of view adjustment.

[0137] After the verification experience is completed, the system asks users about their subjective feelings. If users report that the field of view contraction is too frequent or too large, it indicates that the comfort threshold parameter setting is too conservative, and the system appropriately increases the upper limit of the comfort threshold. If users report that they still feel dizzy or uncomfortable, it indicates that the comfort threshold parameter setting is too aggressive, and the system appropriately lowers the upper limit of the comfort threshold. The fine-tuned parameters based on user feedback are then overwritten and stored in the database.

[0138] This application describes a virtual reality dynamic field-of-view adjustment method: Step S1, Data Acquisition Step. At the beginning of each rendering frame, the multi-source signal acquisition module acquires eye-tracking data and head motion data within the corresponding time interval of the current frame from the sensors of the virtual reality headset. The data synchronization unit performs timestamp alignment and downsampling processing on the two data streams, and stores the acquired data in the sliding window buffer of the data buffer unit for use in subsequent steps.

[0139] Step S2, Nystagmus feature extraction step. The nystagmus signal extraction module processes the eye-tracking data within the sliding window. First, the eye velocity calculation unit performs preprocessing and velocity calculation, removes blink interference and high-frequency noise, fuses binocular data, and calculates the eye angular velocity sequence. Then, the fast-phase / slow-phase separation unit separates eye movements into fast-phase and slow-phase segments based on velocity thresholds. Next, the nystagmus pattern detection unit analyzes the directional relationship between the fast and slow phases within the sliding time window. When the angle between the slow-phase drift direction and the fast-phase jump direction is greater than the reverse judgment threshold, a vestibular nystagmus pattern is determined to exist. Finally, the nystagmus intensity quantification unit calculates the nystagmus intensity index when a nystagmus pattern is detected; if no nystagmus pattern is detected, the nystagmus intensity index is set to zero.

[0140] Step S3: Vestibular Mismatch Index Calculation. The vestibular mismatch index calculation engine receives the nystagmus intensity index and calculates the other three component indices. The motion cumulative load calculation unit performs time-decayed cumulative calculations based on head angular velocity data to calculate the motion cumulative load index. The scene complexity calculation unit obtains the optical flow information of the current rendered frame, calculates the ratio of the mean optical flow amplitude to the head angular velocity, and obtains the scene complexity index. The historical vertigo cumulative calculation unit reads the vestibular mismatch indexes of several past frames from the historical data cache, performs time-decayed weighted summation, and obtains the historical vertigo cumulative index. The weighted fusion calculation unit reads the current user's weight coefficient group from the user profile database, performs weighted summation of the four indices, and normalizes them to obtain a vestibular mismatch index with a value range of zero to one. After calculation, the vestibular mismatch index of the current frame is stored in the historical data cache.

[0141] Step S4: Field of View Calculation and Smoothing. The adaptive field of view controller receives the vestibular mismatch index. The user profile management unit reads the current user's personalized parameters and calculates the target field of view value using a segmented mapping strategy: the maximum field of view is output when the vestibular mismatch index is below the lower limit of the comfort threshold; the minimum field of view is output when the vestibular mismatch index is above the upper limit of the comfort threshold; and a smoothing interpolation function is used to calculate the target field of view when the vestibular mismatch index is between the two thresholds. The smoothing gradation unit reads the actual field of view value of the previous frame, calculates the difference between the target value and the actual value, limits the rate of change of the field of view to within the perception threshold, and outputs the actual field of view value of the current frame.

[0142] Step S5, Rendering Parameter Update Step. The rendering pipeline adaptation module receives the actual field of view angle value of the current frame. The frustum reconstruction unit recalculates the virtual camera's frustum parameters and perspective projection matrix based on the field of view angle value, and submits the updated projection matrix to the rendering pipeline. The edge mask generation unit determines whether the current field of view angle is less than the maximum field of view angle. If so, it calculates the inner and outer boundary radii of the mask and generates a ring-shaped gradient mask texture with a smooth transition in transparency from the inner boundary to the outer boundary. The foveation point rendering optimization unit obtains the user's current foveation point position and detects whether the foveation point is close to the mask area. If it is close, it locally expands the effective field of view angle in that direction to avoid occluding the foveation content. The foveation point rendering optimization unit also adjusts the hierarchical rendering resource allocation strategy based on the field of view angle state, reallocating the resources saved in the edge area to the foveation center area. After all rendering parameters are updated, the rendering engine executes scene rendering for the current frame. After rendering is complete, the actual field of view angle value of the current frame is stored in the frame state cache for use in the next frame.

[0143] Steps S1 to S5 are repeated in each rendering frame, with the latency of the entire processing flow controlled within two milliseconds. This allows processing to be completed between rendering frames without affecting the rendering frame rate or the smoothness of the image. By executing continuously frame by frame, the system achieves continuous monitoring of the user's vestibular state and real-time dynamic adjustment of the field of view.

[0144] The embodiments of this application are subject to the understanding of those skilled in the art, and various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of this application. The scope of this application is defined by the appended claims and their equivalents.

Claims

1. A virtual reality dynamic field-of-view adjustment system, characterized in that, include: A multi-source signal acquisition module is used to acquire eye-tracking data and head motion data of the user in the virtual reality environment in real time. The eye-tracking data includes the gaze angle of both eyes and the corresponding timestamp, and the head motion data includes head angular velocity and angular acceleration information. The nystagmus signal extraction module is connected to the multi-source signal acquisition module and is used to extract the fast phase and slow phase features of eye movement from the eye tracking data, identify the vestibular nystagmus pattern, and calculate the nystagmus intensity index based on the slow phase amplitude and nystagmus frequency. The vestibular mismatch index calculation engine is connected to the nystagmus signal extraction module and is used to weight and fuse the nystagmus intensity index, the cumulative motion load index calculated based on head movement, the scene complexity index calculated based on scene optical flow, and the historical vertigo cumulative index calculated based on historical data to obtain the vestibular mismatch index. An adaptive field of view controller, connected to the vestibular mismatch index calculation engine, is used to calculate the target field of view value through nonlinear mapping based on the vestibular mismatch index and user-personalized parameters, and generate a field of view adjustment command through smooth interpolation; The rendering pipeline adaptation module is connected to the adaptive field of view controller and is used to adjust the view frustum parameters of the virtual camera according to the field of view adjustment command, and generate an edge gradient mask when the field of view shrinks.

2. The virtual reality dynamic field of view adjustment system according to claim 1, characterized in that, The nystagmus signal extraction module includes: The eye velocity calculation unit is used to perform time differentiation on eye tracking data to obtain an eye angular velocity sequence; The fast-phase and slow-phase separation unit is used to separate eye movements into fast-phase segments and slow-phase segments according to a velocity threshold, wherein the velocity threshold ranges from 30 to 50 degrees per second. The nystagmosis mode detection unit is used to analyze the relationship between the slow phase drift direction and the fast phase jump direction within a sliding time window. When the angle between the two is greater than 120 degrees, it is determined that vestibular nystagmosis exists. The nystagmus intensity quantification unit is used to calculate the nystagmus intensity index based on the slow phase amplitude and nystagmus frequency of the nystagmus period.

3. The virtual reality dynamic field of view adjustment system according to claim 1, characterized in that, The vestibular mismatch index calculation engine includes: The cumulative motion load calculation unit is used to perform integral calculations on the head angular velocity with a time decay factor to obtain the cumulative motion load index. The scene complexity calculation unit is used to use the ratio of the average amplitude of scene optical flow to the head angular velocity as a scene complexity index. The historical vertigo accumulation calculation unit is used to perform a weighted summation of the vestibular mismatch index of historical frames with a decay factor to obtain the historical vertigo accumulation index. The weighted fusion calculation unit is used to sum the above four indicators according to the weight coefficients corresponding to the user profile to obtain the normalized vestibular mismatch index.

4. The virtual reality dynamic field-of-view adjustment system according to claim 1, characterized in that, The adaptive field-of-view controller includes: The user profile management unit is used to store users' personalized parameters, including weight coefficient groups, upper and lower limits of comfort thresholds and historical decay coefficients, and classifies users into high tolerance and fast recovery type, medium tolerance and medium recovery type, low tolerance and slow recovery type or fluctuating adaptation type according to their physiological tolerance characteristics. The field of view mapping unit is used to calculate the target field of view value using a piecewise mapping strategy. When the vestibular mismatch index is lower than the lower limit of the comfort threshold, it outputs the maximum field of view; when it is higher than the upper limit of the comfort threshold, it outputs the minimum field of view; when it is in between, it uses a smooth interpolation function to calculate. The smooth gradient unit is used to limit the rate of change of the field of view to no more than 10 to 20 degrees per second, so as to avoid abrupt changes in the field of view.

5. The virtual reality dynamic field-of-view adjustment system according to claim 1, characterized in that, The rendering pipeline adaptation module includes: The view frustum reconstruction unit is used to update the projection matrix of the virtual camera according to the view angle adjustment command; The edge masking generation unit is used to generate a ring mask that gradually transitions outward from the edge of the effective field of view to complete occlusion, and a cosine function is used to achieve a smooth transition. The gaze point rendering optimization unit is used to dynamically allocate rendering resources based on the user's gaze point position, allocating the resources saved in the masked area to the gaze center area to improve image quality.

6. A method for adjusting the dynamic field of view in virtual reality, implemented based on the virtual reality dynamic field of view adjustment system of any one of claims 1-5, characterized in that, include: Step S1: Collect the user's eye-tracking data and head movement data in real time; Step S2: Extract nystagmus features from eye-tracking data, identify vestibular nystagmus by analyzing fast and slow phase motion patterns, and calculate nystagmus intensity index. Step S3: The nystagmus intensity index, cumulative motion load index, scene complexity index, and historical vertigo cumulative index are weighted and fused to obtain the vestibular mismatch index. Step S4: Calculate the target field of view value based on the vestibular mismatch index and user-personalized parameters, and generate a field of view adjustment sequence through smooth interpolation; Step S5: Update the view frustum parameters according to the view angle adjustment sequence, generate an edge gradient mask, and optimize the rendering resource allocation of the foveation point region.

7. The virtual reality dynamic field of view adjustment method according to claim 6, characterized in that, Step S2 includes: Preprocess the eye-tracking data to remove blinking interference and high-frequency noise; Calculate the eyeball angular velocity sequence and separate fast-phase and slow-phase segments based on velocity thresholds; Within the sliding time window, the average drift direction of the slow phase and the jump direction of the fast phase are calculated. When the angle between the two is greater than the reverse judgment threshold, the presence of vestibular nystagmus is determined. The nystagmus intensity index is calculated by multiplying the slow phase amplitude of the nystagmus period by the cumulative sum of the nystagmus frequency.

8. The virtual reality dynamic field of view adjustment method according to claim 6, characterized in that, Step S3 includes: The cumulative motion load index is obtained by performing an integral calculation on the head angular velocity with time decay, and the decay rate is determined according to the user profile type. The scene complexity index is obtained by dividing the average optical flow amplitude of the scene by the head angular velocity. The historical vertigo cumulative index is obtained by performing a weighted summation with attenuation on the vestibular mismatch index of historical frames. The attenuation coefficient is determined based on the user's recovery time constant. The four indicators are weighted, summed, and normalized according to the weight coefficients corresponding to the user profiles to obtain the vestibular mismatch index, which has a value range of zero to one.

9. The virtual reality dynamic field of view adjustment method according to claim 6, characterized in that, The user calibration step is included before step S1: During the baseline acquisition phase, eye movement baseline data of the user in a resting state are collected for 20 to 40 seconds. During the progressive stimulation phase, the scene was rotated at an increasing speed, and the rotation speed at which the nystagmus intensity first significantly increased was recorded as the sensitivity threshold estimate. During the recovery observation phase, the scene motion was stopped, and the time required for the vestibular mismatch index to recover to the baseline was recorded as the recovery time constant. Users are categorized into corresponding profile types based on sensitivity thresholds and recovery time constants, and a set of personalized parameters is determined.

10. The virtual reality dynamic field of view adjustment method according to claim 6, characterized in that, Step S5 includes: Calculate the difference between the target field of view and the current field of view. If the absolute value of the difference exceeds the maximum change in a single frame, adjust it gradually at a limited rate. Update the virtual camera's frustum parameters and projection matrix based on the actual field of view angle values; Generate a ring-shaped gradient mask with an inner boundary corresponding to the effective field of view and an outer boundary corresponding to the maximum field of view, and use a cosine curve to smoothly transition the transparency; Detect the user's gaze point position, and when the gaze point is close to the occlusion area, locally expand the effective field of view to avoid obscuring the gazed content; The rendering resources saved in the masked area are redistributed to the gaze center area, improving the image quality in the core area.

Images