Real-time tracking system for continuous roaming of eye-level vr perspective

Abstract

The present application relates to computer graphics and real-time rendering field, disclose a kind of naked-eye VR view angle continuous roaming real-time tracking system, comprising: multi-source viewpoint acquisition unit, rendering feedback prediction unit and viewport consistency arbitration unit connected with depth buffer and template buffer, the present system is through rendering feedback prediction unit, according to the object attribute purification motion vector of template buffer data hierarchical solution to deduce viewpoint, viewport consistency arbitration unit is when external sensing data deviation overrun, forcedly use deduced viewpoint to take over rendering, the present application constructs independent hot backup and abnormal takeover mechanism based on rendering pipeline data, uses picture visual residual inertia to fill in external sensor data blank, avoid rendering collapse caused by sensor signal interruption or asynchronization.

Description

Technical Field

[0001] This invention relates to a real-time tracking system for continuous roaming in naked-eye VR, belonging to the field of computer graphics and real-time rendering technology. Background Technology

[0002] In the current viewport generation process of naked-eye virtual reality and augmented reality, the graphics rendering pipeline relies on external tracking modules to provide spatial pose data to construct the projection matrix. The industry typically adopts a serial architecture of perception transmission and rendering, in which physical coordinates are collected by an inertial measurement unit or visual sensor and filtered to drive the rendering pipeline to refresh the screen. However, the open-loop architecture has inherent defects in asynchronous frequency sampling and complex dynamic scenes. There is a timing mismatch between the high-frequency sampling of the sensor and the low-frequency refresh of the rendering pipeline, which causes high-frequency noise to be introduced into the inter-frame interpolation of viewport data, resulting in slight screen flickering. External sensors may experience a decrease in data confidence due to occlusion or changes in ambient light. The rendering pipeline may be forced to reuse old frames or stop updating due to a lack of viewpoint input, causing visual stuttering or jumps.

[0003] Besides the timing misalignment problem, some solutions attempt to track users through external physical compensation mechanisms, but this introduces more serious lag and discontinuity issues. For example, Chinese invention patent CN107390622A discloses a smart VR projection device with real-time tracking function. This solution uses a mechanical movement mechanism composed of a motor, drive shaft, and slider to track users by physically translating the main body of the projection device. This approach uses physical motion to respond to changes in user pose. The mechanical inertia and response delay are much greater than the data refresh cycle of the rendering pipeline. Not only does it fail to solve the high-frequency jitter problem of the rendering viewport, but the physical lag characteristics also exacerbate the deviation between the viewport and the user's actual position, failing to meet the real-time and low-latency requirements of VR / AR systems. Existing technologies often alleviate the above problems by introducing high-cost sensors or external predictive filtering algorithms, but such methods are outside the rendering pipeline and lack scene depth information and dynamic object semantic perception capabilities.

[0004] Therefore, the technical problem to be solved by this invention is how to mine the intermediate process data inside the graphics rendering pipeline and build a scene semantic awareness viewpoint adaptive inference and compensation mechanism that does not rely on external hardware. Summary of the Invention

[0005] To address the problems mentioned in the background art, the technical solution of the present invention is as follows: a real-time tracking system for continuous roaming of naked-eye VR viewpoint, the system running in a graphics rendering pipeline environment including a depth buffer and a stencil buffer, the system comprising:

[0006] A multi-source viewpoint acquisition unit is used to acquire external sensor signals and calculate them to output first viewpoint parameters;

[0007] The rendering feedback prediction unit, connected to the depth buffer and the stencil buffer, is used to read the global motion vector field of the previous rendering frame. The rendering feedback prediction unit identifies and removes vector data corresponding to non-static objects in the global motion vector field based on the object attribute identifiers in the stencil buffer to generate static vector data. The rendering feedback prediction unit divides the static vector data into a far-field vector set and a near-field vector set based on the depth value distribution in the depth buffer. It uses the far-field vector set to calculate the rotation component of the viewpoint pose and the near-field vector set to calculate the translation component of the viewpoint position, and reverse-derives to generate the second viewpoint parameters.

[0008] The viewport consistency arbitration unit is used to calculate the deviation between the first viewpoint parameter and the second viewpoint parameter. When the deviation is less than the preset continuity threshold, the viewport consistency arbitration unit generates the target viewpoint parameter based on the weighted calculation of the first viewpoint parameter and the second viewpoint parameter. When the deviation is greater than the continuity threshold, the viewport consistency arbitration unit performs an abnormal takeover operation and directly replaces the first viewpoint parameter with the second viewpoint parameter as the target viewpoint parameter.

[0009] The real-time viewport mapping unit is used to receive target viewpoint parameters and construct a projection matrix to perform image rendering.

[0010] Preferably, the rendering feedback prediction unit performs a depth weight allocation operation: the rendering feedback prediction unit establishes a mapping relationship between each pixel in the static vector data and the depth value in the depth buffer; for the solution of the translation component, the rendering feedback prediction unit sets the solution weight of the corresponding pixel to be inversely proportional to the depth value of the pixel; for the solution of the rotation component, the rendering feedback prediction unit sets the solution weight of the corresponding pixel to be proportional to the depth value of the pixel, thereby separating the coupled components of translation and rotation motion on the two-dimensional projection plane under monocular vision conditions.

[0011] Preferably, the rendering feedback prediction unit performs culling of the global motion vector field through a bit mask operation: the rendering feedback prediction unit reads the bit plane data in the template buffer and generates a binary mask; the rendering feedback prediction unit performs a bitwise AND operation with the global motion vector field using the binary mask, sets all motion vectors corresponding to the pixel positions of the dynamic attribute identifier to zero, and retains only the motion vectors corresponding to the pixel positions of the static background attribute identifier as static vector data.

[0012] Preferably, the rendering feedback prediction unit is also used to perform texture validity evaluation: while extracting the global motion vector field, the rendering feedback prediction unit calculates the grayscale histogram of the current rendering frame image and derives the texture information entropy value; the rendering feedback prediction unit compares the texture information entropy value with a preset feature richness threshold; if the texture information entropy value is lower than the feature richness threshold, the rendering feedback prediction unit generates a confidence blocking signal; in response to the confidence blocking signal, the viewport consistency arbitration unit locks the trigger state of the abnormal takeover operation to prohibited and switches to a mode that relies solely on the inertial measurement component in the multi-source viewpoint acquisition unit for integral inference.

[0013] Preferably, the viewport consistency arbitration unit is connected to the rendering granularity controller: when the viewport consistency arbitration unit performs an abnormal takeover operation, the viewport consistency arbitration unit synchronously sends a degradation trigger command to the rendering granularity controller; in response to the degradation trigger command, the rendering granularity controller performs rendering parameter adjustment operations to reduce the rendering resolution or switch to a low level of detail model, thereby shortening the reverse inference time step of the global motion vector field.

[0014] Preferably, when the deviation value is less than the continuity threshold, the viewport consistency arbitration unit performs a dynamic weighted fusion operation: the viewport consistency arbitration unit calculates the dynamic fusion factor based on the deviation value and generates the target viewpoint parameters according to the following formula: ,in, For target viewpoint parameters, For the first viewpoint parameters, For the second viewpoint parameters, The dynamic fusion factor is set as a monotonically increasing function of the deviation value, and its value range is limited to a closed interval between zero and one.

[0015] Preferably, the viewport consistency arbitration unit is also used to perform drift correction: after an abnormal takeover operation is triggered and the duration reaches a preset duration threshold, the viewport consistency arbitration unit performs a re-anchoring operation; the re-anchoring operation gradually reduces the weight of the second viewpoint parameter with a preset linear rate of change, and gradually increases the weight of the first viewpoint parameter, until the target viewpoint parameter is completely restored to the first viewpoint parameter.

[0016] Preferably, the multi-source viewpoint acquisition unit includes an inertial measurement unit and a visual positioning sensor: the inertial measurement unit outputs angular velocity and acceleration data at a first frequency; the visual positioning sensor outputs spatial position coordinates at a second frequency lower than the first frequency; the multi-source viewpoint acquisition unit integrates the angular velocity and acceleration data, corrects them using the spatial position coordinates, and calculates and generates first viewpoint parameters.

[0017] Preferably, the real-time viewport mapping unit is used to perform temporal anti-aliasing processing: the real-time viewport mapping unit uses the second viewpoint parameters to perform pixel-level reprojection alignment between the current rendering frame and the historical rendering frames; the rendering feedback prediction unit directly reads the pixel motion data generated during the temporal anti-aliasing process as a global motion vector field.

[0018] Preferably, the system runs on a mobile terminal device, which includes a graphics processor: a depth buffer and a stencil buffer reside in the graphics processor's video memory; a rendering feedback prediction unit uses the graphics processor's computational shader to perform culling and layering calculations on the global motion vector field; and a viewport consistency arbitration unit runs as a logical thread independent of the rendering thread, completing the generation of target viewpoint parameters before the vertical synchronization signal of the rendering frame arrives.

[0019] Compared with the prior art, the beneficial effects of the present invention are:

[0020] 1. A viewport continuity maintenance mechanism based on rendering feedback loop is constructed within the graphics rendering pipeline. A global motion vector-based back-inference logic is built to establish a visual inertial reference system independent of external physical sensors. Pixel motion data generated when the rendering pipeline generates the previous frame image is used to calculate the instantaneous motion state of the camera through inverse optical flow analysis, serving as a predictive constraint for the next frame's viewport generation. The graphics rendering system transforms from a data consumer to a viewpoint data producer, utilizing the visual residual inertia of the image itself to fill data gaps caused by external sensors during sampling intervals or signal interruptions. The closed-loop feedback mechanism eliminates viewport jitter caused by the asynchrony between sensor sampling frequency and rendering frame rate from the data source without introducing additional hardware clock synchronization devices. This ensures that the rendered viewport maintains a smooth transition based on visual logic even under extreme conditions such as sudden loss of sensor data or a decrease in confidence.

[0021] 2. By utilizing the inherent depth buffer data of the graphics rendering pipeline as the geometric constraint for 3D motion calculation, the system resolves the ambiguity in the projection of translational and rotational motions onto the 2D screen flow field under monocular vision conditions. Based on the optical principle that parallax is inversely proportional to depth in perspective projection, the system establishes a dynamic mapping relationship between depth values ​​and calculation weights. It uses the high parallax characteristics of near-field pixels to lock the camera translation vector and the low parallax characteristics of far-field pixels to lock the camera rotation component. The layered calculation logic reuses the existing depth data structure of the rendering pipeline. Without introducing binocular cameras or LiDAR ranging hardware, it achieves accurate decomposition of the camera's 3D spatial motion, avoiding perspective distortion and depth drift phenomena that occur when a single optical flow method processes complex composite motions.

[0022] 3. By using the template buffer or object attribute identifier in the rendering pipeline as a viewpoint tracking semantic filter, deterministic shielding of dynamic interference sources within the scene is achieved. The motion vector calculation process uses pixel-level object classification information provided by the template buffer. Bit mask operations are used to filter out corresponding non-static rendering objects and dynamic interactive interface vector data, while retaining static background motion information for camera pose inference. The rendering system provides prior knowledge of the virtual scene, replacing the traditional machine vision's computationally intensive probabilistic outlier culling algorithm. This ensures that in complex dynamic environments with numerous mobile devices or personnel occlusion in industrial settings, the system can lock a stable static spatial reference system at extremely low computational cost, avoiding viewpoint misjudgment due to dynamic environmental changes. Attached Figure Description

[0023] Figure 1 This is a schematic diagram of the system rendering feedback and arbitration architecture of the present invention;

[0024] Figure 2 This is a comparison diagram of tracking error and inter-frame jitter under high-frequency jitter conditions according to the present invention;

[0025] Figure 3 This is a timing diagram of the viewpoint parameter generation process in the normal tracking mode of the present invention. Detailed Implementation

[0026] To make the objectives, technical solutions, and effects of the present invention clearer and more explicit, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, in the absence of conflict, the embodiments of the present invention and the technical features therein can be combined with each other, and the specific embodiments described herein are only used to explain the present invention and are not intended to limit the scope of protection of the present invention.

[0027] This invention discloses a real-time tracking system for continuous roaming in naked-eye VR. The system operates within a graphics rendering pipeline environment that includes a depth buffer and a stencil buffer. The system includes: a multi-source viewpoint acquisition unit, a rendering feedback prediction unit, a viewport consistency arbitration unit, and a real-time viewport mapping unit. The multi-source viewpoint acquisition unit is used to acquire external sensor signals and calculate and output first viewpoint parameters. The rendering feedback prediction unit, connected to the depth buffer and the stencil buffer, is used to reverse-engineer and generate second viewpoint parameters based on pipeline data of the rendered frame. The viewport consistency arbitration unit is used to compare the first viewpoint parameters and the second viewpoint parameters and, based on the deviation value... The system generates target viewpoint parameters; the real-time viewport mapping unit constructs a projection matrix based on the target viewpoint parameters and performs final image rendering; the multi-source viewpoint acquisition unit includes an inertial measurement unit and a visual positioning sensor; the inertial measurement unit outputs high-frequency angular velocity and acceleration data at a first frequency (1000Hz), while the visual positioning sensor (SLAM camera) outputs spatial position coordinates at a second frequency (60Hz) lower than the first frequency; the multi-source viewpoint acquisition unit integrates and extrapolates the angular velocity and acceleration data, and periodically corrects using the spatial position coordinates, ultimately calculating and generating the first viewpoint parameters, which are used for viewpoint tracking. The main basis for this is the system's construction of a feedback prediction mechanism based on rendering pipeline data. The rendering feedback prediction unit is deployed in the graphics processor and executed using the compute shader. At the end of each rendering frame, the unit, connected to the graphics processor's video memory, is used to read the global motion vector field, depth buffer data, and stencil buffer data generated in the previous rendering frame. The global motion vector field does not require additional calculation; it directly reuses the pixel motion data generated by the real-time viewport mapping unit during the execution of temporal anti-aliasing (TAA) processing for historical frame reprojection alignment. The rendering feedback prediction unit, based on the premise of culling dynamic interference from the stencil buffer, is: real-time viewport... The mapping unit performs a stencil buffer write operation during the geometry processing stage or depth prepass of the previous frame rendering. The rendering pipeline marks each rendered object in the 3D scene as a static background or a dynamic entity based on its preset object attributes. During rasterization, the pixel position corresponding to the static background attribute is written to the first stencil identifier value, and the pixel position corresponding to the dynamic entity attribute is written to the second stencil identifier value. The rendering feedback prediction unit reads the bit plane data of the stencil buffer, generates a binary mask that retains the position corresponding to the first stencil identifier value, and performs bitwise operations with the global motion vector field using the mask to achieve deterministic removal of dynamic interference.

[0028] To eliminate dynamic interference, the rendering feedback prediction unit (RFP) semantically purifies the global motion vector field based on the object attribute identifiers in the stencil buffer. During rendering, the system writes different stencil buffer identifier values ​​for static background objects and dynamic objects. The RFP reads the bit-plane data in the stencil buffer and generates a binary mask. The unit performs a bitwise AND operation between the binary mask and the global motion vector field, setting all motion vectors corresponding to pixel positions with dynamic attribute identifiers to zero, while retaining motion vectors corresponding to pixel positions with static background attribute identifiers, generating static vector data. The RFP uses the depth value distribution in the depth buffer to perform layered computation on the static vector data. The system then converts the static vector data into... The data is divided into far-field vector sets and near-field vector sets. The rendering feedback prediction unit performs depth weight allocation: for the translation component, the calculation weight of the corresponding pixel is set to be inversely proportional to the depth value of that pixel; for the rotation component, the calculation weight of the corresponding pixel is set to be directly proportional to the depth value of that pixel. Through weight allocation, the system uses the far-field vector set to calculate the rotation component of the viewpoint pose and the near-field vector set to calculate the translation component of the viewpoint position, and reverse-engineers the second viewpoint parameters. The procedure for the rendering feedback prediction unit to perform layered calculation using the depth buffer data is as follows: when processing static vector data in parallel using the computation shader, for each pixel, the corresponding depth value is read. Based on the calibrated near-field cutoff depth Far-field starting depth Establish weight mapping; solve for rotation components using depth values. Greater than The far-field pixel set, with The solution weights are increased by increasing the value, and the global rotation components are calculated using a weighted average; for the translation components, the element depth value is used for solution. Less than The near-field pixel set, after deducting the projection contribution of the previously calculated rotation components onto the two-dimensional plane, is... The solution weights are reduced and enhanced, and the global translation components are calculated using a weighted average. and The values ​​are determined through offline parameter calibration procedures to adapt to the camera intrinsics of different terminal devices or the working scale of specific application scenarios.

[0029] The viewport consistency arbitration unit runs as a logical thread independent of the rendering thread. It arbitrates the target viewpoint parameters before the vertical synchronization signal arrives for each rendering frame. The unit calculates the deviation between the first and second viewpoint parameters, which can be the Euclidean distance between the position coordinates and the angle difference between the pose quaternions. The unit compares the deviation with a preset continuity threshold. When the deviation is less than the continuity threshold, the system performs a dynamic weighted fusion operation, adjusting the target viewpoint parameters. According to the formula Generate, where For target viewpoint parameters, For the first viewpoint parameters, For the second viewpoint parameters, As a dynamic fusion factor, The function is set to a monotonically increasing function with respect to the deviation value to smooth out minor jitter from external sensors using internal prediction data. When the deviation value exceeds a continuity threshold, the viewport consistency arbitration unit performs an anomaly takeover operation, directly replacing the first viewport parameter with the second viewport parameter as the target viewport parameter output. The real-time viewport mapping unit receives the target viewport parameter provided by the viewport consistency arbitration unit, constructs a view matrix and projection matrix based on the target viewport parameter, performs rasterization rendering on the 3D scene, and finally outputs a naked-eye VR view image. To address the issue of motion vector field failure caused by low-texture scenes (white walls or overexposed areas), the rendering feedback prediction unit performs texture validity evaluation. While extracting the global motion vector field, the unit calculates the grayscale histogram of the current rendered frame image and derives the texture information entropy value. The unit compares the texture information entropy value with a preset feature richness threshold. If the texture information entropy value is lower than the feature richness threshold, the rendering feedback prediction unit generates a confidence blocking signal. The viewport consistency arbitration unit responds to the confidence blocking signal and locks the viewport. The abnormal takeover operation is triggered in a prohibited state, switching to a mode that relies on the integral deduction of the inertial measurement component in the multi-source viewpoint acquisition unit. The system also establishes a state-driven adaptive adjustment mechanism for rendering granularity. The viewport consistency arbitration unit is connected to the rendering granularity controller. When the viewport consistency arbitration unit performs an abnormal takeover operation, it synchronously sends a degradation trigger command to the rendering granularity controller. In response to the degradation trigger command, the rendering granularity controller performs rendering parameter adjustment operations, including reducing the rendering resolution or switching to a low level of detail (LOD) model, physically shortening the reverse deduction time step of the global motion vector field. The viewport consistency arbitration unit is also used to perform drift correction. After the abnormal takeover operation is triggered and the duration reaches a preset duration threshold (2 seconds), when the first viewpoint parameter recovers to stability, the viewport consistency arbitration unit performs a re-anchoring operation. The re-anchoring operation gradually reduces the weight of the second viewpoint parameter and increases the weight of the first viewpoint parameter at a preset linear rate of change until the target viewpoint parameter is completely restored to be dominated by the first viewpoint parameter.

[0030] Example 1: In a naked-eye AR industrial inspection scenario with dynamic occlusion, the operator uses the mobile terminal of this invention to perform AR-assisted maintenance on a large static compressor. An automated logistics forklift, marked as a non-static object during rendering, drives laterally across the near-field of view between the operator and the compressor. At the instant the forklift enters the field of view, the visual positioning sensor of the multi-source viewpoint acquisition unit, due to large-area contamination of observed feature points, calculates and outputs the first viewpoint parameters (…). This results in a drastic translation jump; before arbitration is performed in the rendering frame, the rendering feedback prediction unit has already executed vector mask filtering logic based on the stencil buffer when processing the (N-1)th frame; the logic, based on the object attribute identifiers in the stencil buffer, removes all motion vectors generated by pixels corresponding to non-static objects (logistics forklifts), and retains static vector data corresponding to distant static background objects (compressors); the unit uses depth buffer data to perform layered solution logic on the static vector data, distinguishes between translation and rotation, and reverse-derives the second viewpoint parameters that only reflect the slight movements of the operator's hand ( In the rendering frame, the viewport consistency arbitration unit calculates and finds that... and The deviation value between them instantaneously exceeds the preset continuity threshold due to the drastic change in the external sensor; the system determines that the external tracking has failed, triggers an abnormal takeover operation, and changes the second viewpoint parameters ( ) as the final target viewpoint parameter ( The output is sent to the rendering pipeline; the real-time viewport mapping unit constructs a projection matrix based on the target viewpoint parameters derived from internal calculations; the AR digital twin model superimposed on the compressor is attached to the static equipment surface behind it; until the forklift completely leaves the field of view, Once stability is restored and the deviation value returns to within the threshold, the viewport consistency arbitration unit automatically switches back to dynamic weighted fusion operation, and re-anchors the viewpoint to external physical coordinates through drift correction logic.

[0031] Example 2: In a standardized graphics rendering test platform, the viewport continuity maintenance capability of the system of the present invention under abnormal external sensor signal conditions was quantitatively evaluated. The platform runs on a terminal equipped with a graphics processor, which has the ability to execute compute shaders, read and write depth buffers and stencil buffers. The test scene is set as a three-dimensional virtual space containing static backgrounds and dynamic interference objects, and a smooth ground real viewpoint path is preset as the evaluation benchmark. Multiple control experiments are set up. Sample groups B and D of the present invention use the complete system and simultaneously enable the multi-source viewpoint acquisition unit, the rendering feedback prediction unit (stencil buffer culling and depth buffer layered calculation), and the viewport consistency arbitration unit. Control groups A and C use the existing system. One approach relies on a multi-source viewpoint acquisition unit but lacks rendering feedback and arbitration takeover functions; control group E enables the arbitration unit, but the rendering feedback prediction unit disables data reading from the template buffer and depth buffer, making it unable to eliminate dynamic interference or decouple motion; the experiment is divided into two conditions: Condition 1 is high-frequency jitter, simulating the small noise generated by external sensors during continuous operation, injecting position noise with a mean of 0 and a standard deviation of 2.0 mm and attitude noise of 0.1 degrees into the real ground viewpoint path; Condition 2 is signal interruption, which, based on Condition 1, additionally simulates visual positioning sensor occlusion, forcibly interrupting the first viewpoint parameters during the period from frame 100 to frame 130 of the test cycle (lasting approximately 500 milliseconds). The system outputs 60Hz frame rate during the experiment, records the final viewport parameters of each group in real time, and calculates the deviation between the final viewport parameters and the actual viewpoint path on the ground. The comparison data of key performance indicators are shown in Table 1.

[0032] Table 1: Comparison of viewpoint tracking errors for each test group under different working conditions.

[0033] ;

[0034] Referring to Table 1, in the high-frequency jitter condition, control group A had a larger error and jitter, resulting in slight image trembling; sample group B of this invention, through dynamic weighted fusion of the viewport consistency arbitration unit, utilized... smooth The noise was reduced, the average position error was reduced by approximately 80.5%, and the inter-frame jitter was reduced by approximately 88.2%. In the second signal interruption scenario, the control group C lost input, and the viewpoint was locked in the last frame before the interruption, causing the position error to accumulate linearly with the actual movement (45.31 mm). This resulted in a huge jump (maximum jitter 75.40 mm) when the signal was restored, i.e., rendering collapse. The present invention's sample group D, after detecting a deviation value greater than the continuity threshold, triggered an abnormal takeover operation, utilizing... Filling in a 500-millisecond data gap resulted in a prediction drift of only 3.81 mm, with inter-frame jitter maintained at 0.31 mm. The control group E simulated a scenario where depth and template buffer coordination was not employed. Contaminated by dynamic interference objects in the scene; when the same abnormal takeover is triggered, its error (22.43 mm) is much higher than that of sample group D of this invention, indicating that the semantic culling (template) and hierarchical solution (depth) mechanism in the rendering feedback prediction unit is to ensure the second viewpoint parameters ( It provides the basis for accuracy and supports the arbitration unit's takeover operation in complex and dynamic scenarios.

[0035] Example 3: This example combines Figures 1 to 3 A description of a real-time tracking system for continuous roaming in naked-eye VR, such as... Figure 1As shown, external sensor signal IMU data / spatial coordinates are input to the multi-source viewpoint acquisition unit, which acquires signals and calculates them to output the first viewpoint parameters. The pipeline data storage unit includes a depth buffer, a template buffer, and a global motion vector field. The rendering feedback prediction unit reads the depth, template, and vector field data, and generates the second viewpoint parameters through layered calculation and reverse inference of static vector data. The viewport consistency arbitration unit receives the first and second viewpoint parameters, calculates the deviation and abnormal takeover, and generates the target viewpoint parameters. The target viewpoint parameters are input to the real-time viewport mapping unit, which constructs a projection matrix to perform image rendering and finally outputs a naked-eye VR image. At the same time, this unit writes the data to the pipeline data storage for the next frame. In addition, the viewport consistency arbitration unit can also send a degradation trigger command to the rendering granularity controller.

[0036] like Figure 2 As shown in the figure, this comparison shows that control group A only Compared with the complete system of sample group B of the present invention, the horizontal axis of the figure shows three performance indicators: average position error (mm), maximum position error (mm), and average inter-frame jitter (mm / frame). The vertical axis represents the corresponding error value. The data shows that the error value of sample group B of the present invention is lower than that of control group A in all three indicators; Figure 3 As shown, this process involves a multi-source viewpoint acquisition unit, a rendering feedback prediction unit, a viewport consistency arbitration unit, a real-time viewport mapping unit, and a graphics processor. The process begins with the multi-source viewpoint acquisition unit acquiring sensor data and outputting the first viewpoint parameters. Simultaneously, the graphics processor (GPU) executes the rendering of the previous frame. The GPU provides the rendering feedback prediction unit (RFP) with depth buffer data, stencil buffer data, and the global motion vector field. The RFP then sequentially performs dynamic object culling based on the stencil buffer and layered resolution based on the depth buffer, and outputs the second viewpoint parameters. Viewport consistency arbitration unit receives and After calculating the deviation value, dynamic weighted fusion is performed to output the target viewpoint parameters. Finally, the real-time viewport mapping unit receives... It constructs a projection matrix and executes rendering of the current frame, outputting the rendered image.

[0037] Example 4: The specific algorithm implementation and parameter determination procedure within the rendering feedback prediction unit and the viewport consistency arbitration unit are as follows: The rendering feedback prediction unit needs to reverse-engineer the second viewpoint parameters from the static vector data that has been freed of dynamic interference. , and provides a dynamic fusion factor for the viewport consistency arbitration unit ( The calculation basis for the rendering feedback prediction unit is as follows: The rendering feedback prediction unit performs the following steps in the computation shader: First, establish a weight mapping, and for each pixel in the static vector data, read the corresponding depth value. Based on the preset near-field cutoff depth (1.5 meters) and far-field starting depth (10.0 meters), calculate the translation solution weights. With rotational solution weights ;in Set as Increases and decreases, while Set as Increase and enhance; the second step is to solve for the rotational components, using only the elements. The far-field pixels and their motion vectors, through Weighted average, calculate the global rotation matrix The third step is to solve for the translation components, using the element... Near-field pixels and their motion vectors, after deducting the near-field pixels and their motion vectors, are analyzed. After contributing the rotation amount, through Weighted average, calculate the global translation vector The fourth step is to synthesize parameters, and the unit will... and Combined together, the second viewpoint parameters are generated. The viewport consistency arbitration unit obtains... and Deviation value between Then, the dynamic fusion factor is calculated according to the following procedure. The system presets a smoothing start threshold. (0.5 mm) and as a continuity threshold (5.0 mm); when the deviation value hour, Set to 0, at this time fully utilized When the deviation value hour, When set to 1, an exception takeover is triggered, and full implementation is adopted. ;when hour, Calculated by linear interpolation .

[0038] Example 5: When the system is deployed on a new mobile terminal or in a specific rendering scenario, an offline parameter calibration procedure needs to be performed to determine the near-field cutoff depth. Far-field starting depth The procedure is executed in a controlled environment with rich texture features and known depth: the terminal device performs pure translational motion along a single axis, the rendering feedback prediction unit continuously calculates the translational components and compares them with the integral data of the inertial measurement unit, and the upper limit of the depth interval with the highest correlation and most stable error is calibrated as... The terminal device performs pure rotational motion around a single axis. The unit continuously calculates the rotational component and compares it with the angular velocity data of the inertial measurement unit. The depth value where the correlation between the two reaches saturation (99.5%) and no longer changes with increasing depth is calibrated as... ; after calibration and The values ​​(1.2 meters and 11.5 meters) are fixed as the default parameters for this deployment environment.

[0039] Continuity threshold of viewport consistency arbitration unit Smoothing start threshold The feature richness threshold and drift correction duration threshold for the rendering feedback prediction unit were also determined through a standardized testing process. During this process, the system operated in normal tracking mode within the aforementioned controlled environment, recording at least 10,000 frames of data. and Deviation value between The baseline noise distribution was calibrated to three times the standard deviation (0.7 mm). In this process, a series of analog signal jumps with increasing amplitude are artificially introduced, and the deviation value (4.5 mm) perceived as a viewport jump by the first test subject is calibrated as... The feature richness threshold is determined by statistically sampling the texture information entropy values ​​of a series of known low-texture scenes and taking the 95th percentile value; the duration threshold is determined based on the abnormal takeover mode. The cumulative drift rate was calculated by inversely using the maximum acceptable drift amount (10 mm) (1.8 seconds).

[0040] Example 6: The fusion calculation procedure inside the multi-source viewpoint acquisition unit is used to calculate the first viewpoint parameters by combining the high-frequency angular velocity and acceleration data of the inertial measurement unit with the low-frequency spatial position coordinates of the visual positioning sensor. The unit employs complementary filtering logic: high-pass filtering and real-time integration of angular velocity and acceleration data generate high-frequency viewpoint attitude and position; low-pass filtering of spatial position coordinates generates a low-frequency correction reference; the system uses a preset dynamic time constant to weightedly fuse the high-frequency viewpoint and the low-frequency correction reference, ultimately outputting the first viewpoint parameters. The specific algorithm for texture validity evaluation executed by the rendering feedback prediction unit, which calculates texture information entropy values ​​and generates confidence blocking signals, is as follows: The unit samples the grayscale image of the current rendering frame in parallel on the graphics processor, and statistically analyzes the texture information entropy values ​​and generates confidence blocking signals. gray levels ( ) grayscale histogram, calculate each gray level probability of occurrence The unit calculates the texture information entropy value based on the Shannon entropy definition. ,in, hour Recorded as 0; the unit will calculate the result. The value is compared with the preset feature richness threshold. If If the value is below this threshold, a confidence blocking signal is generated for the viewport consistency arbitration unit to invoke.

[0041] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0042] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A real-time tracking system for continuous walkthrough of a VR view in a naked eye, characterized in that, The system operates within a graphics rendering pipeline environment that includes a depth buffer and a stencil buffer. The system includes: A multi-source viewpoint acquisition unit is used to acquire external sensor signals and calculate them to output first viewpoint parameters; The rendering feedback prediction unit, connected to the depth buffer and the stencil buffer, reads the global motion vector field of the previous rendering frame. Based on the object attribute identifiers in the stencil buffer, the rendering feedback prediction unit identifies and removes vector data corresponding to non-static objects in the global motion vector field to generate static vector data. Based on the depth value distribution in the depth buffer, the rendering feedback prediction unit divides the static vector data into a far-field vector set and a near-field vector set. It uses the far-field vector set to calculate the rotation component of the viewpoint pose and the near-field vector set to calculate the translation component of the viewpoint position, thus generating the second viewpoint parameters. Specifically, the rendering feedback prediction unit performs a depth weight allocation operation: it establishes a mapping relationship between each pixel in the static vector data and the depth values ​​in the depth buffer; for the translation component calculation, the rendering feedback prediction unit sets the calculation weight of the corresponding pixel to be inversely proportional to the depth value of that pixel; for the rotation component calculation, the rendering feedback prediction unit sets the calculation weight of the corresponding pixel to be directly proportional to the depth value of that pixel, thereby separating the coupled components of translation and rotation motion on the two-dimensional projection plane under monocular vision conditions. The viewport consistency arbitration unit is used to calculate the deviation between the first viewpoint parameter and the second viewpoint parameter. When the deviation is less than the preset continuity threshold, the viewport consistency arbitration unit generates the target viewpoint parameter based on the weighted calculation of the first viewpoint parameter and the second viewpoint parameter. When the deviation is greater than the continuity threshold, the viewport consistency arbitration unit performs an abnormal takeover operation and directly replaces the first viewpoint parameter with the second viewpoint parameter as the target viewpoint parameter. The real-time viewport mapping unit is used to receive target viewpoint parameters and construct a projection matrix to perform image rendering.

2. The real-time tracking system for continuous walkthrough in VR view according to claim 1, wherein, The Render Feedback Prediction Unit is also used to perform texture validity evaluation: while extracting the global motion vector field, the Render Feedback Prediction Unit calculates the grayscale histogram of the current rendered frame image and derives the texture information entropy value. The rendering feedback prediction unit compares the texture information entropy value with a preset feature richness threshold. If the texture information entropy value is lower than the feature richness threshold, the rendering feedback prediction unit generates a confidence blocking signal. In response to the confidence blocking signal, the viewport consistency arbitration unit locks the trigger state of the abnormal takeover operation to prohibit and switches to the mode of relying solely on the inertial measurement component in the multi-source viewpoint acquisition unit for integral inference.

3. The real-time tracking system for continuous walkthrough in VR view according to claim 1, wherein, Viewport consistency arbitration unit, connected to the rendering granularity controller: When the viewport consistency arbitration unit performs an exception takeover operation, the viewport consistency arbitration unit synchronously sends a degradation trigger command to the rendering granularity controller; The rendering granularity controller responds to the degradation trigger command by performing rendering parameter adjustment operations to reduce the rendering resolution or switch to a low level of detail model, thereby shortening the reverse inference time step of the global motion vector field.

4. The real-time tracking system for continuous roaming of naked-eye VR viewpoint according to claim 1, characterized in that, The viewport consistency arbitration unit performs a dynamic weighted fusion operation when the deviation value is less than the continuity threshold: the viewport consistency arbitration unit calculates a dynamic fusion factor according to the deviation value, and generates a target viewpoint parameter according to the following formula: wherein, is the target viewpoint parameter, is the first viewpoint parameter, is the second viewpoint parameter, is the dynamic fusion factor; the dynamic fusion factor is set as a monotonically increasing function of the deviation value, and the value range is limited in the closed interval of zero to one.

5. A real-time tracking system for continuous roaming of naked-eye VR perspective according to claim 1, characterized in that, The viewport consistency arbitration unit is also used to perform drift correction: after an abnormal takeover operation is triggered and the duration reaches a preset duration threshold, the viewport consistency arbitration unit performs a re-anchoring operation; the re-anchoring operation gradually reduces the weight of the second viewpoint parameter with a preset linear rate of change, and gradually increases the weight of the first viewpoint parameter, until the target viewpoint parameter is completely restored to the first viewpoint parameter.

6. A real-time tracking system for continuous roaming of naked-eye VR perspective according to claim 1, characterized in that, The multi-source viewpoint acquisition unit includes an inertial measurement unit and a visual positioning sensor: the inertial measurement unit outputs angular velocity and acceleration data at a first frequency; the visual positioning sensor outputs spatial position coordinates at a second frequency lower than the first frequency; the multi-source viewpoint acquisition unit integrates the angular velocity and acceleration data, corrects them using the spatial position coordinates, and calculates and generates the first viewpoint parameters.

7. A real-time tracking system for continuous roaming of naked-eye VR perspective according to claim 1, characterized in that, The real-time viewport mapping unit is used to perform temporal anti-aliasing: the real-time viewport mapping unit uses the second viewpoint parameters to perform pixel-level reprojection alignment between the current rendering frame and the historical rendering frames; the rendering feedback prediction unit directly reads the pixel motion data generated during the temporal anti-aliasing process as a global motion vector field.

8. A real-time tracking system for continuous roaming of naked-eye VR perspective according to claim 1, characterized in that, The system runs on a mobile terminal device, which includes a graphics processor: the depth buffer and the stencil buffer reside in the graphics processor's video memory; The rendering feedback prediction unit uses the graphics processor's computational shader to perform culling and layered solution of the global motion vector field; The viewport consistency arbitration unit runs as a logical thread independent of the rendering thread, and completes the generation of target viewpoint parameters before the vertical synchronization signal of the rendering frame arrives.

Images