A programmable visual interaction processing method for immersive visual rendering

Abstract

This invention relates to the field of virtual reality software technology, and more particularly to a programmable visual interaction processing method for immersive visual rendering. The method includes: acquiring multimodal interaction commands and environmental data through integrated sensors and generating feature vectors; using a programmable logic parsing engine to match scripts to generate a rendering control command set; subsequently performing dynamic rendering optimization of the video scene, and using an automated virtual camera algorithm and spline interpolation to achieve smooth transitions in character scene transitions; finally, synthesizing a visual stream and using an asynchronous prediction algorithm for pose feedback compensation. This invention improves the flexibility and naturalness of interaction by introducing programmable logic and intelligent scene control, and reduces latency and user dizziness by combining dynamic rendering optimization and prediction compensation, thus achieving immersive visual interaction.

Description

Technical Field

[0001] This invention relates to the field of virtual reality software technology, and in particular to a programmable visual interaction processing method for immersive visual rendering. Background Technology

[0002] With the rapid development of virtual reality and augmented reality technologies, immersive visual rendering has become a key element in the generation of digital content. However, existing technologies often rely on predefined static rendering logic when handling immersive visual interactions, lacking the ability to dynamically program and reconstruct complex interactive behaviors in real time.

[0003] Traditional rendering engines, when handling the transition between video scenes and character shots, lack semantic-level correlation analysis, resulting in abrupt scene transitions and visual stuttering, making it difficult to meet the immersive requirements of high-frequency interactions.

[0004] Meanwhile, visual processing for multi-branch interaction paths typically employs simple linear stacking, which fails to effectively resolve the conflict between computational power allocation between multi-source interaction commands and high-resolution image rendering. This results in a significant sensory delay between scene switching and user operations, severely impacting the interactive quality of the virtual environment. Summary of the Invention

[0005] The purpose of this invention is to provide a programmable visual interaction processing method for immersive visual rendering, which can effectively solve the technical problems in the background art.

[0006] This invention provides a programmable visual interaction processing method for immersive visual rendering, comprising: S1. By integrating a sensor array into the virtual reality hardware terminal, the system acquires the user's gesture trajectory, eye movement coordinates, voice commands, and pose data of the head-mounted display in real time. Simultaneously, it retrieves the light intensity, object spatial topology, and global coordinate system parameters in the current virtual scene. Using a multi-source data fusion algorithm, the system aligns the timestamps and unifies the spatial coordinates of the multimodal interaction commands to generate standardized interaction feature vectors. S2. Input the standardized interaction feature vector into the preset programmable logic parsing engine, perform real-time compilation and logic matching according to the user-defined interaction logic script, determine the type of visual event triggered by the current interaction behavior, and generate the corresponding rendering control instruction set according to the weight allocation mechanism in the logic script. S3. Based on the scene control parameters in the rendering control instruction set, the shader program of the graphics processing unit is called to dynamically reconstruct the virtual video scene, and the rendering calculation of the non-visible area is filtered by the view frustum culling and occlusion culling algorithms, and the scene basic rendering frame is output. S4. Based on the character storyboard parameters in the rendering control instruction set, the visual orientation of the current character in the virtual space is calculated using an automated virtual camera control algorithm. By adjusting the focal length, aperture, depth of field, and shooting angle of the virtual camera, a smooth transition from the environmental perspective to the character storyboard is achieved. The camera motion path is smoothed using a cubic Hermite spline interpolation algorithm. S5. Perform depth synthesis of the scene base rendering frame and the character storyboard image, add motion blur, tone mapping and anti-aliasing processing using the post-processing pipeline, generate an immersive visual rendering image sequence, and output the frame synchronously according to the refresh rate of the head-mounted display. At the same time, monitor the rendering latency data and use an asynchronous prediction algorithm to compensate for the visual pose of the next frame.

[0007] In some embodiments, in S1, the sensor array includes at least a high-precision inertial measurement unit, an infrared optical tracking camera, an eye-tracking module, and a microphone array.

[0008] In some embodiments, the inertial measurement unit captures micro-motion data of the user's head at a preset sampling frequency; The infrared optical tracking camera acquires the user's gesture trajectory by capturing infrared sensor signals; The eye-tracking module uses an infrared light source to illuminate the pupil and capture corneal reflection points to calculate eye movement coordinates. When generating standardized interaction feature vectors, a hard synchronization mechanism based on the system clock is used to add a timestamp to each frame of data, and the gesture coordinate system, eye-tracking coordinate system, and head-mounted display local coordinate system are uniformly transformed to the global world coordinate system of the virtual scene through an affine transformation matrix. The Kalman filter algorithm is used to suppress sensor noise. By establishing a state transition model and an observation model, and combining posterior state estimation, prior state prediction, Kalman gain, and sensor observations, the user's pose data is smoothed to output standardized interaction feature vectors.

[0009] In some embodiments, in S2, the programmable logic parsing engine deconstructs the standardized interaction feature vector into data fields that include at least pose change rate, gaze dwell time, and voice keyword confidence. The logic script polls and matches the data fields through preset conditional branch statements. When it is detected that the gaze point stays on a specific virtual object for a longer time than a preset time threshold and the gesture trajectory matches a preset directional action, it determines that a character storyboard adjustment event is triggered. The weighting mechanism calculates based on the urgency and visual salience of the interaction behavior, assigning priority weights to task levels for perspective changes caused by head movements.

[0010] In some embodiments, during the dynamic rendering process of the video scene in S3: For objects within a first preset distance range from the viewpoint, load a high-precision geometric model and enable anisotropic filtering at a preset multiple. For objects that are outside the second preset distance from the viewpoint, switch to a low-poly model and reduce the shading frequency; The physically based rendering lighting model includes bidirectional reflection distribution function calculation, which simulates the light and shadow interaction effect in a physical environment by comprehensively calculating metallicity, roughness, normals and environment occlusion maps; the view frustum culling filters out objects outside the field of view by judging the intersection of the object boundary sphere and the virtual camera view frustum; the occlusion culling uses depth buffering technology to cull background geometry that is completely occluded by foreground objects.

[0011] In some embodiments, in S4, the automated virtual camera control algorithm dynamically adjusts the composition ratio according to the intensity of the character's movements: When the camera detects that the virtual character is moving at high speed, it automatically increases the focal length and moves the camera position back to switch to a distant view. When a character performs a fine interactive action, the camera switches to a close-up or extreme close-up shot by calculating the center position of the character's facial feature points. During the perspective transition, the camera motion path is calculated using a cubic Hermite spline interpolation algorithm. The interpolation algorithm combines the camera's position coordinates, starting point coordinates, ending point coordinates, and velocity vectors between the starting and ending points at the current moment to generate a smooth motion trajectory. The transition time of the perspective transition is set within a preset transition time range, and the perspective movement is controlled by a nonlinear velocity curve.

[0012] In some embodiments, in S5, the post-processing pipeline employs a temporal anti-aliasing algorithm to eliminate jagged edges of objects by combining sub-pixel offset data from the current frame and historical frames; the tone mapping employs high dynamic range imaging technology to map the preset bit-width floating-point color data generated by rendering to the color space of the display device to preserve highlight and shadow details; the asynchronous prediction algorithm uses the second derivative to predict the position of the next frame based on the pose data of a preset number of frames in the past, and sets the prediction lead to a preset time lead.

[0013] In some embodiments, before display output, the image is spatially warped and offset according to the prediction results to offset the delay caused by system rendering and transmission; at the same time, frame synchronization output is performed according to the refresh rate of the head-mounted display, and rendering delay data is monitored in real time to complete the visual interaction processing closed loop.

[0014] Compared with the prior art, the present invention has the following beneficial effects: 1. This invention breaks the limitations of static interactive logic in traditional rendering systems by introducing a programmable logic parsing engine. Users can flexibly define the triggering conditions and performance forms of video scenes and character storyboards according to specific application scenario requirements. 2. By utilizing automated virtual camera control algorithms and smooth interpolation technology, this invention achieves the transition from macro scenes to micro character storyboards. By simulating a cinematic narrative perspective and combining it with human visual physiological characteristics, it effectively solves the problems of abrupt perspective switching and strong visual jumps in existing technologies. 3. This invention, combined with high-precision pose tracking and asynchronous prediction compensation, significantly reduces dizziness for users during complex interactions and improves comfort during extended wear.

[0015] 3. This invention achieves extremely low system link latency while ensuring high-fidelity visual output at a preset resolution through frustum culling and rendering resource pre-allocation strategies. It also ensures priority rendering of core visual features by optimizing the computing power allocation mechanism for complex scenes and multi-user interactions. 4. The real-time quality assessment and degradation rendering of this invention can ensure the stable operation of the system under extreme loads. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart of the programmable visual interaction processing method for immersive visual rendering according to the present invention; Figure 2 This is a diagram illustrating the dynamic rendering process of a video scene in this invention. Detailed Implementation

[0018] The following will be based on embodiments of the present invention. Figures 1-2 The technical solutions in the embodiments of the present invention will be clearly and completely described together. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0019] Application Overview Programmable visual interaction processing technology focuses on the dynamic collaboration between video scenes and character storyboards. Through flexible logic definitions, it achieves fine-grained control over the rendering pipeline. The core of this technology lies in the deep integration of preset visual materials with real-time interactive commands, aiming to build an intelligent rendering system that can automatically adjust the storyboard perspective and scene composition based on user behavior feedback. By orchestrating the visual flow in real time, this approach strives to ensure a deep logical integration of character actions and environmental feedback, thereby enhancing the coherence of visual content.

[0020] Based on the foregoing description, the following embodiments are provided for further detailed explanation.

[0021] Example 1 This embodiment provides a programmable visual interaction processing method for immersive visual rendering, specifically including the following: First, in S1, the sensor array integrated into the virtual reality hardware terminal acquires the user's gesture trajectory, eye movement coordinates, voice commands, and pose data of the head-mounted display in real time. Simultaneously, it retrieves the light intensity, object spatial topology, and global coordinate system parameters in the current virtual scene. Using a multi-source data fusion algorithm, it aligns the timestamps and unifies the spatial coordinates of the multimodal interaction commands to generate standardized interaction feature vectors.

[0022] Specifically, a sensor array integrated into the virtual reality hardware terminal acquires the user's gesture trajectory, eye coordinates, voice commands, and pose data from the head-mounted display in real time. The sensor array includes a high-precision inertial measurement unit (IMU), an infrared optical tracking camera, an eye-tracking module, and a microphone array. The IMU's sampling frequency is set within a preset frequency range, such as 1000Hz, to capture high-frequency micro-movements of the user's head. The infrared optical tracking system, through infrared supplementary lights arranged around the perimeter of the space and in conjunction with infrared sensors on the terminal, achieves centimeter-level tracking of the user's gesture trajectory, with recognition accuracy controlled within a preset error range. The eye-tracking module uses an infrared light source to illuminate the pupil and capture corneal reflection points, calculating the user's eye coordinates in real time. With a tracking error less than a preset tracking error threshold, the system determines the user's visual attention focus.

[0023] In S1 above, the illumination intensity, object spatial topology, and global coordinate system parameters of the current virtual scene are synchronously retrieved. A multi-source data fusion algorithm is used to align the timestamps and unify the spatial coordinates of multimodal interaction commands. Due to differences in the sampling frequencies of different sensors, a hard synchronization mechanism based on a high-precision system clock is adopted to assign a high-precision timestamp to each frame of collected data. During the spatial coordinate unification process, the gesture coordinate system, eye-tracking coordinate system, and head-mounted display local coordinate system are uniformly transformed to the global world coordinate system of the virtual scene using an affine transformation matrix. A Kalman filter algorithm is used to suppress sensor noise, and the user's pose data is smoothed by establishing a state transition model and an observation model. The update equation of the Kalman filter is as follows: in, X(k|k) for k Posterior state estimation at time t; X(k|k-1) Predicting prior states; K(k) For Kalman gain, Z(k) These are sensor observations; H The observed matrix is ​​used; through this filtering process, the generated standardized interactive feature vector has high stability and accuracy.

[0024] Next, in S2, the standardized interaction feature vector is input into the preset programmable logic parsing engine, which performs real-time compilation and logic matching based on the user-defined interaction logic script to determine the type of visual event triggered by the current interaction behavior, and generates the corresponding rendering control instruction set according to the weight allocation mechanism in the logic script. Specifically, after receiving the standardized interaction feature vector, the programmable logic parsing engine first deconstructs it into multiple data fields, including pose change rate, gaze duration, and confidence of speech keywords. The logic script polls and matches these fields using preset conditional branch statements. For example, when it detects that the user's gaze lingers on a virtual character's face for more than a preset duration threshold, and the gesture trajectory shows a directional movement, the script parsing engine determines that a "character scene adjustment event" is triggered. The weight allocation mechanism calculates based on the urgency and visual salience of the interaction behavior. For core visual events that affect user immersion, such as changes in perspective caused by head movements, a priority weight greater than a preset weight threshold is assigned. When computing power is limited, rendering tasks are managed through a priority queue, prioritizing the rendering quality of high-weight visual flows. The parsing latency of the logic script is strictly controlled within a preset latency threshold to ensure real-time interaction responses.

[0025] Next, in S3, based on the scene control parameters in the rendering control instruction set, the shader program of the graphics processing unit is called to dynamically reconstruct the virtual video scene, and the view frustum culling and occlusion culling algorithms are used to filter the rendering calculation of non-visible areas, and output the scene base rendering frame.

[0026] Specifically, a multi-level hierarchical detailing technique is employed to manage complex geometric models. For core objects within a first preset distance from the viewpoint, a high-precision geometric model is loaded, and anisotropic filtering at a preset multiple is enabled to eliminate texture blurring when viewed from an angle. For background objects beyond a second preset distance from the viewpoint, a low-poly model is automatically switched, and the shading frequency is reduced. In terms of lighting calculations, the physically based rendering lighting model includes bidirectional reflection distribution function calculations. Through comprehensive calculations of metallicity, roughness, normals, and environment occlusion maps, it simulates the light and shadow interaction effects in a realistic physical environment.

[0027] To optimize performance, frustum culling and occlusion culling algorithms are used. Frustum culling filters out objects outside the field of view by judging the intersection of the object boundary sphere and the virtual camera's frustum. Occlusion culling uses depth buffering technology to remove background geometry that is completely occluded by foreground objects, thereby reducing invalid pixel shading calculations and outputting high-fidelity scene base rendering frames.

[0028] Then, in S4, based on the character storyboard parameters in the rendering control instruction set, the automated virtual camera control algorithm is used to calculate the current visual position of the character in the virtual space. By adjusting the focal length, aperture, depth of field, and shooting angle of the virtual camera, a smooth transition from the environmental perspective to the character storyboard is achieved, and the camera motion path is smoothed using a cubic Hermite spline interpolation algorithm.

[0029] Specifically, in real-time character shot arrangement and perspective transitions, based on the character shot parameters in the rendering control instruction set, an automated virtual camera control algorithm calculates the optimal visual position of the current character in virtual space. This algorithm follows cinematic narrative logic, automatically adjusting the composition ratio according to the intensity of the character's movements. When a virtual character is detected to be performing high-speed movement or combat actions, the automated virtual camera control algorithm automatically increases the focal length and moves the camera position back, switching to a long-range perspective to provide a wider field of view. When a character is engaged in dialogue or subtle interactive actions, the algorithm automatically switches to a close-up or extreme close-up shot by calculating the center position of the character's facial feature points.

[0030] Finally, in S5, the scene base rendering frames and character storyboard images are deeply composited. Motion blur, tone mapping, and anti-aliasing are added using the post-processing pipeline to generate an immersive visual rendering image sequence. The frames are output synchronously according to the refresh rate of the head-mounted display. At the same time, rendering latency data is monitored, and the visual pose of the next frame is compensated using an asynchronous prediction algorithm.

[0031] Specifically, the post-processing pipeline employs a temporal anti-aliasing algorithm, which eliminates jagged edges and flickering by combining sub-pixel offset data from the current frame and historical frames. The tone mapping process uses high dynamic range imaging technology to map the rendered preset bit-width floating-point color data to the color space supported by the display device, ensuring rich details are preserved even in extremely bright or dark environments.

[0032] The system outputs frames synchronously based on the refresh rate of the head-mounted display and monitors rendering latency data in real time. When the rendering latency approaches a preset threshold, an asynchronous prediction algorithm is used to compensate for the visual pose of the next frame. This asynchronous prediction algorithm uses the second derivative of the user's past pose data (a preset number of frames) to predict the possible position of the next frame, with the prediction lead set to a preset time lead. This compensation mechanism effectively counteracts the latency caused by system rendering and transmission by making a slight spatial distortion shift to the image before display output, significantly reducing the user's dizziness.

[0033] It should be noted that, at the hardware implementation level, the method of this invention is applied to a high-performance virtual reality workstation, configured with multiple graphics processors with preset memory capacities, supporting hardware-level ray tracing acceleration. The total memory capacity is not less than a preset capacity threshold, and the data transmission bandwidth is not less than a preset bandwidth threshold. When processing immersive visual streams at preset resolutions and frame rates, load balancing optimization of the PCIe bus ensures efficient data exchange between the graphics processor and the central processing unit, making the overall system link latency less than or equal to a preset link latency threshold.

[0034] Example 2 This embodiment, based on Embodiment 1, describes a visual rendering scheme for social interaction in a virtual reality environment. The focus of this scheme is on the detailed reproduction of human expressions and movements.

[0035] In S1, a facial camera was added to capture the user's facial expression feature points. The standardized interaction feature vector includes expression encoding based on action units.

[0036] In S2, the logic script is responsible for parsing social intent. For example, when two virtual avatars are close to each other and make eye contact, a social scene mode is triggered.

[0037] In S3, the rendering engine uses high-precision skin shaders and hair rendering technology, and utilizes hardware-accelerated ray tracing technology to calculate environmental occlusion and mutual reflection between avatars in real time, making the visual presentation in social scenes more realistic.

[0038] In S4, the automated virtual camera control algorithm employs classic cinematic dialogue framing (such as over-the-shoulder shots and medium shots of two people). It dynamically adjusts the pace of shot transitions based on the volume and speed of the speakers' voices. When one party is speaking intensely, the camera automatically zooms in for a close-up; when both parties are silent, the camera slowly zooms out to show the surrounding atmosphere.

[0039] In S5, to eliminate the uncanny valley effect in social interactions, a deep learning model is used to fine-tune the collected raw facial expression data, making it appear more natural in the virtual form. An asynchronous prediction algorithm ensures that the lip-sync latency of the speakers is less than a preset threshold, guaranteeing a sense of immersion in social interactions.

[0040] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered illustrative and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the scope of the invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

[0041] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A programmable visual interaction processing method for immersive visual rendering, characterized in that, include: S1. By integrating a sensor array into the virtual reality hardware terminal, the system acquires the user's gesture trajectory, eye movement coordinates, voice commands, and pose data of the head-mounted display in real time. Simultaneously, it retrieves the light intensity, object spatial topology, and global coordinate system parameters in the current virtual scene. Using a multi-source data fusion algorithm, the system aligns the timestamps and unifies the spatial coordinates of the multimodal interaction commands to generate standardized interaction feature vectors. S2. Input the standardized interaction feature vector into the preset programmable logic parsing engine, perform real-time compilation and logic matching according to the user-defined interaction logic script, determine the type of visual event triggered by the current interaction behavior, and generate the corresponding rendering control instruction set according to the weight allocation mechanism in the logic script. S3. Based on the scene control parameters in the rendering control instruction set, the shader program of the graphics processing unit is called to dynamically reconstruct the virtual video scene, and the rendering calculation of the non-visible area is filtered by the view frustum culling and occlusion culling algorithms, and the scene basic rendering frame is output. S4. Based on the character storyboard parameters in the rendering control instruction set, the visual orientation of the current character in the virtual space is calculated using an automated virtual camera control algorithm. By adjusting the focal length, aperture, depth of field, and shooting angle of the virtual camera, a smooth transition from the environmental perspective to the character storyboard is achieved. The camera motion path is smoothed using a cubic Hermite spline interpolation algorithm. S5. Perform depth synthesis of the scene base rendering frame and the character storyboard image, add motion blur, tone mapping and anti-aliasing processing using the post-processing pipeline, generate an immersive visual rendering image sequence, and output the frame synchronously according to the refresh rate of the head-mounted display. At the same time, monitor the rendering latency data and use an asynchronous prediction algorithm to compensate for the visual pose of the next frame.

2. The method according to claim 1, characterized in that, In S1, the sensor array includes at least a high-precision inertial measurement unit, an infrared optical tracking camera, an eye-tracking module, and a microphone array.

3. The method according to claim 2, characterized in that, The inertial measurement unit captures the micro-motion data of the user's head at a preset sampling frequency; The infrared optical tracking camera acquires the user's gesture trajectory by capturing infrared sensor signals; The eye-tracking module uses an infrared light source to illuminate the pupil and capture corneal reflection points to calculate eye movement coordinates. When generating standardized interaction feature vectors, a hard synchronization mechanism based on the system clock is used to add a timestamp to each frame of data, and the gesture coordinate system, eye-tracking coordinate system, and head-mounted display local coordinate system are uniformly transformed to the global world coordinate system of the virtual scene through an affine transformation matrix. The Kalman filter algorithm is used to suppress sensor noise. By establishing a state transition model and an observation model, and combining posterior state estimation, prior state prediction, Kalman gain, and sensor observations, the user's pose data is smoothed to output standardized interaction feature vectors.

4. The method according to claim 1, characterized in that, In S2, the programmable logic parsing engine deconstructs the standardized interaction feature vector into data fields that include at least pose change rate, gaze dwell time, and speech keyword confidence. The logic script polls and matches the data fields through preset conditional branch statements. When it is detected that the gaze point stays on a specific virtual object for a longer time than a preset time threshold and the gesture trajectory matches a preset directional action, it determines that a character storyboard adjustment event is triggered. The weighting mechanism calculates based on the urgency and visual salience of the interaction behavior, assigning priority weights to task levels for perspective changes caused by head movements.

5. The method according to claim 1, characterized in that, In S3, during the dynamic rendering process of the video scene: For objects within a first preset distance range from the viewpoint, load a high-precision geometric model and enable anisotropic filtering at a preset multiple. For objects that are outside the second preset distance from the viewpoint, switch to a low-poly model and reduce the shading frequency; The physically based rendering lighting model includes bidirectional reflection distribution function calculation, which simulates the light and shadow interaction effects in a physical environment by comprehensively calculating metallicity, roughness, normals, and environment occlusion maps; the view frustum culling filters out objects outside the field of view by judging the intersection of the object boundary sphere and the virtual camera view frustum. The occlusion culling technique uses depth buffering to remove background geometry that is completely occluded by foreground objects.

6. The method according to claim 1, characterized in that, In S4, the automated virtual camera control algorithm dynamically adjusts the composition ratio based on the intensity of the character's movements: When the camera detects that the virtual character is moving at high speed, it automatically increases the focal length and moves the camera position back to switch to a distant view. When a character performs a fine interactive action, the camera switches to a close-up or extreme close-up shot by calculating the center position of the character's facial feature points. During the perspective transition, the camera motion path is calculated using a cubic Hermite spline interpolation algorithm. The interpolation algorithm combines the camera's position coordinates, starting point coordinates, ending point coordinates, and velocity vectors between the starting and ending points at the current moment to generate a smooth motion trajectory. The transition time of the perspective transition is set within a preset transition time range, and the perspective movement is controlled by a nonlinear velocity curve.

7. The method according to claim 1, characterized in that, In S5, the post-processing pipeline employs a temporal anti-aliasing algorithm, which eliminates the jagged edges of objects by combining sub-pixel offset data from the current frame and historical frames; the tone mapping uses high dynamic range imaging technology to map the preset bit-width floating-point color data generated by rendering to the color space of the display device to preserve highlight and shadow details; the asynchronous prediction algorithm uses the second derivative to predict the position of the next frame based on the pose data of a preset number of frames in the past, and sets the prediction lead to a preset time lead.

8. The method according to claim 7, characterized in that, Before the display output, the image is spatially warped and offset according to the prediction results to offset the delay caused by system rendering and transmission; at the same time, the frame is output synchronously according to the refresh rate of the head-mounted display, and the rendering delay data is monitored in real time to complete the visual interaction processing closed loop.

Images