Three-dimensional virtual simulation stereoscopic rendering video generation method and device, equipment and medium

Abstract

The invention relates to a three-dimensional virtual simulation stereoscopic rendering video generation method and device, equipment and a medium, and aims to adapt to the eye distance or scene depth of a user by dynamically adjusting the distance between a left virtual camera assembly and a right virtual camera assembly, so that the visual comfort of a stereoscopic video is remarkably improved. And meanwhile, a high-precision picture synchronization technology is adopted, so that the picture time sequence consistency of the left and right virtual camera assemblies is ensured, the picture tearing and delay phenomena are avoided, and the immersion feeling is improved. Through path smooth interpolation and real-time adjustment technologies, a continuous and smooth camera motion track is generated, and creation flexibility is enhanced. In addition, the application of the parallax perception compression algorithm optimizes the code rate and the image quality of the video file, so that the video generation efficiency is higher and the file size is smaller. The introduction of the visual editing panel supports the user to adjust and preview the camera spacing, the mirror moving track and the compression parameters in real time, avoids frequent re-rendering, and significantly improves the working efficiency.

Description

Methods, apparatus, equipment and media for generating 3D virtual simulation stereoscopic rendering videos Technical Field

[0001] This invention belongs to the field of computer graphics and virtual reality technology, and specifically relates to a method, apparatus, device and medium for generating three-dimensional virtual simulation stereoscopic rendering videos. Background Technology

[0002] 3D virtual simulation stereoscopic rendering video generation technology is an important research direction in the fields of virtual reality and computer graphics, and is widely used in film and television production, industrial simulation, medical training, and other fields. However, traditional stereoscopic video generation methods have the following core technical defects: First, the distance between the left and right virtual cameras (eye distance) is fixed and cannot be dynamically adjusted according to the user's physiological characteristics (such as different age groups) or scene depth (switching between close-up and distant views), resulting in distorted 3D effects or visual fatigue; Second, the rendering sequence of the left and right eye images relies on manual calibration, which easily leads to screen tearing or delay in moving scenes, seriously damaging the sense of immersion; Third, it only supports preset camera trajectories and cannot adjust the camera path in real time or integrate manual / automatic control, limiting creative flexibility; Fourth, it is difficult to modify camera parameters or camera trajectories after the video is generated, requiring the entire sequence to be re-rendered, which is inefficient. Summary of the Invention

[0003] To address the aforementioned problems, this invention proposes a three-dimensional virtual simulation stereoscopic rendering video generation method, apparatus, equipment, and medium to solve the problems of poor comfort, low synchronization accuracy, insufficient flexibility, and low efficiency in traditional stereoscopic video generation.

[0004] To achieve the above objectives, the technical solution of the present invention is as follows: On one hand, a method for generating three-dimensional virtual simulation stereoscopic rendering video is provided, comprising the following steps: constructing a target three-dimensional scene using a simulation engine; defining two camera movement nodes based on the left and right eye positions; constructing left and right virtual camera components at the two camera movement nodes; adjusting the spacing between the left and right virtual camera components to match the user's eye distance or scene depth; setting the left and right virtual camera components to construct a projection matrix for the same target object; setting binocular viewport offsets; associating the left and right virtual camera components with rendering textures; binding the left and right virtual camera components to the same time base to ensure consistent frame timing and pre-rendering future multiple frames; defining manually / automatically fused camera movement trajectories and generating continuous and smooth camera movement trajectories through path smoothing interpolation and real-time adjustment; determining a disparity map for the left and right eye images output by the left and right virtual camera components based on the continuous and smooth camera movement trajectory; performing regional compression on the left and right eye images according to the disparity map to generate a standard stereoscopic video file; setting a visual editing panel to adjust the camera spacing, movement trajectory, or compression parameters of the generated standard stereoscopic video file, and previewing the adjusted standard stereoscopic video file in real time.

[0005] On the other hand, a three-dimensional virtual simulation stereoscopic rendering video generation device is provided. The device includes: a three-dimensional scene construction and camera configuration module, used to construct a target three-dimensional scene using a simulation engine, define two camera movement nodes based on the left and right eye positions, construct left and right virtual camera components at the two camera movement nodes, and adjust the spacing between the left and right virtual camera components to match the user's eye distance or scene depth; a screen projection and viewport setting module, used to set the left and right virtual camera components to construct a screen projection matrix for the same target object, set binocular viewport offsets, and associate the left and right virtual camera components with rendering textures; and a time synchronization and frame pre-rendering module, used to bind the left and right virtual camera components to the same time base to ensure that the left and right virtual... The camera components display consistent frame sequences and pre-render multiple future frames. A camera movement trajectory processing module defines manually / automatically fused camera movement trajectories and generates continuous, smooth camera movement trajectories through path smoothing interpolation and real-time adjustment. A disparity map generation and compression optimization module determines the disparity map for the left and right eye images output by the left and right virtual camera components based on the continuous, smooth camera movement trajectory, and performs regional compression of the left and right eye images according to the disparity map to generate a standard stereoscopic video file. A visualization editing and parameter adjustment module sets up a visualization editing panel to adjust camera spacing, movement trajectory, or compression parameters of the generated standard stereoscopic video file, and provides real-time preview of the adjusted standard stereoscopic video file.

[0006] In another aspect, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of a three-dimensional virtual simulation stereoscopic rendering video generation method.

[0007] In another aspect, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps of a method for generating a three-dimensional virtual simulation stereoscopic rendering video.

[0008] This invention significantly improves the visual comfort of stereoscopic videos by dynamically adjusting the spacing between the left and right virtual camera components to adapt to the user's eye distance or scene depth. Simultaneously, high-precision image synchronization technology ensures consistent image timing between the left and right virtual camera components, avoiding image tearing and latency, and enhancing immersion. Through path smoothing interpolation and real-time adjustment technology, continuous and smooth camera motion trajectories are generated, enhancing creative flexibility. Furthermore, the application of a parallax-aware compression algorithm optimizes the video file's bitrate and image quality, resulting in higher video generation efficiency and smaller file size. The introduction of a visual editing panel allows users to adjust and preview camera spacing, camera movement trajectory, and compression parameters in real time, avoiding frequent re-rendering and significantly improving work efficiency. Attached Figure Description

[0009] Figure 1 is a flowchart of a three-dimensional virtual simulation stereoscopic rendering video generation method according to the present invention; Figure 2 is a flowchart of the implementation of constructing a three-dimensional virtual environment and camera system according to the present invention; Figure 3 is a flowchart of the implementation of constructing a screen projection matrix and associating it with rendering textures according to the present invention; Figure 4 is a flowchart of the implementation of high-precision frame synchronization and double buffering mechanism according to the present invention; Figure 5 is a flowchart of the implementation of defining a rendering path for manual / automatic fusion according to the present invention; Figure 6 is a flowchart of the implementation of parallax perception compression and storage optimization according to the present invention; Figure 7 is a flowchart of the implementation of adjustable post-processing parameters and real-time preview according to the present invention; Figure 8 is a window diagram of camera parameter settings according to the present invention; Figure 9 is a diagram of camera layout according to the present invention; Figure 10 is a schematic diagram of projection matrix according to the present invention; Figure 11 is a schematic diagram of double buffering mechanism according to the present invention; Figure 12 is a flowchart of an example operation step of the three-dimensional virtual simulation stereoscopic rendering video generation method according to the present invention; Figure 13 is a flowchart of another three-dimensional virtual simulation stereoscopic rendering video generation method according to the present invention; Figure 14 is a structural block diagram of a three-dimensional virtual simulation stereoscopic rendering video generation device according to the present invention; Figure 15 is an internal structural diagram of the computer device according to the present invention. Detailed Implementation

[0010] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0011] As shown in Figure 1, Embodiment 1 of this application provides a method for generating three-dimensional virtual simulation stereoscopic rendering video, including steps S1-S6.

[0012] Step S1) Construct a 3D virtual environment and camera system. Use a simulation engine (such as Unity3D / Unreal Engine) to construct the target 3D scene, load the model, materials and lighting parameters; define the camera movement node, construct left and right virtual camera components in its child nodes, and dynamically adjust the camera spacing to match the user's eye distance or scene depth, as shown in Figure 2.

[0013] In this embodiment, the key sub-steps in step S1 are operated as follows: Step S11) Construct a three-dimensional scene by dynamically importing external three-dimensional model resources in FBX format through the simulation engine, and creating a DirectionalLight in the scene to simulate sunlight.

[0014] Step S12) Configure the camera movement node: Create an empty GameObject at the origin of the scene world coordinate system in the simulation engine as the root node of the camera system. Create two child empty GameObjects under the CameraRig_Master node, named LeftEye_Camera (left eye camera node) and RightEye_Camera (right eye camera node) as the left and right screen rendering channels, respectively. These two nodes define the relative position origin of the left and right eye cameras. Attach the simulation engine's camera component to each child node.

[0015] Step S13) Dynamic eye distance adjustment uses real-time facial feature tracking technology to detect the user's interpupillary distance in real time and directly map it to the horizontal X-axis offset of the left and right eye camera nodes (LeftEye_Camera / RightEye_Camera) relative to their parent node CameraRig_Master.

[0016] Step S14) Adapting Scene Depth: The depthTextureMode function under the Camera component of the engine is called to generate a depth cache, obtain the average depth value of objects in the focus area, identify the visual focus area in the scene, and dynamically adjust the stereo camera spacing according to the distance of the main object (reducing to 55mm for near objects and increasing to 75mm for distant objects). When the focus object is close (small depth value), the stereo camera baseline is increased to enhance the prominence and depth of near objects. When the focus object is far away (large depth value), the stereo camera baseline is decreased to avoid discomfort or difficulty in blending distant objects due to excessive parallax, while maintaining the overall stereoscopic layering. Specific parameters are shown in Table 1.

[0017] Table 1 Depth Value Matching Camera Focal Length Table Step S2) Construct the image projection matrix and associate it with the rendering texture. Construct the image projection matrix for the same target object, define the viewport range (e.g., left eye offset -0.033m, right eye offset +0.033m) and perspective parameters (FOV, near and far clipping planes), and bind the outputs of the left and right virtual cameras to the rendering texture component respectively to realize the real-time association between the rendering texture and the camera, as shown in Figure 3.

[0018] In this embodiment, the key sub-steps in step S2 are operated as follows: Step S21) Define the projection matrix parameters. Based on the standard symmetric projection matrix, introduce horizontal viewport offsets for the left and right eye cameras respectively. The viewport offset of the left eye camera is... m, the negative sign indicates a leftward offset. The viewport offset for the right eye camera is... m, with a plus sign indicating a rightward offset. The simulation engine automatically calculates and generates dedicated perspective projection matrices for the left and right eyes based on the configured viewport offset, perspective parameters (see step S22), and rendering target size, and transforms the vertex coordinates in the 3D scene to the homogeneous clipping space of the 2D screen.

[0019] Step S22) Configure perspective parameters to set consistent perspective imaging rules for both left and right eye cameras, ensuring matching of the left and right eye images in geometric projection, including field of view, clipping plane, and projection mode. Define the vertical field of view (FOV) as the main parameter. For a 16:9 image, a 60° vertical FOV corresponds to a 78° horizontal FOV. Keep the FOVs of the left and right eyes completely consistent. Define the camera angle, near clipping plane distance of 0.1m, and far clipping plane distance of 1000m. Set the camera near clipping plane distance ncp=0.1m and far clipping plane distance fcp=1000m, and set the camera to perspective mode, as shown in Figure 8. Simultaneously, ensure that the camera's near clipping plane remains consistent, as shown in Figure 9.

[0020] At this point, the coordinates of the lower left point of the near clipping plane are LowerLeft, the lower right point of the near clipping plane are LowerRight, and the upper left point of the near clipping plane are UpperLeft, as shown in Figure 10. Vector vu is a unit vector pointing from UpperLeft to LowerLeft, and vector vr is a unit vector pointing from LowerRight to LowerLeft. Vector va points from LowerLeft to the camera, vector vb points from LowerRight to the camera, and vector vc points from UpperLeft to the camera.

[0021] At this point, the angle between the vectors can be calculated using the dot product and cross product of the vectors: Where nod is the scaling factor; d is the focal length; and l, r, b, and t are parameters for conveniently representing the projection of the clipping plane onto the screen. The matrix M of the clipping plane projected onto the standard cube is then: Where n is the distance from the near cut plane to the camera, and f is the distance from the far cut plane to the camera.

[0022] By dynamically adjusting the parameters LowerLeft, LowerRight, UpperLeft, and FOV, the rendered image can be adaptively optimized according to the screen size.

[0023] Step S23) Binding Render Textures: Create two RenderTexture components, LeftEye_RT and RightEye_RT, in the engine. Set the LeftEye_RT resource as the output target of the LeftEye_Camera component, and set the RightEye_RT resource as the output target of the RightEye_Camera component. After binding, whenever the engine renders a frame, the view rendering result of LeftEye_Camera (including color, depth, and other information) will be written to the LeftEye_RT texture in real time, and the same applies to RightEye_Camera.

[0024] Step S3) Implement synchronized rendering for left and right eyes and a double buffering mechanism. Use hardware to generate a global time signal to ensure that the timing of rendering, compression and output of left and right eye images is consistent throughout the entire process; integrate a double buffering mechanism to pre-render multiple future frames to avoid screen tearing in motion scenes; as shown in Figure 4.

[0025] In this embodiment, the key sub-steps in step S3 are operated as follows: Step S31) The left and right eye synchronous rendering generates a global timestamp based on the hardware clock. The left and right eye rendering pipelines are bound to the same time base, forcing the left and right eye images to complete rendering and compression within the same time window, ensuring consistent timing and a synchronization delay of less than 1ms.

[0026] Step S32) Integrating a double-buffering mechanism: This embodiment assumes that the brightness of objects in the image remains constant between consecutive frames, meaning that the RGBa of the same point will not change drastically in the next frame. The common motion vector of all points within the window is solved using the least squares method. The motion vector records the direction and distance of movement of each pixel block in the image from the previous frame to the current frame.

[0027] As shown in Figure 11, the pre-rendering buffer capacity is set to accommodate multiple future frames to avoid real-time rendering pressure. The system runs the algorithm on five consecutive frames to generate an optical flow field of the same size as the image. By analyzing this optical flow field, the possible positions of objects in subsequent frames can be inferred. After the current buffer queue is finished displaying, the system immediately switches to the pre-rendering frames in the back buffer queue, while simultaneously clearing the front buffer queue and receiving new rendering tasks. The rendering thread and the display thread run independently, achieving seamless transition between cached frames.

[0028] Step S4) Define the rendering path for manual / automatic blending. Manually or automatically define the camera movement trajectory, supporting smooth path interpolation and real-time adjustment; as shown in Figure 5.

[0029] In this embodiment, the key sub-steps in step S4 are as follows: Step S41) Manually adjust the camera movement trajectory. This provides a manual camera movement trajectory editing function, allowing creators to directly control the virtual camera in 3D space. The camera movement trajectory can be modified in real time via the engine timeline or by dragging camera nodes. Users can freely move the virtual camera in the 3D viewport, and the system automatically records its spatial position (X, Y, Z coordinates) and orientation as keyframes on the timeline. The timeline parameter settings are shown in Table 2.

[0030] Table 2 Timeline Parameters Step S42) Automatic path smoothing interpolation achieves a smooth and natural transition in the camera motion trajectory. It automates the processing of keyframe paths, whether manually set or algorithmically generated, effectively eliminating abrupt changes during motion. A smoothing component, such as Cinemachine, is used to generate transitional trajectories, avoiding sudden motion changes. For position smoothing, a B-spline bicubic spline interpolation algorithm is used to fit key points; for attitude smoothing, a quaternion-based spherical linear interpolation method achieves continuous azimuth transitions; simultaneously, Bézier curves are introduced to control motion speed. Through the combination of multi-dimensional interpolation and curve control, a continuous and smooth camera motion trajectory is ultimately generated.

[0031] Step S5) Perform disparity-aware compression and storage optimization. Based on the disparity map, the left and right eye images are compressed in different regions. The common region (background) is compressed using MPEG-4 (compression ratio 20:1), and the different region (foreground) is compressed using H.264 intra-frame prediction (compression ratio 10:1). That is, the compression ratio of the common region is 20:1 and the compression ratio of the different region is 10:1, reducing the overall bitrate by 30%-50%. Define the storage path and video parameters (resolution, frame rate, encoding format) to generate a standard stereoscopic video file, as shown in Figure 6.

[0032] In this embodiment, the key sub-steps in step S5 are as follows: Step S51) Generate a disparity map. Based on the left and right eye images after stereo correction and alignment in step S4, calculate the disparity value of each pixel. The generated disparity map is a grayscale image, and its pixel brightness value directly reflects the horizontal position offset of the corresponding scene point in the left and right views, used to distinguish the foreground and background in the image. Areas with large disparity usually correspond to nearby foreground objects, while areas with small or zero disparity correspond to distant background or overlapping parts.

[0033] Step S52) Regional Compression Strategy: Based on the disparity map generated in step S51, the system divides the left and right view images into two key regions: a common region (background) and a difference region (foreground). For the large common region marked as background (low disparity) in the disparity map, the system uses MPEG-4 encoding. This encoding excels at utilizing inter-frame redundancy, providing high compression efficiency, with a target compression ratio set at 20:1, significantly reducing the bitrate of background data while maintaining basic image quality. For the difference region marked as foreground (high disparity) in the disparity map, containing important stereoscopic information or moving objects, the system uses H.264 intra-frame encoding. This encoding only utilizes the spatial redundancy within the current frame for compression, avoiding the accumulation of errors that may be caused by inter-frame prediction, and better preserving the details and edge sharpness of foreground objects, ensuring the core quality of the stereoscopic viewing experience. Its target compression ratio is set at 10:1.

[0034] Step S53) Define Video Parameters: Before compression or when generating the final file, the system needs to clearly define and record the key parameters of the stereoscopic video. Define the storage path and video parameters (resolution, frame rate, encoding format) to generate a standard stereoscopic video file. Based on the parameters defined above, the system will finally output a standardized stereoscopic video file for easy storage, transmission, and subsequent playback. Set the output resolution (720P-4K), frame rate (30fps-60fps), and encoding format (H.264 / H.265).

[0035] Step S6) Enables adjustable post-processing parameters and real-time preview. Provides a visual editing panel that supports adjusting camera spacing, camera movement trajectory, or compression parameters after generation; real-time preview of the adjustment effect avoids repeatedly rendering the entire video sequence, as shown in Figure 7.

[0036] In this embodiment, the key sub-steps in step S6 are as follows: Step S61) Develop a visual adjustment panel. Integrate sliders, buttons, and other controls into the engine editor to display the current camera spacing, camera movement trajectory, and compression parameters; design and implement a user-friendly graphical user interface panel as the core interactive entry point for post-adjustment. It supports adjustments to the following parameters: camera spacing, adjusting camera movement trajectory, and editing compression parameter adjustments.

[0037] Step S62) Real-time Preview and Export: The preview window instantly displays the stereoscopic video effect after parameter adjustments. The preview supports regular playback controls (play, pause, frame-by-frame) and stereoscopic display mode. Once the user is satisfied with the preview effect and confirms the final parameters, an export function is provided. A preview screen is generated in real time after parameter adjustments, and the modified video file can be exported, avoiding the need to re-render the entire sequence.

[0038] The embodiment discloses a method for generating stereoscopic rendering videos in three-dimensional virtual simulation. As shown in Figure 12, taking the Unity three-dimensional virtual simulation engine as an example, the left and right eye virtual camera components are configured to render the images from the corresponding viewpoints, and finally the stereoscopic format video is encoded and generated. The method includes the following steps: Simulation initialization stage: When the simulation process starts, the Unity three-dimensional engine environment is initialized, external three-dimensional model resources (FBX / OBJ / glTF format) are imported, and mesh topology optimization is performed on the model.

[0039] Camera system configuration phase: Create an empty object CameraRig_Master at the origin of the scene coordinate system, and attach child nodes LeftEye_Camera and RightEye_Camera to it. Bind the Unity Camera component to the child nodes, set the perspective projection mode, unify the vertical field of view (FOV) to 60°, configure the rendering target channel independently, and set the X-axis offset based on the interpupillary distance.

[0040] Rendering texture binding stage: Generate two RenderTexture objects, LeftEye_RT and RightEye_RT, and bind the textures to the corresponding camera component output ports to realize real-time rendering data stream.

[0041] Camera trajectory generation stage: Camera keyframe nodes in the 3D viewport are dragged and dropped directly in Unity Timeline to adjust spatial position and rotation angle in real time; then the path is automatically interpolated and fitted by an algorithm.

[0042] Stereoscopic video export stage: Set basic parameters: resolution (3840×1080), frame rate (30fps), encoding format (H.265 / HEVC), and export the left and right eye video streams independently.

[0043] Post-production compositing and compression stage: Analyze the disparity maps of the left and right eye videos, segment the common background area and the different foreground area, compress the area by region and then encapsulate it into a standard stereoscopic video, and store it to a specified path.

[0044] Example 2 includes all the technical features of Example 1.

[0045] As shown in Figure 13, a three-dimensional virtual simulation stereoscopic rendering video generation method is provided in Embodiment 2, including the following steps: Step S10: Construct a target three-dimensional scene using a simulation engine, define two camera movement nodes according to the left and right eye positions, construct left and right virtual camera components at the two camera movement nodes, and adjust the spacing between the left and right virtual camera components to match the user's eye distance or scene depth; Step S20: Set the left and right virtual camera components to construct a screen projection matrix for the same target object, set the binocular viewport offset, and associate the left and right virtual camera components with the rendering texture; Step S30: Bind the left and right virtual camera components to the same time base to ensure that the left and right virtual camera components are aligned with the same time base. The right virtual camera component displays images in the same time sequence and pre-renders multiple future frames; Step S40: Define a manual / automatic camera movement trajectory and generate a continuous and smooth camera movement trajectory through path smoothing interpolation and real-time adjustment; Step S50: Determine the disparity map of the left and right eye images output by the left and right virtual camera components based on the continuous and smooth camera movement trajectory, and compress the left and right eye images by region according to the disparity map to generate a standard stereoscopic video file; Step S60: Set a visual editing panel to adjust the camera spacing, camera movement trajectory, or compression parameters of the generated standard stereoscopic video file, and preview the adjusted standard stereoscopic video file in real time.

[0046] This application significantly improves the visual comfort of stereoscopic videos by dynamically adjusting the spacing between the left and right virtual camera components to adapt to the user's eye distance or scene depth. Simultaneously, high-precision image synchronization technology ensures consistent image timing between the left and right virtual camera components, avoiding screen tearing and latency, and enhancing immersion. Through path smoothing interpolation and real-time adjustment technology, continuous and smooth camera motion trajectories are generated, enhancing creative flexibility. Furthermore, the application of a parallax-aware compression algorithm optimizes the video file's bitrate and image quality, resulting in higher video generation efficiency and smaller file size. The introduction of a visual editing panel allows users to adjust and preview camera spacing, camera movement trajectory, and compression parameters in real time, avoiding frequent re-rendering and significantly improving work efficiency.

[0047] Among them, through the deep integration of 3D simulation technology and active stereoscopic technology, dynamic eye distance adaptive technology is used to match the user's physiological characteristics or scene depth, combined with high-precision frame synchronization protocol and double buffer mechanism to eliminate screen tearing, support for manual / automatic fusion rendering path definition to improve creative flexibility, and parallax perception compression algorithm is used to optimize the balance between bit rate and image quality, and finally the cost of repeated rendering is reduced through post-processing parameter adjustment function.

[0048] This invention's method for generating 3D virtual simulation stereoscopic rendering videos achieves temporal consistency between the left and right eye views and optimizes stereoscopic comfort through dynamic eye distance adaptation, high-precision frame synchronization, and a double-buffered rendering mechanism. Combining manual / automatic camera movement trajectories and smooth interpolation algorithms enhances the coherence of camera movement and creative flexibility. The introduction of a parallax-aware compression algorithm improves compression efficiency while maintaining image quality. A visual editing panel supports real-time adjustment and preview of camera spacing, camera movement trajectories, and compression parameters, significantly reducing the cost of repetitive rendering. This method effectively solves the problems of poor comfort, low synchronization accuracy, insufficient flexibility, and low efficiency in existing technologies, and is suitable for stereoscopic vision applications such as film and television production, industrial simulation, and virtual reality.

[0049] In this embodiment, the step of constructing the target 3D scene using a simulation engine, defining two camera movement nodes based on the left and right eye positions, constructing left and right virtual camera components on the two camera movement nodes, and adjusting the spacing of the left and right virtual camera components to match the user's interpupillary distance or scene depth includes: importing an external 3D model into the simulation engine to construct the target 3D scene, loading the model, materials, and lighting parameters; creating an empty object in the simulation engine as a camera movement node, defining two camera movement nodes based on the left and right eye positions, adding left and right virtual camera components to the camera movement nodes; using real-time facial feature tracking to detect the user's interpupillary distance in real time and dynamically adjusting the spacing of the left and right virtual camera components; and adjusting the spacing of the left and right virtual camera components according to the distance to the main object in the scene.

[0050] In this embodiment, setting the left and right virtual camera components to construct a projection matrix for the same target object, setting the binocular viewport offset, and associating the left and right virtual camera components with the rendering texture includes: setting the left and right virtual camera components to construct a projection matrix for the same target object, setting the binocular viewport offset to simulate the parallax effect of the human eye; configuring perspective parameters by defining the camera angle, near clipping plane distance, and far clipping plane distance; and mapping the outputs of the left and right virtual camera components to the engine's renderTexture component to associate the left and right virtual camera components with the rendering texture.

[0051] In this embodiment, binding the left and right virtual camera components to the same time base to ensure consistent timing of the images from the left and right virtual camera components, and pre-rendering multiple future frames, includes: generating a global timestamp based on a hardware clock, binding the left and right virtual camera components to the same time base so that the left and right eye images share the same timestamp during rendering, ensuring consistent timing; setting the pre-rendering buffer capacity to the number of future frames, generating an optical flow field of the same size as the image, inferring the position of the object in subsequent frames by analyzing the optical flow field, pre-rendering the future multiple frames in the buffer queue using the rendering thread to form a cached image, with the rendering thread and display thread running independently, and the display thread seamlessly connecting the cached image to the display after the current buffer queue is finished displaying.

[0052] In this embodiment, defining the manually / automatically fused camera movement trajectory and generating a continuous and smooth camera movement trajectory through path smoothing interpolation and real-time adjustment includes: manually or automatically defining the camera movement trajectory, modifying the camera movement trajectory in real time through the engine timeline or dragging camera nodes; using a smoothing component to generate a transition trajectory through path smoothing interpolation, and adjusting the camera movement trajectory according to the transition trajectory to generate a continuous and smooth camera movement trajectory.

[0053] In this embodiment, the process of determining a disparity map for the left and right eye images output by the left and right virtual camera components based on a continuous and smooth camera motion trajectory, and then performing regional compression of the left and right eye images according to the disparity map to generate a standard stereoscopic video file includes: performing stereoscopic correction and alignment on the left and right eye images output by the two camera movement nodes based on a continuous and smooth camera motion trajectory; calculating the disparity value of each pixel in the stereoscopically corrected and aligned left and right eye images, generating a grayscale image as a disparity map based on the disparity values, and marking common and difference regions; setting the compression ratio of the common and difference regions; defining the storage path and video parameters before compression or when generating the final file, and generating a standard stereoscopic video file, wherein the video parameters include output resolution, frame rate, and encoding format; and performing regional compression of the left and right eye images according to the compression ratio of the common and difference regions to generate a standard stereoscopic video file.

[0054] In this embodiment, setting a visual editing panel to adjust the camera spacing, camera movement trajectory, or compression parameters of the generated standard stereoscopic video file, and previewing the adjusted standard stereoscopic video file in real time includes: setting a visual editing panel in the engine editor, the visual editing panel integrating sliders and button controls for adjusting and displaying the current camera spacing, camera movement trajectory, and compression parameters; generating the adjusted standard stereoscopic video file in real time after adjusting the current camera spacing, camera movement trajectory, and compression parameters, and generating a preview screen of the adjusted standard stereoscopic video file.

[0055] In one embodiment, as shown in Figure 14, a three-dimensional virtual simulation stereoscopic rendering video generation device is provided, comprising: a three-dimensional scene construction and camera configuration module, used to construct a target three-dimensional scene using a simulation engine, define two camera movement nodes according to the left and right eye positions, construct left and right virtual camera components at the two camera movement nodes, and adjust the spacing between the left and right virtual camera components to match the user's eye distance or scene depth; a screen projection and viewport setting module, used to set the left and right virtual camera components to construct a screen projection matrix for the same target object, set the binocular viewport offset, and associate the left and right virtual camera components with rendering textures; and a time synchronization and frame pre-rendering module, used to bind the left and right virtual camera components to the same time base to ensure that the left and right virtual camera components are aligned with the same time reference. The right virtual camera component displays images in a consistent time sequence and pre-renders multiple future frames. A camera movement trajectory processing module defines manually / automatically fused camera movement trajectories and generates continuous, smooth camera movement trajectories through path smoothing interpolation and real-time adjustment. A disparity map generation and compression optimization module determines the disparity map for the left and right eye images output by the left and right virtual camera components based on the continuous, smooth camera movement trajectory, and performs regional compression of the left and right eye images according to the disparity map to generate a standard stereoscopic video file. A visual editing and parameter adjustment module sets up a visual editing panel to adjust camera spacing, movement trajectory, or compression parameters of the generated standard stereoscopic video file, and provides real-time preview of the adjusted standard stereoscopic video file.

[0056] In the aforementioned 3D virtual simulation stereoscopic rendering video generation device, the visual comfort of stereoscopic videos is significantly improved by dynamically adjusting the spacing between the left and right virtual camera components to adapt to the user's eye distance or scene depth. Simultaneously, high-precision image synchronization technology ensures consistent image timing between the left and right virtual camera components, avoiding image tearing and delay, and enhancing immersion. Path smoothing interpolation and real-time adjustment technology generate continuous and smooth camera motion trajectories, enhancing creative flexibility. Furthermore, the application of a parallax-aware compression algorithm optimizes the video file's bitrate and image quality, resulting in higher video generation efficiency and smaller file size. The introduction of a visual editing panel allows users to adjust and preview camera spacing, camera movement trajectory, and compression parameters in real time, avoiding frequent re-rendering and significantly improving work efficiency.

[0057] Specific limitations regarding the 3D virtual simulation stereoscopic rendering video generation device can be found in the limitations of the 3D virtual simulation stereoscopic rendering video generation method described above, and will not be repeated here. Each module in the aforementioned 3D virtual simulation stereoscopic rendering video generation device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0058] In one embodiment, a computer device, which may be a server, is provided, and its internal structure is shown in Figure 15. The computer device includes a processor, memory, a network interface, and a database connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database stores data for generating 3D virtual simulation stereoscopic rendering videos. The network interface communicates with external terminals via a network connection. When the computer program is executed by the processor, it implements a method for generating 3D virtual simulation stereoscopic rendering videos.

[0059] Those skilled in the art will understand that the structure shown in Figure 15 is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or may combine certain components, or may have different component arrangements.

[0060] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it performs the following steps: constructing a target 3D scene using a simulation engine; defining two camera movement nodes based on the left and right eye positions; constructing left and right virtual camera components at the two camera movement nodes; adjusting the spacing between the left and right virtual camera components to match the user's eye distance or scene depth; setting the left and right virtual camera components to construct a screen projection matrix for the same target object; setting binocular viewport offsets; associating the left and right virtual camera components with rendering textures; binding the left and right virtual camera components to the same time base to ensure consistent screen timing for the left and right virtual camera components; pre-rendering multiple future frames; defining a manually / automatically fused camera movement trajectory; generating a continuous and smooth camera movement trajectory through path smoothing interpolation and real-time adjustment; determining a disparity map for the left and right eye images output by the left and right virtual camera components based on the continuous and smooth camera movement trajectory; performing regional compression on the left and right eye images according to the disparity map to generate a standard stereoscopic video file; setting a visual editing panel to adjust the camera spacing, movement trajectory, or compression parameters of the generated standard stereoscopic video file; and previewing the adjusted standard stereoscopic video file in real time.

[0061] For specific limitations on the steps a processor takes when executing a computer program, please refer to the limitations on the method for generating 3D virtual simulation stereoscopic rendering video mentioned above, which will not be repeated here.

[0062] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When executed by a processor, the computer program performs the following steps: constructing a target 3D scene using a simulation engine; defining two camera movement nodes based on the left and right eye positions; constructing left and right virtual camera components at the two camera movement nodes; adjusting the spacing between the left and right virtual camera components to match the user's eye distance or scene depth; setting the left and right virtual camera components to construct a screen projection matrix for the same target object; setting binocular viewport offsets; associating the left and right virtual camera components with rendering textures; binding the left and right virtual camera components to the same time base to ensure consistent screen timing for the left and right virtual camera components; pre-rendering multiple future frames; defining a manually / automatically fused camera movement trajectory; generating a continuous and smooth camera movement trajectory through path smoothing interpolation and real-time adjustment; determining a disparity map for the left and right eye images output by the left and right virtual camera components based on the continuous and smooth camera movement trajectory; performing regional compression on the left and right eye images according to the disparity map to generate a standard stereoscopic video file; setting a visual editing panel to adjust the camera spacing, movement trajectory, or compression parameters of the generated standard stereoscopic video file; and previewing the adjusted standard stereoscopic video file in real time.

[0063] For specific limitations on the steps implemented when a computer program is executed by a processor, please refer to the limitations on the method of generating 3D virtual simulation stereoscopic rendering video mentioned above, which will not be repeated here.

[0064] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0065] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for generating three-dimensional virtual simulation stereoscopic rendering video, characterized in that, Includes the following steps: A simulation engine is used to construct the target 3D scene. Two camera movement nodes are defined based on the left and right eye positions. Left and right virtual camera components are constructed on these two camera movement nodes, and the spacing between the left and right virtual camera components is adjusted to match the user's eye distance or scene depth. The left and right virtual camera components are set to construct a projection matrix of the image onto the same target object, and binocular viewport offsets are set. The left and right virtual camera components are associated with rendering textures. The left and right virtual camera components are bound to the same time base to ensure that the image sequence of the left and right virtual camera components is consistent, and multiple future frames are pre-rendered. Manual / automatic camera movement trajectories are defined, and continuous and smooth camera movement trajectories are generated through path smoothing interpolation and real-time adjustment. Based on the continuous and smooth camera movement trajectories, a disparity map is determined for the left and right eye images output by the left and right virtual camera components. The left and right eye images are compressed in different regions according to the disparity map to generate a standard stereoscopic video file. A visual editing panel is set up to adjust the camera spacing, movement trajectory, or compression parameters of the generated standard stereoscopic video file, and the adjusted standard stereoscopic video file is previewed in real time.

2. The method according to claim 1, characterized in that, The process of constructing a target 3D scene using a simulation engine, defining two camera movement nodes based on the left and right eye positions, constructing left and right virtual camera components on these two camera movement nodes, and adjusting the spacing between the left and right virtual camera components to match the user's interpupillary distance or scene depth includes: importing an external 3D model into the simulation engine to construct the target 3D scene, loading the model, materials, and lighting parameters; creating an empty object in the simulation engine as a camera movement node, defining two camera movement nodes based on the left and right eye positions, adding left and right virtual camera components to these camera movement nodes; using real-time facial feature tracking to detect the user's interpupillary distance in real time and dynamically adjusting the spacing between the left and right virtual camera components; and adjusting the spacing between the left and right virtual camera components based on the distance to the main object in the scene.

3. The method according to claim 1, characterized in that, The steps of setting the left and right virtual camera components to construct a projection matrix for the same target object, setting binocular viewport offsets, and associating the left and right virtual camera components with the rendering texture include: setting the left and right virtual camera components to construct a projection matrix for the same target object, setting binocular viewport offsets to simulate the parallax effect of the human eye; configuring perspective parameters by defining camera angle, near clipping plane distance, and far clipping plane distance; and mapping the outputs of the left and right virtual camera components to the engine's renderTexture component to associate the left and right virtual camera components with the rendering texture.

4. The method according to claim 1, characterized in that, The step of binding the left and right virtual camera components to the same time base to ensure consistent timing of the images from the left and right virtual camera components and pre-rendering multiple future frames includes: generating a global timestamp based on the hardware clock, binding the left and right virtual camera components to the same time base so that the left and right eye images share the same timestamp during rendering, ensuring consistent timing; setting the pre-rendering buffer capacity to the number of future frames, generating an optical flow field of the same size as the image, inferring the position of the object in subsequent frames by analyzing the optical flow field, pre-rendering the future frames in the buffer queue using the rendering thread to form a cached image, with the rendering thread and display thread running independently, and the display thread seamlessly connecting the cached image to the display after the current buffer queue is finished displaying.

5. The method according to claim 1, characterized in that, The definition of manually / automatically fused camera movement trajectories, and the generation of continuous and smooth camera movement trajectories through path smoothing interpolation and real-time adjustment, includes: manually or automatically defining camera movement trajectories, modifying the camera movement trajectories in real time through the engine timeline or by dragging camera nodes; using a smoothing component to generate transition trajectories through path smoothing interpolation, and adjusting the camera movement trajectories according to the transition trajectories to generate continuous and smooth camera movement trajectories.

6. The method according to claim 1, characterized in that, The process of determining a disparity map for the left and right eye images output by the left and right virtual camera components based on a continuous and smooth camera motion trajectory, and then performing regional compression on the left and right eye images according to the disparity map to generate a standard stereoscopic video file includes: performing stereoscopic correction and alignment on the left and right eye images output by the two camera movement nodes based on a continuous and smooth camera motion trajectory; calculating the disparity value of each pixel in the stereoscopically corrected and aligned left and right eye images, generating a grayscale image as a disparity map based on the disparity values, and marking common and difference regions; setting the compression ratio of the common and difference regions; defining the storage path and video parameters before compression or when generating the final file, and generating a standard stereoscopic video file, wherein the video parameters include output resolution, frame rate, and encoding format; and performing regional compression on the left and right eye images according to the compression ratio of the common and difference regions to generate a standard stereoscopic video file.

7. The method according to claim 1, characterized in that, The step of setting up a visual editing panel to adjust the camera spacing, camera movement trajectory, or compression parameters of the generated standard stereoscopic video file and to preview the adjusted standard stereoscopic video file in real time includes: setting up a visual editing panel in the engine editor, the visual editing panel integrating sliders and button controls for adjusting and displaying the current camera spacing, camera movement trajectory, and compression parameters; generating the adjusted standard stereoscopic video file in real time after adjusting the current camera spacing, camera movement trajectory, and compression parameters; and generating a preview screen of the adjusted standard stereoscopic video file.

8. A three-dimensional virtual simulation stereoscopic rendering video generation device, characterized in that, The device includes: a 3D scene construction and camera configuration module, used to construct a target 3D scene using a simulation engine, define two camera movement nodes based on the left and right eye positions, construct left and right virtual camera components at the two camera movement nodes, and adjust the spacing between the left and right virtual camera components to match the user's eye distance or scene depth; a screen projection and viewport setting module, used to set the screen projection matrix of the left and right virtual camera components for the same target object, set the binocular viewport offset, and associate the left and right virtual camera components with rendering textures; and a time synchronization and frame pre-rendering module, used to bind the left and right virtual camera components to the same time base to ensure that the screen timing of the left and right virtual camera components is consistent, and The system includes: a pre-rendering module for multiple future frames; a camera movement trajectory processing module for defining manually / automatically fused camera movement trajectories and generating continuous, smooth camera movement trajectories through path smoothing interpolation and real-time adjustment; a disparity map generation and compression optimization module for determining disparity maps for the left and right eye images output by the left and right virtual camera components based on the continuous, smooth camera movement trajectories, compressing the left and right eye images into regions according to the disparity maps, and generating standard stereoscopic video files; and a visualization editing and parameter adjustment module for setting up a visualization editing panel to adjust the camera spacing, movement trajectory, or compression parameters of the generated standard stereoscopic video files, and previewing the adjusted standard stereoscopic video files in real time.

9. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images