A naked-eye three-dimensional display rendering method and system based on a zero-parallax reference surface

Abstract

The application discloses a naked-eye three-dimensional display rendering method and system based on a zero-parallax reference surface, relates to the technical field of computer graphics rendering, and comprises the following steps: creating a quadrilateral mesh object as a reference surface geometry, setting an affine correspondence and performing scaling setting on a scale; obtaining a left-eye camera corner point coordinate set and a right-eye camera corner point coordinate set; constructing left-eye and right-eye asymmetric perspective projection matrices; acquiring a preset pupil distance parameter value and calculating a half pupil distance value, and offsetting a left-eye virtual camera and a right-eye virtual camera; outputting left-eye rendering texture and right-eye rendering texture; creating a shader program, and outputting the left-eye rendering texture and the right-eye rendering texture to a frame buffer of a display terminal after fusion; and through explicit configuration of the zero-parallax reference surface and asymmetric perspective projection of the corner point constraint, the depth reference of the stereoscopic content is aligned with a physical screen surface, the visual drift is reduced, and the stability and calibrability of the naked-eye three-dimensional display are improved.

Description

Technical Field

[0001] This invention relates to the field of computer graphics rendering technology, and in particular to a naked-eye 3D display rendering method and system based on a zero parallax reference plane. Background Technology

[0002] Naked-eye 3D display is a display technology that enables observers to obtain stereoscopic visual perception without wearing auxiliary glasses or helmets. Its basic principle is to use beam-splitting elements such as gratings, lens arrays, or barriers to project two images with parallax onto the observer's left and right eyes respectively. The brain then fuses these two images to generate a sense of depth.

[0003] Currently, in naked-eye 3D display rendering, the lack of explicit rigid geometric constraints between the zero-parallax plane of the virtual scene and the physical display screen leads to nonlinear drift of the depth reference of the stereoscopic content under different viewing angles, and the amount of drift varies non-uniformly with the viewing distance. Existing binocular frustum construction methods mostly use symmetrical perspective projection or rely on post-processing depth image correction, without using the four corner points of the physical display screen as constraints for the projection matrix. This results in an uncalibrated systematic deviation between the frustum and the screen surface. This deviation cannot be compensated by simple scaling when switching between scenes. Furthermore, in conventional methods, the offset of the left and right cameras and the rendering order are not linked with the geometric pose of the reference plane, causing the virtual object edges to fail to achieve sub-pixel alignment with the screen surface boundary, resulting in edge jaggedness and ghosting problems.

[0004] Therefore, a naked-eye 3D display rendering method and system based on a zero parallax reference plane is proposed to solve the above problems. Summary of the Invention

[0005] The main objective of this invention is to provide a naked-eye 3D display rendering method and system based on a zero parallax reference plane to solve the problems mentioned in the background.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows: a naked-eye 3D display rendering method based on a zero parallax reference plane, comprising the following steps: Create a quadrilateral mesh object as the reference surface geometry, set the world transformation matrix of the reference surface geometry to be affinely related to the calibration pose of the physical display screen, and scale the reference surface geometry. Obtain the corner point position vectors of the reference plane in the world coordinate system, store them in an ordered corner point queue according to a preset order, and perform order normalization to obtain the left eye camera corner point coordinate set and the right eye camera corner point coordinate set. Projecting the corner points of the left eye camera's corner point coordinate set onto the near clipping plane of the left eye camera yields the projection point coordinates. The minimum value of the horizontal coordinate among the projection point coordinates is taken as the left boundary, the maximum value as the right boundary, the minimum value of the vertical coordinate as the lower boundary, and the maximum value as the upper boundary, to construct the left eye asymmetric perspective projection matrix. Projecting the corner points of the right eye camera's corner point coordinate set onto the near clipping plane of the right eye camera yields the projection point coordinates. The minimum value of the horizontal coordinate among the projection point coordinates is taken as the left boundary, the maximum value as the right boundary, the minimum value of the vertical coordinate as the lower boundary, and the maximum value as the upper boundary, to construct the right eye asymmetric perspective projection matrix. Take the preset interpupillary distance parameter value and calculate the half-interpupillary distance value, and offset the left and right virtual cameras; Configure the camera execution order in the rendering pipeline, execute the scene drawing calls of the left virtual camera and the right virtual camera in sequence, and output the left rendering texture and the right rendering texture. Create a shader program that takes the left and right eye rendering textures as input and blends them according to a preset blending strategy, then outputs the blending result to the frame buffer of the display terminal.

[0007] Preferably, the setting of the affine correspondence includes the following steps: Create a quadrilateral mesh object as a reference surface geometry through the scene configuration interface; Set the world transformation matrix of the reference surface geometry to have an affine correspondence with the calibration pose in the coordinate system of the physical display screen, so that the reference surface geometry represents the zero parallax plane of the physical display screen in the virtual scene; The scale of the reference surface geometry is scaled according to the reference resolution of the display device, so that the size of the reference surface geometry in the virtual space has an affine correspondence with the size of the physical display screen.

[0008] Preferably, obtaining the corner coordinate set includes the following steps: The coordinate acquisition module acquires the position vectors of at least four corner points of the zero parallax reference surface in the world coordinate system. The position vectors are stored in a preset order to obtain an ordered corner point queue, and the ordered corner point queue is normalized. The preset order corresponds one-to-one with the corner point calibration order of the physical display screen. The normalized ordered corner point queues are transformed to the left-eye camera coordinate system and the right-eye camera coordinate system respectively to obtain the left-eye camera corner point coordinate set and the right-eye camera corner point coordinate set.

[0009] Preferably, constructing the left-eye asymmetric perspective projection matrix includes the following steps: Project each corner point from the left eye camera's corner point coordinate set onto the near clipping plane of the left eye camera to obtain the projection point coordinates of each corner point on the near clipping plane; The minimum value of the abscissa among the coordinates of each projection point is taken as the left boundary of the left eye camera's viewing cone, the maximum value of the abscissa is taken as the right boundary of the left eye camera's viewing cone, the minimum value of the ordinate is taken as the lower boundary of the left eye camera's viewing cone, and the maximum value of the ordinate is taken as the upper boundary of the left eye camera's viewing cone. Based on the left boundary, right boundary, lower boundary, upper boundary, and the distances to the near and far clipping planes, an asymmetric perspective projection matrix for the left eye is constructed.

[0010] Preferably, constructing the right-eye asymmetric perspective projection matrix includes the following steps: Project each corner point in the right eye camera corner point coordinate set onto the near clipping plane of the right eye camera to obtain the projection point coordinates of each corner point on the near clipping plane; The minimum value of the x-coordinate among the coordinates of each projection point is taken as the left boundary of the right eye camera's viewing cone, the maximum value of the x-coordinate is taken as the right boundary of the right eye camera's viewing cone, the minimum value of the y-coordinate is taken as the lower boundary of the right eye camera's viewing cone, and the maximum value of the y-coordinate is taken as the upper boundary of the right eye camera's viewing cone. Based on the left boundary, right boundary, lower boundary, upper boundary, and the distances to the near and far clipping planes, an asymmetric perspective projection matrix for the right eye is constructed.

[0011] Preferably, the offsetting of the left and right virtual cameras includes the following steps: Obtain a preset interpupillary distance parameter value, which is dynamically adjusted according to user input or calibration program; Calculate the half pupillary distance value, which is equal to half the pupillary distance parameter value; The left virtual camera is offset by a negative half-pupil distance value along the right vector direction of the camera's local coordinate system, and the right virtual camera is offset by a positive half-pupil distance value along the right vector direction of the camera's local coordinate system.

[0012] Preferably, the offsetting of the left and right virtual cameras includes the following steps: Obtain the right unit vector of the local coordinate system of the left virtual camera, multiply the right unit vector by the negative half pupil distance to obtain the left offset vector, and add the left offset vector to the world position coordinates of the left virtual camera to complete the offset of the left virtual camera. Obtain the right unit vector of the local coordinate system of the right virtual camera, multiply the right unit vector by the positive half pupil distance to obtain the right offset vector, and add the right offset vector to the world position coordinates of the right virtual camera to complete the offset of the right virtual camera. Keep the orientation of the left and right virtual cameras consistent with the direction of the main viewpoint, so that the viewpoint centers of the left and right virtual cameras are symmetrically distributed with the main viewpoint after the offset.

[0013] Preferably, the output of the left-eye rendering texture and the right-eye rendering texture includes the following steps: Configure the left and right virtual cameras in the rendering pipeline to execute before the main rendering camera; Bind the rendering target of the left virtual camera to the first rendering texture object, and bind the rendering target of the right virtual camera to the second rendering texture object; The scene rendering calls for the left and right virtual cameras are executed sequentially, and the left and right rendering textures are output respectively.

[0014] Preferably, the process of fusing and outputting the left and right eye images includes the following steps: Shader programs are created using a programmable rendering pipeline, and these shader programs are configured with a stereo compositing strategy. The left and right rendered textures are used as inputs and bound to the sampler unit of the shader procedure; The shader program executes the left and right images to blend them according to a preset blending strategy, and outputs the blending result to the frame buffer of the display terminal.

[0015] A naked-eye 3D display rendering system based on a zero parallax reference plane, the system comprising: Zero parallax reference plane configuration module: used to create a quadrilateral mesh object as a zero parallax reference plane, and set the world transformation matrix of the zero parallax reference plane to have an affine correspondence with the calibration pose of the physical display screen; Coordinate acquisition module: used to acquire the corner coordinates of the zero parallax reference plane in the world coordinate system, transform them to the left and right camera coordinate systems after sequential normalization, and output the left and right camera corner coordinate sets. Asymmetric perspective projection matrix calculation module: used to construct the left-eye asymmetric perspective projection matrix and the right-eye asymmetric perspective projection matrix based on the left-eye camera corner point coordinate set and the right-eye camera corner point coordinate set, respectively; Binocular offset module: used to obtain preset interpupillary distance parameter values ​​and calculate half-interpupillary distance values, offset the left virtual camera along the right vector direction of the local coordinate system by a negative half-interpupillary distance value, and offset the right virtual camera along the right vector direction of the local coordinate system by a positive half-interpupillary distance value. Binocular rendering module: Used to set the rendering targets of the left and right virtual cameras as independent rendering texture objects, configure the rendering order of the left and right virtual cameras to be before the main rendering camera, and output the left and right rendering textures after the rendering operation is performed. Composite output module: Creates a shader program through a programmable rendering pipeline, binds the left and right eye rendering textures as inputs to the shader program, and merges the left and right eye images according to a preset blending strategy before outputting them to the frame buffer of the display terminal.

[0016] The present invention has the following beneficial effects: 1. This invention creates a quadrilateral mesh object as a reference surface geometry and sets its world transformation matrix to have an affine correspondence with the calibration pose of the physical display screen. This allows the reference surface geometry to accurately represent the zero-parallax plane of the physical display screen in the virtual scene, rigidly binding the depth reference of the virtual scene with the geometric position of the physical screen, thus solving the problem of uncertainty in the depth reference caused by the lack of an explicit correspondence between the zero-parallax plane and the screen. At the same time, by scaling the scale of the reference surface geometry, the size of the reference surface geometry in the virtual space is made to have an affine correspondence with the size of the physical display screen, further eliminating the overall depth drift caused by scale mismatch and improving the calibrability and initial alignment accuracy of naked-eye 3D display.

[0017] 2. This invention obtains the corner position vectors of the reference plane in the world coordinate system and transforms them to the coordinate systems of the left and right cameras respectively. It then uses the extreme values ​​of the projection point coordinates of each corner point on the near clipping plane to construct the left and right asymmetric perspective projection matrices, respectively. This ensures precise matching between the left and right camera frustums and the zero-parallax reference plane. The four corner points of the physical display screen serve as hard constraints on the frustum boundaries, solving the system deviation problem caused by the inability of traditional symmetrical perspective projection to align geometrically with the fixed screen surface. Furthermore, by obtaining preset interpupillary distance parameters and calculating the half-interpupillary distance value, the left virtual camera is offset by a negative half-interpupillary distance value along the right vector direction, and the right virtual camera is offset by a positive half-interpupillary distance value. This ensures that the centers of the left and right viewpoints are symmetrically distributed with the main viewpoint. This achieves binocular parallax that conforms to human physiological characteristics while maintaining the alignment of the frustum with the screen surface, avoiding visual fatigue and ghosting caused by improper offset, and improving the comfort and stability of stereoscopic rendering.

[0018] 3. This invention, by configuring the execution order of the left and right virtual cameras in the rendering pipeline to precede that of the main rendering camera, and binding the rendering targets of the left and right cameras to independent rendering texture objects respectively, and executing scene drawing calls sequentially to output the left and right rendering textures, ensures that both left and right images are completely generated before stereoscopic compositing, solving the problem of image loss or timing misalignment caused by disordered rendering order. Furthermore, by creating a shader program with a stereoscopic compositing strategy through the programmable rendering pipeline, the left and right rendering textures are bound as input to the sampler unit, fused according to a preset blending strategy, and output to the frame buffer of the display terminal. This achieves sub-pixel alignment between the left and right images and the display hardware, eliminating edge jaggedness and ghosting, and ultimately aligning the depth reference in the virtual scene with the zero parallax plane of the physical display screen, improving the long-term stability and cross-scene reproducibility of naked-eye 3D display. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating a naked-eye 3D display rendering method based on a zero parallax reference plane according to the present invention. Figure 2 This is a schematic diagram of the overall architecture of a naked-eye 3D display rendering system based on a zero parallax reference plane according to the present invention. Detailed Implementation

[0020] To make the technical means, creative features, objectives and effects of this invention easier to understand, the invention will be further described below in conjunction with specific embodiments.

[0021] Example 1, please refer to Figures 1-2 As shown: A naked-eye 3D display rendering method based on a zero parallax reference plane includes the following steps: Reference surface configuration and corner point acquisition: S1. Create a quadrilateral mesh object as a reference surface geometry, set the world transformation matrix of the reference surface geometry to be affinely related to the calibration pose of the physical display screen, and scale the reference surface geometry. S2. Obtain the corner position vectors of the reference plane in the world coordinate system, store them in an ordered corner queue according to a preset order and perform order normalization to obtain the left eye camera corner coordinate set and the right eye camera corner coordinate set. Asymmetric projection and binocular offset: S3. Project each corner point in the left eye camera corner point coordinate set onto the near clipping plane of the left eye camera to obtain the projection point coordinates. Take the minimum value of the horizontal coordinate of each projection point coordinate as the left boundary, the maximum value as the right boundary, the minimum value of the vertical coordinate as the lower boundary, and the maximum value as the upper boundary to construct the left eye asymmetric perspective projection matrix. S4. Project each corner point in the right eye camera corner point coordinate set onto the near clipping plane of the right eye camera to obtain the projection point coordinates. Take the minimum value of the horizontal coordinate of each projection point coordinate as the left boundary, the maximum value as the right boundary, the minimum value of the vertical coordinate as the lower boundary, and the maximum value as the upper boundary to construct the right eye asymmetric perspective projection matrix. S5. Obtain the preset pupil distance parameter value and calculate the half pupil distance value, and offset the left and right virtual cameras; Binocular rendering and stereoscopic compositing: S6. Configure the camera execution order in the rendering pipeline, execute the scene drawing calls of the left virtual camera and the right virtual camera in sequence, and output the left rendering texture and the right rendering texture. S7. Create a shader program that takes the left and right rendering textures as input and blends them according to a preset blending strategy. Output the blending result to the frame buffer of the display terminal.

[0022] Setting the affine correspondence in S1 includes the following steps: S11. Create a quadrilateral mesh object as a reference surface geometry through the scene configuration interface. In practice: The scene configuration interface is a scene management function provided by the rendering engine (such as Unity, Unreal Engine, etc.). It calls the mesh creation method in the interface, sets the mesh vertex coordinates and index buffer. The four vertices of the quadrilateral mesh object are located at the four corner points in the local coordinate system. In this embodiment, the vertex coordinates are set as (-1, -1, 0), (1, -1, 0), (1, 1, 0), and (-1, 1, 0), thus forming a unit plane with a width of 2 and a height of 2. The index buffer defines two triangles ((0, 1, 2) and (0, 2, 3)) to form a complete mesh. After creation, the quadrilateral mesh object is added to the root node or a specified parent node of the virtual scene so that it can participate in scene transformation and rendering.

[0023] S12. Set the world transformation matrix of the reference surface geometry to have an affine correspondence with the calibration pose in the coordinate system of the physical display screen, so that the reference surface geometry represents the zero parallax plane of the physical display screen in the virtual scene. In specific implementation: Obtain the calibration data of the physical display screen: the position vector in the world coordinate system (e.g., the coordinates of the center point of the display screen) and the rotation angle (Euler angles or quaternions), and construct the affine transformation relationship as follows: ; in Let be the position vector of any point on the reference plane in the local coordinate system; R is the world coordinates of the transformed point; R is a 3×3 rotation matrix, determined by the normal direction and horizontal direction of the display screen, which rotates the reference plane from the default orientation (front + Z-axis) to the actual orientation of the display screen; T is a translation vector, which takes the world coordinates of the center of the display screen (or a specified corner point).

[0024] Combine R and T into a 4×4 homogeneous transformation matrix. The world transformation matrix of the reference surface geometry is assigned to it, which only handles rotation and translation, and does not change the original size of the reference surface (width 2, height 2).

[0025] S13. Scale the reference surface geometry according to the reference resolution of the display device, so that the size of the reference surface geometry in virtual space has an affine correspondence with the size of the physical display screen. In specific implementation: Get the physical width of the physical display screen and physical height (Unit: mm or actual length), since the original width and height of the reference surface in S11 are 2, calculate the scaling factor: ; ; To ensure no deformation, it is usually required (That is, the aspect ratio of the display screen is consistent with the original aspect ratio of the reference surface). If the actual ratio is different, it can be set independently, but it is necessary to ensure that the four corner points of the reference surface are still rectangles and coplanar.

[0026] Construct the scaling matrix S (the scaling part of a 3×3 diagonal matrix or a 4×4 homogeneous matrix): ; Multiply the scaling matrix by the world transformation matrix of S12 (scaling first, then rotation and translation): This ensures that the world coordinates of the four corner points of the reference plane are precisely linearly mapped to the four corner points of the physical display screen.

[0027] Obtaining the set of corner coordinates in S2 includes the following steps: S21. Obtain the position vectors of at least four corner points of the zero-parallax reference plane in the world coordinate system through the coordinate acquisition module. Specifically: The coordinate acquisition module first reads the world transformation matrix of the reference surface geometry. (4×4 homogeneous matrix), this matrix is ​​composed of translations, rotations, and scaling operations set in S12 and S13. Since the coordinates of the four corner points of the reference surface in S11 are fixed in the local coordinate system as (-1, -1, 0), (1, -1, 0), (1, 1, 0), and (-1, 1, 0), it is not necessary to traverse the vertex buffer. For each local coordinate... World coordinates are calculated using matrix multiplication in homogeneous coordinates: ; in This represents the homogeneous coordinates of the reference face corner point in the world coordinate system (the first three components represent the 3D position). This multiplication can be directly implemented by calling the coordinate transformation function provided by the rendering engine (such as TransformPoint). After calculation, four world coordinate vectors are obtained, denoted as... , , , , used for subsequent sequential processing.

[0028] S22. Store the position vectors according to a preset order to obtain an ordered corner point queue, and perform order normalization processing on the ordered corner point queue. The preset order corresponds one-to-one with the corner point calibration order of the physical display screen. In specific implementation: The default order is to store the four corner points clockwise, starting from the top left corner of the screen (i.e., top left, top right, bottom right, and bottom left). The four world coordinates obtained in S21 are stored in the array in this order to obtain an ordered corner point queue. Calculate the world normal vector n of the reference surface: Take the normal (0, 0, 1) in the local coordinate system of the reference surface, and rotate it to the world coordinate system, i.e. Where T represents transpose and R is The 3×3 rotating part.

[0029] Check the direction of the vertex order: take the first three corner points in the queue. , , Calculate the edge vectors , Then calculate the cross product. ,like If the current order is counterclockwise, reverse the entire queue to make it clockwise.

[0030] Identify and calibrate the top-left corner: Given the world upward vector up (usually (0, 1, 0)) and the horizontal rightward vector right (usually (1, 0, 0)), calculate the center of the reference plane. For each corner point, calculate its offset relative to the center. Having the smallest and maximum The corner point is the top left corner. The queue is moved cyclically so that the corner point is at the head of the queue, thus obtaining a normalized ordered corner point queue, whose order is completely consistent with the corner point marking order of the physical display screen.

[0031] S23. Transform the normalized ordered corner point queue to the left-eye camera coordinate system and the right-eye camera coordinate system respectively to obtain the left-eye camera corner point coordinate set and the right-eye camera corner point coordinate set. In practice: First, obtain the view matrix of the left virtual camera. and the view matrix of the right virtual camera The view matrix is ​​a 4×4 homogeneous matrix, representing the transformation from the world coordinate system to the camera's local coordinate system, and its mathematical definition is... ,in It is the transformation matrix of the camera in the world (which can be obtained through the rendering engine's GetWorldTransform).

[0032] For each world coordinate in the ordered corner queue Calculate its coordinates in the left and right camera coordinate systems respectively: ; ; The multiplication is a matrix-vector multiplication in homogeneous coordinates, and the result is... and The coordinates are four-dimensional homogeneous coordinates. The first three components are taken as three-dimensional coordinates, and the four transformed coordinate points are stored in the left eye camera corner coordinate set. and the right eye camera corner coordinate set .

[0033] Example 2: Constructing the left-eye asymmetric perspective projection matrix in S3 includes the following steps: S31. Project each corner point in the left eye camera's corner point coordinate set onto the near clipping plane of the left eye camera to obtain the coordinates of the projected points of each corner point on the near clipping plane. In specific implementation: Let the coordinates of any corner point in the left eye camera corner point coordinate set be... ,in The distance to the near clipping plane of the left camera is n (a positive value), and this plane lies in the camera's local coordinate system. Location (camera facing) (Direction), according to the principle of similar triangles, corner points Projection point on the near-cut plane The calculation formula is: ; ; in To ensure the corner points are in front of the camera, perform the above projection calculations sequentially on the four corner points in the left eye camera's corner point coordinate set, obtaining the coordinates of the four projected points. , , , .

[0034] S32. Take the minimum value of the x-coordinate among the coordinates of each projection point as the left boundary of the left eye camera's viewing cone, the maximum value of the x-coordinate as the right boundary of the left eye camera's viewing cone, the minimum value of the y-coordinate as the lower boundary of the left eye camera's viewing cone, and the maximum value of the y-coordinate as the upper boundary of the left eye camera's viewing cone. In specific implementation: Let the x-coordinates of the four projection points be respectively , , , The vertical axes are respectively , , , ,but: ; ; ; ; Where l, r, b, and t are the left, right, lower, and upper boundaries, respectively. These boundary values ​​constitute the rectangular cross-section of the asymmetric view cone on the near clipping plane, precisely covering the projection range of the four corner points of the reference plane on the near clipping plane of the left eye camera.

[0035] S33. Based on the left boundary, right boundary, lower boundary, upper boundary, and the distances from the near and far clipping planes, construct the left-eye asymmetric perspective projection matrix. In practice: Asymmetric perspective projection matrix The standard form (taking the right multiplication of column vectors as an example) is: ; Where n is the near clipping plane distance (positive value); f is the far clipping plane distance (positive value); l, r, b, t are the boundary values ​​calculated by S32; substituting the above parameters into the matrix yields the asymmetric perspective projection matrix of the left eye camera. This matrix ensures that the cone boundary of the left eye camera is precisely matched with the corner projection of the zero parallax reference plane, thereby directly encoding the geometric constraints of the reference plane into the projection transformation. The asymmetric perspective projection matrix of the right eye camera is constructed using the same method.

[0036] Constructing the right-eye asymmetric perspective projection matrix in S4 includes the following steps: S41. Project each corner point in the right eye camera corner point coordinate set onto the near clipping plane of the right eye camera to obtain the projection point coordinates of each corner point on the near clipping plane. S42. Take the minimum value of the x-coordinate of each projection point as the left boundary of the right eye camera's viewing cone, the maximum value of the x-coordinate as the right boundary of the right eye camera's viewing cone, the minimum value of the y-coordinate as the lower boundary of the right eye camera's viewing cone, and the maximum value of the y-coordinate as the upper boundary of the right eye camera's viewing cone. S43. Based on the left boundary, right boundary, lower boundary, upper boundary, and the distances to the near and far clipping planes, construct the right eye asymmetric perspective projection matrix.

[0037] The offsetting of the left and right virtual cameras in S5 includes the following steps: S51. Obtain the preset interpupillary distance parameter value. The interpupillary distance parameter value is dynamically adjusted according to user input or calibration program. In specific implementation: The interpupillary distance parameter value represents the horizontal distance between the viewpoints of the left and right virtual cameras. The unit is world units in the virtual scene. The user input method is to set the value through the slider or numerical input box in the graphical interface. The value range is usually 0.05-0.08 (assuming the scene unit is meters). The calibration program reads the preset configuration file or automatically measures and inputs the value through an external calibration device. The obtained interpupillary distance parameter value is recorded as variable d, and subsequent steps use this value for calculation.

[0038] S52. Calculate the half-pupil distance value. The half-pupil distance value is equal to half the pupillary distance parameter value. In specific implementation: Right now The half-pupil distance value represents the offset distance of the left and right virtual cameras relative to the main viewpoint. After calculation, h is stored as a floating-point variable to control the magnitude of the offset.

[0039] S53. Offset the left virtual camera by a negative half-pupil distance value along the right vector direction of the camera's local coordinate system, and offset the right virtual camera by a positive half-pupil distance value along the right vector direction of the camera's local coordinate system. In specific implementation: S531. Obtain the right unit vector of the local coordinate system of the left virtual camera. Multiply the right unit vector by the negative half-pupil distance to obtain the left offset vector. Add the left offset vector to the world position coordinates of the left virtual camera to complete the offset of the left virtual camera. In specific implementation: The right unit vector of the local coordinate system of the left virtual camera can be obtained through the first row component of the camera transformation matrix (or the `right` attribute provided by the engine), denoted as... Its modulus is 1, and the half-pupil distance is h. Then the left eye offset vector The negative sign indicates that the offset direction is opposite to the right vector (i.e., offset to the left). The current world position coordinates of the left virtual camera are obtained, denoted as... The position coordinates of the left virtual camera after offset are: Set the position of the left virtual camera to Complete the offset.

[0040] S532. Obtain the right unit vector of the local coordinate system of the right virtual camera. Multiply the right unit vector by the positive half-pupil distance to obtain the right offset vector. Add the right offset vector to the world position coordinates of the right virtual camera to complete the offset of the right virtual camera. In specific implementation: The right unit vector of the local coordinate system of the right virtual camera is denoted as... Similarly, the right eye offset vector is extracted from the camera transformation matrix. The plus sign indicates that the offset direction is the same as the right vector (i.e., offset to the right), obtaining the current world position coordinates of the right virtual camera. Position after offset Set the position of the right virtual camera to Complete the offset.

[0041] S533. Keep the orientation of the left and right virtual cameras consistent with the direction of the main viewpoint, so that the viewpoint centers of the left and right virtual cameras are symmetrically distributed with the main viewpoint after the offset. In specific implementation: Here, the main viewpoint is defined as the common reference point before the left and right virtual cameras are offset. It is usually the position of the main rendering camera (or the original camera position). The offset operation only changes the camera position and does not change its rotation angle. Specifically, the rotation matrix (or quaternion) of the main viewpoint is obtained and directly assigned to the rotation attributes of the left and right virtual cameras to ensure that the viewing directions of the three are exactly the same.

[0042] After the offset, the positions of the left and right virtual cameras satisfy a symmetrical relationship: the offset of the left camera is... The right eye offset is Where r is the common right vector direction, the midpoint of the left and right eye positions is exactly equal to the position of the main viewpoint, i.e., satisfying ,in As the world position of the main viewpoint, this symmetry ensures zero parallax at the zero parallax reference plane and avoids vertical parallax and geometric distortion caused by camera rotation.

[0043] Example 3: Outputting the left and right view rendering textures in S6 includes the following steps: S61. Configure the execution order of the left and right virtual cameras in the rendering pipeline to precede that of the main rendering camera. In practice: The rendering pipeline typically executes multi-camera rendering in a frame-by-frame loop. The rendering order is controlled by setting the camera rendering priority or explicitly calling rendering commands. By accessing the rendering engine's camera rendering queue management interface, and setting the execution indices of the left and right virtual cameras before the main rendering camera, the rendering functions of the left and right virtual cameras are called first, followed by the main rendering camera's rendering function. This ensures that the left and right virtual camera images are generated before the main rendering camera outputs; the execution order constraints satisfy… and ,in and These represent the rendering completion times of the left and right virtual cameras, respectively. This indicates the start time of rendering by the main rendering camera.

[0044] S62. Bind the rendering target of the left virtual camera to the first rendering texture object, and bind the rendering target of the right virtual camera to the second rendering texture object. Specifically: A rendering texture object is a two-dimensional array that stores pixel data in GPU memory. Two texture resources are generated by calling the texture creation function of the graphics API, namely the first rendering texture object and the second rendering texture object. Their width, height, and pixel format are set to RGBA8. By setting the rendering target function, the output of the left virtual camera is directed to the first rendering texture object, and the output of the right virtual camera is directed to the second rendering texture object. After binding, the color data generated by the left virtual camera is written to the first rendering texture object, and the color data generated by the right virtual camera is written to the second rendering texture object. The two do not interfere with each other.

[0045] S63. Execute the scene rendering calls for the left and right virtual cameras sequentially, and output the left and right rendering textures respectively. Specifically: First, the drawing command of the left virtual camera is called, traversing all visible objects in the virtual scene. Vertex shaders and fragment shaders are executed for each object, and the rendering results are written to the first rendering texture object. After the left virtual camera is drawn, the drawing command of the right virtual camera is called, and the rendering is performed with the same scene but different frustum parameters. The results are written to the second rendering texture object. After the two drawing calls are completed, the left view is stored in the first rendering texture object, and the right view is stored in the second rendering texture object. The two texture objects are used as output results for subsequent compositing modules.

[0046] The steps involved in fusing and outputting the left and right images in S7 include: S71. Create a shader program using the programmable rendering pipeline. The shader program is configured with a stereo compositing strategy. In practice: The shader program consists of a vertex shader and a fragment shader, written in a high-level shading language (such as GLSL or HLSL). It calls the compilation functions of the graphics API to compile and link the vertex shader and fragment shader source code to generate an executable shader program object. The fragment shader defines a stereo compositing strategy, the core of which is determining the color source of the output pixel; for example, for a raster barrier-type glasses-free 3D display, an alternating odd-even column rule can be used: the screen coordinates (x, y) of the current pixel satisfy... Take the left eye texture pixel at time, satisfying The right eye texture pixel is taken at that time.

[0047] S72. Bind the left and right eye rendering textures as input to the sampler unit of the shader program. In specific implementation: Sampler units are unified variables used in shader programs to read texture data. By calling the binding texture function of the graphics API, the left-eye rendering texture is bound to texture unit 0, and this texture unit is associated with the left-eye sampler variable declared in the shader program; the right-eye rendering texture is bound to texture unit 1, and associated with the right-eye sampler variable. After binding, the shader program can access the pixel color values ​​in the left and right-eye rendering textures through the sampler variables during execution.

[0048] S73. The shader program is executed to fuse the left and right images according to a preset blending strategy, and the fusion result is output to the frame buffer of the display terminal. In specific implementation: Calling the graphics API's command to draw a full-screen quadrilateral (or a full-screen triangle) triggers the execution of the shader program. For each pixel in the framebuffer, the vertex shader passes the screen coordinates, and the fragment shader samples color values ​​from the bound left and right rendering textures according to a preset blending strategy, calculating the output color. Taking an alternating odd-even column strategy as an example, the output color... The calculation formula is: ; Where x and y are the integer coordinates of the current pixel in the screen coordinate system (the origin is usually the top left or bottom left corner). The color value is sampled from the left-eye rendered texture; The color value is sampled from the right-eye rendered texture; for other blending strategies (such as row alternation, checkerboard, or dynamic weights based on viewpoint position), only the sampling condition function needs to be changed. The blended image data is written to the frame buffer, which is then read by the display terminal to refresh the screen.

[0049] A naked-eye 3D display rendering system based on a zero parallax reference plane, the system comprising: Zero parallax reference plane configuration module: used to create a quadrilateral mesh object as a zero parallax reference plane, and set the world transformation matrix of the zero parallax reference plane to have an affine correspondence with the calibration pose of the physical display screen; Coordinate acquisition module: used to acquire the corner coordinates of the zero parallax reference plane in the world coordinate system, transform them to the left and right camera coordinate systems after sequential normalization, and output the left and right camera corner coordinate sets. Asymmetric perspective projection matrix calculation module: used to construct the left-eye asymmetric perspective projection matrix and the right-eye asymmetric perspective projection matrix based on the left-eye camera corner point coordinate set and the right-eye camera corner point coordinate set, respectively; Binocular offset module: used to obtain preset interpupillary distance parameter values ​​and calculate half-interpupillary distance values, offset the left virtual camera along the right vector direction of the local coordinate system by a negative half-interpupillary distance value, and offset the right virtual camera along the right vector direction of the local coordinate system by a positive half-interpupillary distance value. Binocular rendering module: Used to set the rendering targets of the left and right virtual cameras as independent rendering texture objects, configure the rendering order of the left and right virtual cameras to be before the main rendering camera, and output the left and right rendering textures after the rendering operation is performed. Composite Output Module: Creates a shader program through the programmable rendering pipeline, binds the left and right eye rendering textures as inputs to the shader program, and merges the left and right eye images according to a preset blending strategy before outputting them to the frame buffer of the display terminal.

[0050] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of this invention is defined by the appended claims and their equivalents.

Claims

1. A naked-eye 3D display rendering method based on a zero-parallax reference plane, characterized in that, Includes the following steps: Create a quadrilateral mesh object as the reference surface geometry, set the world transformation matrix of the reference surface geometry to be affinely related to the calibration pose of the physical display screen, and scale the reference surface geometry. Obtain the corner point position vectors of the reference plane in the world coordinate system, store them in an ordered corner point queue according to a preset order, and perform order normalization to obtain the left eye camera corner point coordinate set and the right eye camera corner point coordinate set. Projecting the corner points of the left eye camera's corner point coordinate set onto the near clipping plane of the left eye camera yields the projection point coordinates. The minimum value of the horizontal coordinate among the projection point coordinates is taken as the left boundary, the maximum value as the right boundary, the minimum value of the vertical coordinate as the lower boundary, and the maximum value as the upper boundary, to construct the left eye asymmetric perspective projection matrix. Projecting the corner points of the right eye camera's corner point coordinate set onto the near clipping plane of the right eye camera yields the projection point coordinates. The minimum value of the horizontal coordinate among the projection point coordinates is taken as the left boundary, the maximum value as the right boundary, the minimum value of the vertical coordinate as the lower boundary, and the maximum value as the upper boundary, to construct the right eye asymmetric perspective projection matrix. Obtain the preset interpupillary distance parameter value and calculate the half-interpupillary distance value, and offset the left and right virtual cameras; Configure the camera execution order in the rendering pipeline, execute the scene drawing calls of the left virtual camera and the right virtual camera in sequence, and output the left rendering texture and the right rendering texture. Create a shader program that takes the left and right eye rendering textures as input and blends them according to a preset blending strategy, then outputs the blending result to the frame buffer of the display terminal.

2. The naked-eye 3D display rendering method based on a zero parallax reference plane according to claim 1, characterized in that, Setting the affine correspondence includes the following steps: Create a quadrilateral mesh object as a reference surface geometry through the scene configuration interface; Set the world transformation matrix of the reference surface geometry to have an affine correspondence with the calibration pose in the coordinate system of the physical display screen, so that the reference surface geometry represents the zero parallax plane of the physical display screen in the virtual scene; The scale of the reference surface geometry is scaled according to the reference resolution of the display device, so that the size of the reference surface geometry in the virtual space has an affine correspondence with the size of the physical display screen.

3. The naked-eye 3D display rendering method based on a zero parallax reference plane according to claim 1, characterized in that, Obtaining the set of corner coordinates includes the following steps: The coordinate acquisition module acquires the position vectors of at least four corner points of the zero parallax reference surface in the world coordinate system. The position vectors are stored in a preset order to obtain an ordered corner point queue, and the ordered corner point queue is normalized. The preset order corresponds one-to-one with the corner point calibration order of the physical display screen. The normalized ordered corner point queues are transformed to the left-eye camera coordinate system and the right-eye camera coordinate system respectively to obtain the left-eye camera corner point coordinate set and the right-eye camera corner point coordinate set.

4. The naked-eye 3D display rendering method based on a zero parallax reference plane according to claim 1, characterized in that, The construction of the left-eye asymmetric perspective projection matrix includes the following steps: Project each corner point from the left eye camera's corner point coordinate set onto the near clipping plane of the left eye camera to obtain the projection point coordinates of each corner point on the near clipping plane; The minimum value of the abscissa among the coordinates of each projection point is taken as the left boundary of the left eye camera's viewing cone, the maximum value of the abscissa is taken as the right boundary of the left eye camera's viewing cone, the minimum value of the ordinate is taken as the lower boundary of the left eye camera's viewing cone, and the maximum value of the ordinate is taken as the upper boundary of the left eye camera's viewing cone. Based on the left boundary, right boundary, lower boundary, upper boundary, and the distances to the near and far clipping planes, an asymmetric perspective projection matrix for the left eye is constructed.

5. The naked-eye 3D display rendering method based on a zero parallax reference plane according to claim 1, characterized in that, The construction of the right-eye asymmetric perspective projection matrix includes the following steps: Project each corner point in the right eye camera corner point coordinate set onto the near clipping plane of the right eye camera to obtain the projection point coordinates of each corner point on the near clipping plane; The minimum value of the x-coordinate among the coordinates of each projection point is taken as the left boundary of the right eye camera's viewing cone, the maximum value of the x-coordinate is taken as the right boundary of the right eye camera's viewing cone, the minimum value of the y-coordinate is taken as the lower boundary of the right eye camera's viewing cone, and the maximum value of the y-coordinate is taken as the upper boundary of the right eye camera's viewing cone. Based on the left boundary, right boundary, lower boundary, upper boundary, and the distances to the near and far clipping planes, an asymmetric perspective projection matrix for the right eye is constructed.

6. The naked-eye 3D display rendering method based on a zero parallax reference plane according to claim 1, characterized in that, The offsetting of the left and right virtual cameras includes the following steps: Obtain a preset interpupillary distance parameter value, which is dynamically adjusted according to user input or calibration program; Calculate the half pupillary distance value, which is equal to half the pupillary distance parameter value; The left virtual camera is offset by a negative half-pupil distance value along the right vector direction of the camera's local coordinate system, and the right virtual camera is offset by a positive half-pupil distance value along the right vector direction of the camera's local coordinate system.

7. The naked-eye 3D display rendering method based on a zero parallax reference plane according to claim 6, characterized in that, The offsetting of the left and right virtual cameras includes the following steps: Obtain the right unit vector of the local coordinate system of the left virtual camera, multiply the right unit vector by the negative half pupil distance to obtain the left offset vector, and add the left offset vector to the world position coordinates of the left virtual camera to complete the offset of the left virtual camera. Obtain the right unit vector of the local coordinate system of the right virtual camera, multiply the right unit vector by the positive half pupil distance to obtain the right offset vector, and add the right offset vector to the world position coordinates of the right virtual camera to complete the offset of the right virtual camera. Keep the orientation of the left and right virtual cameras consistent with the direction of the main viewpoint, so that the viewpoint centers of the left and right virtual cameras are symmetrically distributed with the main viewpoint after the offset.

8. The naked-eye 3D display rendering method based on a zero parallax reference plane according to claim 1, characterized in that, The output of the left-eye rendering texture and the right-eye rendering texture includes the following steps: Configure the left and right virtual cameras in the rendering pipeline to execute before the main rendering camera; Bind the rendering target of the left virtual camera to the first rendering texture object, and bind the rendering target of the right virtual camera to the second rendering texture object; The scene rendering calls for the left and right virtual cameras are executed sequentially, and the left and right rendering textures are output respectively.

9. The naked-eye 3D display rendering method based on a zero parallax reference plane according to claim 1, characterized in that, The process of fusing and outputting the left and right visual images includes the following steps: Shader programs are created using a programmable rendering pipeline, and these shader programs are configured with a stereo compositing strategy. The left and right rendered textures are used as inputs and bound to the sampler unit of the shader procedure; The shader program executes the left and right images to blend them according to a preset blending strategy, and outputs the blending result to the frame buffer of the display terminal.

10. A naked-eye 3D display rendering system based on a zero-parallax reference plane, comprising the naked-eye 3D display rendering method based on a zero-parallax reference plane as described in any one of claims 1-9, characterized in that, The system includes: Zero parallax reference plane configuration module: used to create a quadrilateral mesh object as a zero parallax reference plane, and set the world transformation matrix of the zero parallax reference plane to have an affine correspondence with the calibration pose of the physical display screen; Coordinate acquisition module: used to acquire the corner coordinates of the zero parallax reference plane in the world coordinate system, transform them to the left and right camera coordinate systems after sequential normalization, and output the left and right camera corner coordinate sets. Asymmetric perspective projection matrix calculation module: used to construct the left-eye asymmetric perspective projection matrix and the right-eye asymmetric perspective projection matrix based on the left-eye camera corner point coordinate set and the right-eye camera corner point coordinate set, respectively; Binocular offset module: used to obtain preset interpupillary distance parameter values ​​and calculate half-interpupillary distance values, offset the left virtual camera along the right vector direction of the local coordinate system by a negative half-interpupillary distance value, and offset the right virtual camera along the right vector direction of the local coordinate system by a positive half-interpupillary distance value. Binocular rendering module: Used to set the rendering targets of the left and right virtual cameras as independent rendering texture objects, configure the rendering order of the left and right virtual cameras to be before the main rendering camera, and output the left and right rendering textures after the rendering operation is performed. Composite output module: Creates a shader program through a programmable rendering pipeline, binds the left and right eye rendering textures as inputs to the shader program, and merges the left and right eye images according to a preset blending strategy before outputting them to the frame buffer of the display terminal.

Images