Primitive image array generation method and system based on reverse ray tracing

Abstract

The application provides a primitive image array generation method and system based on reverse ray tracing, and belongs to the technical field of three-dimensional display. A projection array light field model is established, a virtual EIA plane is arranged in the model, and a source point and a direction of a sampling ray of each pixel in the EIA are determined. A reverse ray tracing algorithm is used to generate a ray for each pixel of the EIA, and the EIA is directly generated through GPU parallel calculation. Compared with the prior art, the prior art needs to previously arrange a plurality of sampling cameras for each projector and collect the rays of each view angle sampling ray emitted by the projector, and when the EIA is generated, the sampling ray corresponding to the view point-pixel-projector line is determined from the plurality of sampling rays corresponding to each projector. The application does not need to arrange the plurality of sampling cameras and does not need to perform a sampling ray searching process, all rays of the view point-pixel-projector can be determined at one time through a ray tracing plane, the calculation process is simplified, and real-time performance is improved.

Description

Technical Field

[0001] This invention relates to the field of three-dimensional display technology, and in particular to a method and system for generating primitive image arrays based on reverse ray tracing. Background Technology

[0002] Multi-projection light field 3D display technology is a glasses-free true 3D display technology. It does not require wearing glasses and can provide true 3D images with full parallax, realistic colors, and continuous viewpoints. It can also avoid the visual fatigue caused by abrupt changes in the parallax map in multi-viewpoint free stereoscopic displays, making it an effective solution for glasses-free 3D displays.

[0003] The existing method for generating the element image array (EIA) in a multi-projection light field 3D display system involves creating a virtual camera model on each projector or viewpoint. Then, based on the light field information (viewpoint image or light rays from certain pixels) acquired by each virtual camera at a specific viewpoint, and after determining the specific viewpoint, the system performs secondary sampling and pixel mapping on the numerous light rays acquired by the virtual camera at that viewpoint to calculate the element image array. While this method can obtain the element image array, it is computationally time-consuming, introduces redundant information during viewpoint image acquisition, and the pixel mapping process is time-consuming, limiting viewing performance and hindering the implementation of dynamic real-time display functionality, falling far short of the 24 frames per second requirement for real-time dynamic 3D display. Summary of the Invention

[0004] The purpose of this invention is to provide a method and system for generating primitive image arrays based on reverse ray tracing, which simplifies the calculation process of primitive image arrays and effectively improves the real-time performance of primitive image array generation.

[0005] To achieve the above objectives, the present invention provides the following solution:

[0006] A primitive image array generation method based on inverse ray tracing is applied to a multi-projection light field 3D display system. The multi-projection light field 3D display system includes a holographic scattering screen and a projector array consisting of multiple projectors at the same center distance from the holographic scattering screen. The primitive image array generation method includes the following steps:

[0007] The system parameter file of the multi-projection light field three-dimensional display system is analyzed to obtain the holographic scattering screen parameters and the projector array parameters;

[0008] The plane containing the holographic scattering screen and the plane containing the projector array are determined based on the parameters of the holographic scattering screen and the parameters of the projector array, respectively.

[0009] Establish a center point O with the center of the holographic scattering screen as the center point.c With the plane where the holographic scattering screen is located as X c O c Y c , passing through point O c And perpendicular to the X c O c Y c The straight line is Z c The internal space coordinate system X of the axis c Y c Z c ;

[0010] Generate virtual primitive image planes corresponding to each projector; the virtual primitive image plane corresponding to any projector is perpendicular to the projector and the center O. c The connections between virtual primitive image planes form a primitive image array; the size of each virtual primitive image plane depends on the imaging resolution of the corresponding projector.

[0011] A light field model of the projection array is established based on the plane where the projector array is located, the plane where the holographic scattering screen is located, the ray tracing plane, multiple virtual primitive image planes, and the internal space coordinate system; the ray tracing plane is parallel to the X-axis. c O c Y c The ray tracing plane is used to calculate the ray source point and ray direction;

[0012] Calculate the ray source point and ray direction of each pixel in the primitive image array on the ray tracing plane, and generate a ray for each pixel;

[0013] All rays are rendered in parallel using a reverse ray tracing algorithm to generate a primitive image array.

[0014] Optionally, the system parameter file of the multi-projection light field three-dimensional display system includes a configuration file and a coordinate file;

[0015] The configuration file includes the 3D model file name, texture map file name, projector array parameters, and holographic scattering screen parameters; the projector array parameters include the number of projectors, the projector distribution interval angle, the projector distribution radius, the equivalent pixel size of each projector, the projection ratio of each projector, and the imaging resolution of each projector; the holographic scattering screen parameters include the size of the holographic scattering screen;

[0016] The coordinate file includes the imaging center coordinates of each projector and the coordinates of the imaging center of each projector within the primitive image array.

[0017] Optionally, after parsing the system parameter file of the multi-projection light field 3D display system, the primitive image array generation method further includes:

[0018] Load the 3D model file and the texture map file according to the 3D model file name and the texture map file name;

[0019] The 3D model file and the texture map are parsed and stored in the container Meshfiles;

[0020] Based on the data in Meshfiles, a hierarchical bounding box for the 3D model is established; when a ray intersects with the outer bounding box of the hierarchical bounding box, it is then determined whether the ray intersects with the inner bounding box.

[0021] Optionally, the 3D model file is an .obj file or a .ply file, and the texture map is a .ppm file or an .hdr file.

[0022] Optionally, the computational primitive image array generates rays for each pixel based on the ray source point and ray direction in the ray tracing plane, specifically including:

[0023] For any pixel, determine the projector corresponding to the pixel based on the pixel's coordinates;

[0024] Calculate the coordinates of the pixel and the center point of the projector in the internal space X. c Y c Z c Inner space coordinates;

[0025] The direction vector of the line connecting the two points is calculated based on the internal space coordinates of the pixel and the internal space coordinates of the center point of the projector.

[0026] The intersection of the line connecting the pixel and the center point of the projector with the ray tracing plane is used as the ray source point;

[0027] The light source and the direction of the light are used to generate the light corresponding to the pixel.

[0028] Optionally, determining the projector corresponding to the pixel based on the pixel's coordinates specifically includes:

[0029] Using the center point of any projector as the origin, determine the coordinates of the pixel relative to the center point of the projector;

[0030] Whether a pixel is in the imaging area of ​​the projector is determined by the following formula:

[0031]

[0032] Where, x p,i Let y be the X-axis coordinate of the pixel relative to the center point of the i-th projector. p,iLet W be the Y-axis coordinate of the pixel relative to the center point of the i-th projector. i H represents the number of pixel columns in the i-th projector imaging region. i Let be the number of pixel rows in the i-th projector imaging area.

[0033] Optionally, the calculation of the pixel and the projector center point in the internal spatial coordinate system X... c Y c Z c The internal space coordinates in the text specifically include:

[0034] Calculate the relationship between the first projector and X based on the projector array parameters. c The included angle of the axis; the projector array parameters include the number of projectors, the projector distribution interval angle, the projector distribution radius, the projection ratio of each projector, and the imaging resolution of each projector;

[0035] Based on the index value of the projector, the distribution radius of the projector, and the first projector and the X c Calculate the internal spatial coordinates of the projector's center point by the angle formed by the axes;

[0036] The internal spatial coordinates of the center point of the virtual primitive image plane are calculated based on the distance between the projector and its corresponding virtual primitive image plane, the projector distribution interval angle, the projector distribution radius, and the coordinates of the center point of the projector.

[0037] The pixel size of the virtual primitive image plane is calculated based on the distance between the projector and its corresponding virtual primitive image plane, the projector throw ratio of the projector, and the imaging resolution of the projector.

[0038] Based on the coordinate difference between the pixel and the center point of the virtual primitive image plane, the pixel size of the virtual primitive image plane, the index value of the projector, the projector distribution interval angle, and the first projector and the X... c The angle between the axes is used to calculate the distance between the pixel and the center point of the virtual primitive image plane;

[0039] The internal space coordinates of the pixel are calculated based on the internal space coordinates of the center point of the virtual primitive image plane and the distance between the pixel and the center point of the virtual primitive image plane.

[0040] Optionally, using the double buffering technique in the OpenGL graphics library, while drawing a frame of primitive image array on the holographic scattering screen, the reverse ray tracing algorithm is used to render all the rays of the next frame of primitive image array to obtain the next frame of primitive image array.

[0041] Optionally, the primitive image array generation method further includes:

[0042] The primitive image array is divided according to the imaging area of ​​each projector, and then corrected using the affine transformation matrix of the corresponding projector.

[0043] Corresponding to the aforementioned primitive image array generation method, the present invention also provides a primitive image array generation system based on reverse ray tracing, wherein the primitive image array generation system, when run by a computer, executes the primitive image array generation method as described above.

[0044] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0045] This invention provides a method and system for generating primitive image arrays based on inverse ray tracing. By establishing a light field model of the projection array, setting a virtual EIA plane within the model, and using the ray tracing plane to determine the sampling rays for each pixel, the method generates rays for each pixel of the EIA using an inverse ray tracing algorithm. The EIA is directly generated through parallel GPU computation. Compared to existing technologies that require setting multiple sampling cameras for each projector, pre-collecting sampling rays from various viewpoints of the light emitted by the projector, and determining the sampling ray corresponding to the viewpoint-pixel-projector connection from numerous sampling rays from each projector's sampling cameras during actual EIA generation, this invention eliminates the need for setting up numerous sampling cameras and searching for sampling rays. The ray tracing plane can determine the ray formed by the viewpoint-pixel-projector connection in one step, simplifying the computation process and improving real-time performance. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 A schematic diagram illustrating the sampling principle of existing primitive image array generation methods;

[0048] Figure 2 This is a schematic diagram of the structure of the multi-projection light field three-dimensional display system in the primitive image array generation method provided in Embodiment 1 of the present invention;

[0049] Figure 3 This is a flowchart of the primitive image array generation method based on inverse ray tracing provided in Embodiment 1 of the present invention;

[0050] Figure 4 This is a schematic diagram of the hierarchical bounding box in the primitive image array generation method provided in Embodiment 1 of the present invention;

[0051] Figure 5 This is a schematic diagram of the internal space coordinate system in the primitive image array generation method provided in Embodiment 1 of the present invention;

[0052] Figure 6 This is a diagram showing the relationship between any pixel and the primitive image array in the primitive image array generation method provided in Embodiment 1 of the present invention;

[0053] Figure 7 This is a top view of the multi-projection light field three-dimensional display system in the primitive image array generation method provided in Embodiment 1 of the present invention;

[0054] Figure 8 This is a diagram showing the relationship between any pixel and any virtual primitive image plane in the primitive image array generation method provided in Embodiment 1 of the present invention;

[0055] Figure 9 This is a schematic diagram illustrating affine transformation correction in the primitive image array generation method provided in Embodiment 1 of the present invention;

[0056] Figure 10 This is a structural block diagram of a primitive image array generation system based on reverse ray tracing provided in Embodiment 2 of the present invention. Detailed Implementation

[0057] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0058] Existing methods for generating primitive image arrays involve two steps: first, sampling the 3D scene appropriately to obtain a specific viewpoint map; then, designing a corresponding light field mapping algorithm based on the principle of light field reconstruction, resampling the viewpoint map, coloring each projected ray according to its reconstructed target ray, and drawing the primitive image array.

[0059] The principle of sampling is as follows Figure 1As shown, the ray P1B projected by projector P1 can only be mapped by the ray BC1 captured by camera C1, which is located on its extension line. While camera C2, which can also sample point B, has the same spatial location as camera C1's sampled B, its sampled ray is BC2. BC2 has a different direction and wavelength than BC1, and is not the same ray. This ray can only be used to map the ray P2B projected by projector P2 on its reverse extension line. Therefore, the sampling camera must be positioned to cover every ray projected by each projector in the reconstruction area, located on the extension line of these rays, ensuring direct horizontal sampling is possible.

[0060] The mapping process involves finding the sampled ray corresponding to each ray projected by the projector in the sampled view image. For example, for ray P1B projected by projector P1, the corresponding ray needs to be found in the view image sampled by sampling camera C1. Calculating all the rays projected by a projector requires resampling from the view images obtained by multiple sampling cameras based on geometric relationships, and then drawing the primitive image. In other words, existing primitive image array calculation methods actually require resampling and mapping from the view images of multiple sampling cameras within the projection range of the projector to obtain the primitive image for each projector; it's not simply a matter of directly outputting the images from sampling cameras symmetrically arranged with the projector to the projector.

[0061] The purpose of this invention is to provide a method and system for generating primitive image arrays based on inverse ray tracing, which eliminates the need to pre-establish virtual cameras at each viewpoint to collect relationships and subsequently determine them among numerous rays, thus simplifying the calculation process of primitive image arrays and effectively improving the real-time performance of primitive image array generation.

[0062] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0063] Example 1:

[0064] This embodiment provides a method for generating primitive image arrays based on inverse ray tracing, applicable to, for example... Figure 2 The multi-projection light field 3D display system shown typically includes a holographic diffuser screen (DP) and a projector array consisting of multiple projectors at the same distance from the center of the holographic diffuser screen; such as Figure 3 The flowchart shown illustrates the steps involved in generating a primitive image array:

[0065] S1. Parse the system parameter file of the multi-projection light field three-dimensional display system to obtain the holographic scattering screen parameters and projector array parameters; generally, the system parameter file of the multi-projection light field three-dimensional display system includes a configuration file and a coordinate file.

[0066] The configuration file mainly includes the 3D model file name, texture map file name, projector array parameters, and holographic scattering screen parameters. The projector array parameters include the number of projectors, the projector distribution interval angle, the projector distribution radius, the equivalent pixel size of each projector, the projection ratio of each projector, and the imaging resolution of each projector. The holographic scattering screen parameters include the size of the holographic scattering screen. The 3D model file is an .obj or .ply file, and the texture map is a .ppm or .hdr file.

[0067] Specifically, in this embodiment, the system parameter files include a config.xml file and an EIcenter.csv file. The config.xml file contains the primitive image array resolution EIA_Resolution, the virtual primitive image resolution EI_Resolution, the number of projectors (projectorCount), the projector distribution interval (projectorInterval), the projector distribution radius (radius), the projector equivalent pixel size (pixelSize), the projector projection ratio (projectorThrowRatio), and the filenames of the primitive image center position data file, the 3D model file, and the texture map file. The EIcenter.csv file contains the coordinates of the primitive image center position.

[0068] The coordinate file includes the imaging center coordinates of each projector and the coordinates of the imaging center of each projector within the primitive image array.

[0069] After step S1, the primitive image array generation method may further include:

[0070] Load the 3D model file and texture map based on the 3D model file name and the texture map file name.

[0071] In this embodiment, an open-source function library is used to load the 3D model file and texture map. The 3D model is parsed into mesh data and stored in the string container Meshfiles. The texture map is stored in the optix::float3 type.

[0072] Based on the data in the Meshfiles, the 3D model is constructed as follows: Figure 4 The layered bounding boxes are shown; when a ray intersects with the outer bounding box of the layered bounding boxes, it is then determined whether the ray intersects with the inner bounding box. For example... Figure 4 In (a), there is a long triangle, an equilateral triangle, and a circle. The equilateral triangle and the circle belong to the same bounding box, and the long triangle is surrounded by another bounding box. The hierarchical bounding box acceleration algorithm is as follows: Figure 4 (b) When the ray intersects the bounding box of the long triangle, it is then determined whether it intersects with the long triangle; when the ray intersects the bounding box of the circle and the equilateral triangle, it is then determined whether it intersects with the circle or the equilateral triangle.

[0073] S2. Determine the plane where the holographic scattering screen is located and the plane where the projector array is located; In this embodiment, the plane where the holographic scattering screen is located and the plane where the projector array is located are determined according to the parameters of the holographic scattering screen and the parameters of the projector array, respectively.

[0074] S3. Establish an internal space coordinate system; to unify the coordinate system, establish a coordinate system with the center of the holographic scattering screen as the center point O. c With the plane where the holographic scattering screen is located as X c O c Y c , passing through point O c And perpendicular to the X c O c Y c The straight line is Z c The internal space coordinate system X of the axis c Y c Z c .

[0075] In this embodiment, a right-handed Cartesian coordinate system is first used to establish the coordinate axis as x. w ,y w ,z w The world coordinate system, where O w This is the origin of the world coordinate system. Then, as... Figure 5 As shown, the center of the holographic scattering screen is set as the origin O. c Move the viewpoint from point E to point O. c The resulting vector serves as the Z-axis of the internal space coordinate system. c axis, r in the figure ij For projector P n The emitted ray, after passing through the virtual primitive image plane, intersects the 3D model at point C, and its extension intersects the ray tracing plane ROP at point O. ij Combined with the top vector v of the internal space coordinate system up Calculate the orthonormal basis (u,v,w) of the internal spatial coordinate system, and establish the coordinate axes as X and V based on it. c ,Y c Z c Inner space coordinate system:

[0076]

[0077] Here, "×" represents the cross product of vectors, and normalize() calculates the unit vector of the vector within the parentheses.

[0078] S4. Generate virtual primitive image planes corresponding to each projector; the virtual primitive image plane corresponding to any projector is perpendicular to the projector and the center O. c The lines connecting the virtual primitive image planes form a primitive image array; the size of each virtual primitive image plane depends on the imaging resolution of the corresponding projector.

[0079] S5. Establish a projection array light field model; In this embodiment, a projection array light field model is established based on the plane where the projector array is located, the plane where the holographic scattering screen is located, the ray tracing plane, multiple virtual primitive image planes, and the internal space coordinate system; the ray tracing plane is parallel to X. c O c Y c The ray tracing plane is used to calculate the ray source point and ray direction.

[0080] In this embodiment, a projection array light field model is established based on the plane where the projector array is located, the plane where the holographic scattering screen is located, the ray tracing plane, multiple virtual primitive image planes, and the internal space coordinate system; the ray tracing plane is parallel to the X-axis. c O c Y c The plane, the distance between the ray tracing plane and the origin is equal to the distance along the Z-axis of the 3D model. c The absolute value of the direction, Z, is determined by the fact that the holographic scattering screen is located at the center of the three-dimensional model volume to ensure the display effect.

[0081] S6. Calculate the ray source point and ray direction of each pixel in the primitive image array on the ray tracing plane, and generate a ray for each pixel; specifically including:

[0082] S61. For any pixel, determine the projector corresponding to the pixel based on the pixel's coordinates; specifically including:

[0083] S611. Using the center point of any projector as the origin, determine the coordinates of the pixel relative to the center point of the projector. For example... Figure 6 As shown, the center point of the first projector I1 is taken as the origin (0,0), and the pixel p ij The coordinates relative to the center point of the first projector I1 can be expressed as (x p,i ,y p,i ).

[0084] S612. Determine whether the pixel is in the imaging area of ​​the projector according to the following formula:

[0085]

[0086] Where, x p,i Let y be the X-axis coordinate of the pixel relative to the center point of the i-th projector. p,i Let W be the Y-axis coordinate of the pixel relative to the center point of the i-th projector. i H represents the number of pixel columns in the i-th projector imaging region. i Let be the number of pixel rows in the i-th projector imaging area.

[0087] S62. Calculate the coordinates of the pixel and the center point of the projector in the internal space coordinate system X. c Y c Z c The internal space coordinates in the coordinates; specifically including:

[0088] S621. Calculate the relationship between the first projector and the X based on the projector array parameters. c The included angle α of the axes; such as Figure 7 The diagram shows a top view of a multi-projection light field 3D display system, where α represents the first projector and the X-ray beam. c The angle between the X and X axes; the projector array parameters include the number of projectors N, the projector distribution interval angle θ, the projector distribution radius, the throw ratio of each projector, and the imaging resolution of each projector. In this embodiment, the angle between the first projector and the X axis is calculated according to the following formula. c The included angle α of the axis:

[0089]

[0090] S622, based on the index value n of the projector, the distribution radius r of the projector, and the first projector and the X c Calculate the internal spatial coordinates of the projector's center point by taking the angle α between the axes. For any projector, the coordinates of the nth projector P can be obtained. n The internal space coordinates (x) n ,y n ,z n )for:

[0091]

[0092] S623. Based on the distance between the projector and its corresponding virtual primitive image plane, the projector distribution interval angle, the projector distribution radius, and the coordinates of the projector center point, calculate the internal spatial coordinates of the center point of the virtual primitive image plane. For example... Figure 8 As shown, due to the nth virtual primitive image plane I n Located at the center P of the projectorn With O c On the line connecting the points and at a distance d from the projector, it is easy to obtain I. n The coordinates of the center point are:

[0093]

[0094] S624. Based on the distance d between the projector and its corresponding virtual primitive image plane, the projector throw ratio T of the projector, and the number of pixel columns W in the imaging area of ​​the projector, calculate the pixel size of the virtual primitive image plane according to the following formula:

[0095]

[0096] S625. Based on the coordinate difference between the pixel and the center point of the virtual primitive image plane, the pixel size of the virtual primitive image plane, the index value of the projector, the projector distribution interval angle, and the first projector and the X... c The angle formed by the axes is used to calculate the distance between the pixel and the center point of the virtual primitive image plane according to the following formula:

[0097]

[0098] Where Δu and Δv are the differences between the pixel and the center point of the virtual primitive image plane in the primitive image coordinate system, respectively. Based on the pixel coordinates (u...) of the center point of the virtual primitive image plane... n ,v n From the pixel index ij of the pixel, we can obtain:

[0099]

[0100] S626. Calculate the internal space coordinates of the pixel based on the internal space coordinates of the center point of the virtual primitive image plane and the distance between the pixel and the center point of the virtual primitive image plane. In this embodiment, the internal space coordinates (x, y, z) of the pixel are calculated according to the following formula. ij ,y ij ,z ij ):

[0101]

[0102] S63. Calculate the direction vector of the line connecting the two points based on the internal spatial coordinates of the pixel and the internal spatial coordinates of the projector center point. Subtract the internal spatial coordinates of the pixel from the corresponding internal spatial coordinates of the projector center to obtain the direction vector d of the line connecting the two points. ij This gives us the equation of a straight line in space.

[0103] S64. The intersection of the line connecting the pixel and the center point of the projector with the ray tracing plane is taken as the ray source point O. ij .

[0104] S65, according to the light source point O ij and the direction of the light d ij Generate the light rays corresponding to the pixel.

[0105] S7. Use the reverse ray tracing algorithm to render all rays in parallel and generate a primitive image array.

[0106] Ray tracing algorithms are generally divided into forward ray tracing and backward ray tracing. Forward ray tracing refers to light rays originating from a light source and propagating through the scene to reach the image plane. While this method conforms to the physical laws of the real world, the light rays emitted by each light source travel in different directions, and most of these rays never reach the image plane, making the calculation meaningless for image rendering. Inverse ray tracing, on the other hand, only considers the rays we can see. It typically starts from the viewpoint, points to pixels, and intersects with objects in the scene. The first object that the ray hits represents the object that emitted the ray, and the sampled ray's direction is opposite to the direction of light rays in the real world.

[0107] The primitive image array generation method proposed in this embodiment uses inverse ray tracing. Figure 1 Taking point A as an example, the direction of the sampling ray should be opposite to the direction of the ray P1A projected by the projector, and the ray source should be on the observer's side. Therefore, a ray tracing plane ROP is designed to calculate the direction of ray P1A and find its intersection with the ray tracing plane in order to determine the source and direction of the sampling ray.

[0108] After determining the source and direction of the sampled rays, the hierarchical bounding box acceleration structure and the open-source ray tracing rendering engine Optix are used to calculate the radiance value or color of each ray in parallel and store it in the data structure EIABuffer of the projection array light field model. Until the radiance value of the ray corresponding to each pixel in the primitive image array has been calculated, all the data in the data structure EIABuffer constitutes a frame of primitive image array. All the data in the data structure EIABuffer is copied to the OpenGL output buffer and the data structure EIABuffer is refreshed.

[0109] Using the double buffering technique in the OpenGL graphics library, a frame of primitive image array is drawn on a holographic scattering screen, while simultaneously using the inverse ray tracing algorithm to render all the rays of the next frame of primitive image array, thus obtaining the next frame of primitive image array.

[0110] Furthermore, since most projectors are not perpendicular to the holographic scattering film, and mechanical errors are unavoidable during the installation of the projection array, it is necessary to correct the primitive image array. After generating the primitive image array, the primitive image array generation method also includes:

[0111] S8. The primitive image array is divided according to the imaging area of ​​each projector, and correction is performed using the affine transformation matrix of the corresponding projector. The correction principle is as follows: Figure 9 As shown, a projector P projects a square onto a perspective plane that is tilted towards it. If the perspective plane is rotated to a position M′ perpendicular to the projection direction, the intersection point of the same ray with the perspective plane changes. The same ray at M′ projects an irregular trapezoid. The shapes on the two perspective planes have undergone an affine transformation. Let the pixel coordinates of the same ray at M and M′ be (x, y) and (x′, y′) respectively, then they satisfy the following transformation relationship:

[0112]

[0113] The correction matrix required for each projector correction, which is a 3×3 matrix in the above formula, has 8 unknowns. During the calculation, the coordinates of 4 sets of corresponding points before and after the transformation are measured experimentally, and the affine transformation matrix is ​​calculated and corrected using the corresponding function in the open-source OpenCV.

[0114] Example 2:

[0115] The primitive image array generation method based on inverse ray tracing in Embodiment 1 of this invention can also be used with the aid of Figure 10 The architecture of the primitive image array generation system based on inverse ray tracing is shown below. Figure 10 As shown, the primitive image array generation system may include a parameter reading module, an internal space coordinate system establishment module, a virtual primitive image generation module, a projection array light field model establishment module, a pixel ray determination module, a ray parallel rendering module, and an affine transformation correction module; some modules may also have sub-units for implementing their functions, for example, the pixel ray determination module may also include a ray source point determination unit and a ray direction determination unit. Of course, Figure 10 The architecture shown is merely exemplary. In some implementations, other units can be added to some modules; additionally, when different functions need to be implemented, they can be omitted as needed. Figure 10 One or at least two components of the system shown.

[0116] Specific examples are used in this article, but the above description is only to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. Those skilled in the art should understand that the various modules or steps of the present invention described above can be implemented using general-purpose computer devices. Optionally, they can be implemented using computer-executable program code, and thus, they can be stored in a storage device for execution by a computer device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. The present invention is not limited to any specific combination of hardware and software.

[0117] Furthermore, those skilled in the art will recognize that, based on the principles of this invention, there will be variations in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as limiting the invention.

Claims

1. A method for generating primitive image arrays based on inverse ray tracing, applied to a multi-projection light field 3D display system, wherein the multi-projection light field 3D display system comprises a holographic scattering screen and a projector array consisting of multiple projectors at the same center distance from the holographic scattering screen; characterized in that, The method for generating the primitive image array includes: The system parameter file of the multi-projection light field three-dimensional display system is analyzed to obtain the holographic scattering screen parameters and the projector array parameters; The plane containing the holographic scattering screen and the plane containing the projector array are determined based on the parameters of the holographic scattering screen and the parameters of the projector array, respectively. Establish a center point O with the center of the holographic scattering screen as the center point. c With the plane where the holographic scattering screen is located as X c O c Y c , passing through point O c And perpendicular to the X c O c Y c The straight line is Z c The internal space coordinate system X of the axis c Y c Z c ; Generate virtual primitive image planes corresponding to each projector; the virtual primitive image plane corresponding to any projector is perpendicular to the projector and the center O. c The connections between virtual primitive image planes form a primitive image array; the size of each virtual primitive image plane depends on the imaging resolution of the corresponding projector. A light field model of the projection array is established based on the plane where the projector array is located, the plane where the holographic scattering screen is located, the ray tracing plane, multiple virtual primitive image planes, and the internal space coordinate system; the ray tracing plane is parallel to the X-axis. c O c Y c The ray tracing plane is used to calculate the ray source point and ray direction; Calculate the ray source point and ray direction of each pixel in the primitive image array on the ray tracing plane, and generate a ray for each pixel; All rays are rendered in parallel using a reverse ray tracing algorithm to generate a primitive image array.

2. The method for generating a primitive image array according to claim 1, characterized in that, The system parameter file of the multi-projection light field three-dimensional display system includes a configuration file and a coordinate file; The configuration file includes the 3D model file name, texture map file name, projector array parameters, and holographic scattering screen parameters; the projector array parameters include the number of projectors, the projector distribution interval angle, the projector distribution radius, the equivalent pixel size of each projector, the projection ratio of each projector, and the imaging resolution of each projector; the holographic scattering screen parameters include the size of the holographic scattering screen; The coordinate file includes the imaging center coordinates of each projector and the coordinates of the imaging center of each projector within the primitive image array.

3. The method for generating a primitive image array according to claim 2, characterized in that, After parsing the system parameter file of the multi-projection light field 3D display system, the primitive image array generation method further includes: Load the 3D model file and the texture map file according to the 3D model file name and the texture map file name; The 3D model file and the texture map are parsed and stored in the container Meshfiles; Based on the data in Meshfiles, a hierarchical bounding box for the 3D model is established; when a ray intersects with the outer bounding box of the hierarchical bounding box, it is then determined whether the ray intersects with the inner bounding box.

4. The method for generating a primitive image array according to claim 3, characterized in that, The 3D model file is an .obj file or a .ply file, and the texture map is a .ppm file or an .hdr file.

5. The method for generating a primitive image array according to claim 1, characterized in that, The computational primitive image array generates rays for each pixel based on the ray source point and ray direction in the ray tracing plane, specifically including: For any pixel, determine the projector corresponding to the pixel based on the pixel's coordinates; Calculate the coordinates of the pixel and the center point of the projector in the internal space X. c Y c Z c Inner space coordinates; The direction vector of the line connecting the two points is calculated based on the internal space coordinates of the pixel and the internal space coordinates of the center point of the projector. The intersection of the line connecting the pixel and the center point of the projector with the ray tracing plane is used as the ray source point; The light source and the direction of the light are used to generate the light corresponding to the pixel.

6. The method for generating a primitive image array according to claim 5, characterized in that, The step of determining the projector corresponding to the pixel based on the pixel's coordinates specifically includes: Using the center point of any projector as the origin, determine the coordinates of the pixel relative to the center point of the projector; Whether a pixel is in the imaging area of ​​the projector is determined by the following formula: Where, x p,i Let y be the X-axis coordinate of the pixel relative to the center point of the i-th projector. p,i Let W be the Y-axis coordinate of the pixel relative to the center point of the i-th projector. i H represents the number of pixel columns in the i-th projector imaging region. i Let be the number of pixel rows in the i-th projector imaging area.

7. The method for generating a primitive image array according to claim 5, characterized in that, The calculation of the pixel and the projector center point in the internal space coordinate system X c Y c Z c The internal space coordinates in the text specifically include: Calculate the relationship between the first projector and X based on the projector array parameters. c The included angle of the axis; the projector array parameters include the number of projectors, the projector distribution interval angle, the projector distribution radius, the projection ratio of each projector, and the imaging resolution of each projector; Based on the index value of the projector, the distribution radius of the projector, and the first projector and the X c Calculate the internal spatial coordinates of the projector's center point by the angle formed by the axes; The internal spatial coordinates of the center point of the virtual primitive image plane are calculated based on the distance between the projector and its corresponding virtual primitive image plane, the projector distribution interval angle, the projector distribution radius, and the coordinates of the center point of the projector. The pixel size of the virtual primitive image plane is calculated based on the distance between the projector and its corresponding virtual primitive image plane, the projector throw ratio of the projector, and the imaging resolution of the projector. Based on the coordinate difference between the pixel and the center point of the virtual primitive image plane, the pixel size of the virtual primitive image plane, the index value of the projector, the projector distribution interval angle, and the first projector and the X... c The angle between the axes is used to calculate the distance between the pixel and the center point of the virtual primitive image plane; The internal space coordinates of the pixel are calculated based on the internal space coordinates of the center point of the virtual primitive image plane and the distance between the pixel and the center point of the virtual primitive image plane.

8. The method for generating a primitive image array according to claim 1, characterized in that, Using the double buffering technique in the OpenGL graphics library, a frame of primitive image array is drawn on a holographic scattering screen, while simultaneously using the inverse ray tracing algorithm to render all the rays of the next frame of primitive image array, thus obtaining the next frame of primitive image array.

9. The method for generating a primitive image array according to claim 1, characterized in that, The primitive image array generation method further includes: The primitive image array is divided according to the imaging area of ​​each projector, and then corrected using the affine transformation matrix of the corresponding projector.

10. A primitive image array generation system based on inverse ray tracing, characterized in that, When the primitive image array generation system is run by a computer, it executes the primitive image array generation method based on reverse ray tracing as described in any one of claims 1 to 9.

Images