Computer generated hologram generating device, method and program

Abstract

To calculate the number of post-integration object points in advance for reducing the number of object points in a point cloud by integrating identical or adjacent object points, and to secure a memory size required for registering the object points used for light wave propagation calculation in advance, so that object point registration can be processed in parallel.SOLUTION: A 3D model input unit 10 arranges a 3D model used for hologram calculation in a virtual space. An object point acquisition unit 20 acquires object points on the 3D model for each element hologram on a hologram surface. An object point integration unit 50 registers the acquired object points in a memory area, where those object points having an identical or adjacent position are integrated into a single object point in advance. A post-integration object point estimation unit 30 estimates the number of post-integration object points for each frame. A memory area securing unit 40 secures the memory area in advance with a size corresponding to an estimation result of the number of object points after integration. The object point acquisition unit 20 and the object point integration unit 50 cooperate to acquire, integrate, and register each of the object points in parallel.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an apparatus, method, and program for generating a computer-generated hologram (CGH), and more particularly to an apparatus, method, and program for generating a computer-generated hologram that reduces the number of object points used in CGH calculation by integrating nearby object points. [Background technology]

[0002] Non-Patent Document 1 discloses a CGH calculation method that enables highly realistic rendering based on the CG ray tracing method. In Non-Patent Document 1, the hologram surface that records the propagating light waves from an object is divided into multiple small areas called element holograms as shown in Fig. 15, and the rendering result of the 3D scene is obtained for each element hologram based on the ray tracing method to obtain motion parallax.

[0003] However, when a hologram surface is divided into multiple element holograms, the point cloud on the 3D model visible from the center position of each element hologram must be acquired and stored using the ray tracing method, and the light wave distribution must be calculated. This poses the problem that the number of object points to be stored is at most the number of element holograms E × the number of light rays N, which results in an enormous memory size required for CGH calculations.

[0004] In order to solve these technical problems, the inventors of the present invention have filed a patent application (Patent Document 1) for an invention that reduces the number of object points by merging multiple object points that are close to each other in the point clouds acquired for each element hologram into one, thereby reducing the memory size required to hold the point clouds that are the subject of calculations of object light wave propagation. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent application No. 2021-068626 [Non-patent literature]

[0006] [Non-Patent Document 1] Ichikawa, T. Yoneyama and Y. Sakamoto, "CGH calculation with the ray tracing method for the Fourier transform optical system," Opt. Express 21, 32019-32031 (2013). [Non-Patent Document 2] William E.Lorensen,Harvey E. Cline: Marching Cubes: A high resolution 3D surface construction algorithm.In: Computer Graphics, Vol. 21, Nr. 4, July 1987. [Non-Patent Document 3] J.-D.Boissonnat, Geometric structures for three-dimensional shape representation. ACM Trans.Graph.3, 4 (1984) 266. [Non-Patent Document 4] Ryosuke Watanabe, Takamasa Nakamura, Masaya Mitobe, Yuji Sakamoto and Sei Naito, "Fast calculation method for viewpoint movements in computer-generated holograms using a Fourier transform optical system," Appl.Opt.58,G71-G83 (2019) Summary of the Invention [Problem to be solved by the invention]

[0007] According to Patent Document 1, the number of object points to be calculated for object light wave propagation can be reduced, and the memory size required to store them can be reduced. Specifically, the memory size required to secure a memory area for a point cloud in which the position coordinates and color information of each object point are registered, and a memory area for an occlusion table in which occlusion information is registered can be reduced.

[0008] On the other hand, if such point cloud integration processing could be performed in parallel between object points, it would be possible to reduce memory size and generate CGHs quickly. In order to perform parallel processing efficiently, it is desirable to reserve the necessary memory size in advance for the point cloud and occlusion table so that there is no excess or shortage during processing. However, if the number of object points reduced by the integration process and used to generate the CGH (number of object points after integration) is unknown, the required memory size cannot be grasped and must be overestimated, which prevents effective memory use. Moreover, if the required memory size is underestimated, it becomes necessary to expand the memory size during processing. Moreover, if the expansion amount is kept to a minimum with an eye on efficient memory usage, memory shortages will occur successively, and each time this occurs, it will be necessary to expand the memory area, preventing the full benefit of parallel processing, such as the high-speed generation of CGHs.

[0009] An object of the present invention is to solve the above-mentioned technical problems and to provide an apparatus, method, and program for generating a computer-generated hologram, which estimates in advance the number of object points to be included in the point cloud after integration, thereby reserving in advance a memory area of ​​a size necessary for registering object points that are the subject of calculation of object light wave propagation, and enabling object point integration processing to be performed in parallel between object points without causing a shortage of memory size, thereby generating a CGH at high speed. [Means for solving the problem]

[0010] In order to achieve the above object, the present invention is characterized in that an apparatus for generating a computer-generated hologram for each frame based on calculation of interference between an object beam and a reference beam on a hologram plane has the following configuration.

[0011] (1) The system is equipped with a means for placing a 3D model used for hologram calculation in a virtual space, a means for acquiring object points of the 3D model for each element hologram on the hologram surface, a means for registering the acquired object points in a memory area, and at that time, a means for integrating object points that are in the same position or close to each other into a single object point in advance, a means for estimating the number of object points after integration for each frame, a means for preserving the memory area with a size according to the number of object points after integration, and a means for calculating light wave propagation to each element hologram for each object point registered in the memory area, and the acquisition, integration, and registration of object points in the memory area are performed in parallel between object points.

[0012] (2) The means for estimating the number of integrated object points is configured to multiply the number of integrated object points of the previous frame by a predetermined ratio, and set the result as the number of integrated object points of the current frame.

[0013] (3) The means for acquiring object points acquires, as object points, points of intersection between light rays and the 3D model by ray tracing from each element hologram, and includes a means for calculating the density of light rays passing through the area in which the 3D model is placed in the virtual space, and the means for estimating the number of object points after integration sets the ratio to a smaller value for frames with a higher light ray density in the area in which the 3D model is placed.

[0014] (4) The apparatus includes a means for calculating the amount of movement of the 3D model between frames, and the means for estimating the number of object points after integration sets the ratio to a larger value as the amount of movement of the 3D model increases.

[0015] (5) If the allocated memory area becomes insufficient, some of the registered object points are randomly discarded. Effect of the Invention

[0016] According to the present invention, the following effects can be achieved.

[0017] (1) When integrating identical or adjacent object points to reduce the number of object points in a point cloud, the number of object points after integration is calculated in advance, and a memory area of ​​the size required to register the object points used in the light wave propagation calculation is reserved in advance, so that the occurrence of processing to expand the memory area during the process of performing the processing related to object point registration in parallel can be suppressed. As a result, it is possible to maximize the effect of performing the processing related to object point registration in parallel, that is, the speed-up of CGH generation.

[0018] (2) Since the number of integrated object points in the current frame is estimated based on the number of integrated object points in the previous frame, highly accurate estimation can be achieved with a small amount of calculation.

[0019] (3) The ratio by which the number of integrated object points in the previous frame is multiplied to estimate the number of integrated object points in the current frame is optimized based on the light density distribution in the area in which the 3D model is located, thereby making it possible to estimate the number of integrated object points in the current frame more accurately.

[0020] (4) The ratio by which the number of integrated object points in the previous frame is multiplied to estimate the number of integrated object points in the current frame is optimized based on the amount of movement of the 3D model, thereby making it possible to estimate the number of integrated object points in the current frame more accurately.

[0021] (5) If the allocated memory area becomes insufficient, registered object points are randomly discarded, so that the processing related to object point registration can be performed in parallel within the range of the allocated memory area without significantly degrading the quality of the CGH. [Brief description of the drawings]

[0022] [Figure 1] 1 is a functional block diagram showing a configuration of a computer-generated hologram generating device according to a first embodiment of the present invention. [Diagram 2] FIG. 13 is a diagram showing a method of integrating multiple object point light sources at the same or nearby positions and each registered in a different elementary hologram into a single common object point light source. [Diagram 3] FIG. 13 is a diagram showing an example of defining an integrated range of an object point light source. [Figure 4] FIG. 13 is a diagram showing an example of integrating multiple object point light sources in the same voxel into one. [Diagram 5] FIG. 13 is a diagram showing an example of a method for calculating occlusion. [Figure 6] FIG. 13 is a diagram showing an example of an occlusion table. [Figure 7] FIG. 11 is a functional block diagram showing the configuration of a computer-generated hologram generating device according to a second embodiment of the present invention. [Figure 8] This is a diagram showing an example of the distribution of rays in the ray tracing method used to obtain object points in a virtual space where a 3D model is placed. [Figure 9] FIG. 13 is a diagram showing an example of calculating a one-dimensional light ray density distribution by obtaining the number of passing light rays for each grid. [Figure 10] FIG. 13 is a diagram showing an example of the arrangement distribution of 3D models. [Figure 11] FIG. 13 is a diagram showing an example in which the light density is high near the central axis. [Figure 12] FIG. 13 is a diagram showing an example of defining a central region of a virtual space. [Figure 13] FIG. 13 is a diagram showing an example of a filter in which the probability distribution becomes large near the center. [Figure 14] FIG. 11 is a functional block diagram showing a configuration of a computer-generated hologram generating device according to a third embodiment of the present invention. [Figure 15] FIG. 13 is a diagram showing an example of obtaining motion parallax by obtaining rendering results of a 3D scene for each element hologram based on a ray tracing method. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0023] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. Fig. 1 is a functional block diagram showing the configuration of a computer-generated hologram generating device 1 according to a first embodiment of the present invention, and includes a 3D model input unit 10, an object point acquisition unit 20, a post-integration object point number estimation unit 30, a memory area allocation unit 40, an object point integration unit 50, a light wave propagation calculation unit 60, an interference fringe calculation unit 70, a CGH output unit 80, and a global memory 90 as main components. The object point integration unit 50 includes an occlusion determination unit 501.

[0024] The hologram is constructed by virtually arranging a number of element holograms in a two-dimensional space. A 3D model is input to the 3D model input unit 10 on a frame-by-frame basis, and a holographic video is output from the CGH output unit 80.

[0025] Such computer-generated hologram generating device 1 can be configured by implementing applications (programs) that realize each function on at least one general-purpose computer or server equipped with a CPU, ROM, RAM, storage, bus, interface, etc. Alternatively, it can be configured as a dedicated machine or a single-function machine in which part of the application is implemented as hardware or software.

[0026] The 3D model input unit 10 acquires a 3D model used for hologram calculation on a frame-by-frame basis and places it in the virtual space as mesh data. The 3D model may be directly input as mesh data such as an OBJ file, or voxel data may be converted into mesh data using a meshing algorithm such as the Martin Cube method described in Non-Patent Document 2 and then input. Alternatively, point cloud data may be converted into mesh data using the Boissonnat method described in Non-Patent Document 3 and then input.

[0027] The object point acquisition unit 20 acquires points on the 3D model that can be observed from the center of each element hologram as object points to be used in light wave propagation calculations by the ray tracing method disclosed in Non-Patent Document 1. The ray tracing method is a method that enables realistic rendering that takes into account hidden surface removal, shading, etc. by casting multiple light rays from a certain point and registering position information and color information of polygons of the 3D model that intersect with the light rays.

[0028] In this embodiment, an object point is placed at the intersection of a ray from the center of each element hologram and a 3D model using the ray tracing method, and its three-dimensional coordinates (x, y, z) and color information (R, G, B) are acquired and stored. Because the position from which the ray is projected differs for each element hologram, a different point group can be acquired for each element hologram, enabling hologram reproduction with smooth continuous parallax.

[0029] The object point integration unit 50 targets the object points acquired by the object point acquisition unit 20 for each element hologram, and as shown in Figure 2, integrates object points detected from different element holograms that have the same three-dimensional coordinates or are close in distance into a single common point, thereby reducing the number of object points used in the light wave propagation calculation.

[0030] In this embodiment, the three-dimensional space in which object points exist is divided into a cubic voxel grid as shown in Fig. 3, and multiple object points in the same voxel are integrated into one object point as shown in Fig. 4. For example, if a total of eight object points registered in eight selected element holograms are in the same voxel, the eight pieces of coordinate information and color information are represented by one piece of coordinate information and color information, so the amount of data relating to the registration of the coordinate information and color information is reduced to 1 / 8.

[0031] For the position information of one integrated point, the average value of the 3D coordinates of the integrated multiple points can be used as the integrated 3D coordinate. Alternatively, to reduce the amount of calculation, the barycentric coordinates of the voxel can be used as the integrated 3D coordinate. For the color information of one integrated point, the average value of the color information of the integrated multiple points can be used, or the color information of any of the integrated multiple points can be used.

[0032] In this embodiment, object points to be integrated are assumed to be close enough to be regarded as the same point when observed by the human eye, and the voxel size is automatically set based on the shortest distance l0 between the hologram surface and the surface of the 3D model from the distance l between the 3D model and the hologram surface, using the following formula (1), so that the length L of one side is observed to be equal to or less than the spatial resolution of human vision (1 / 60°). As a result, the voxel size is set based on the spatial resolution of human vision, so that it is possible to reduce the memory size without compromising the subjective image quality even when multiple point groups are integrated into one.

[0033]

number

[0034] In this embodiment, a rectangular parallelepiped voxel grid is used, but the present invention is not limited to this. For example, when considering the detection of object points using the ray tracing method in this embodiment, the distance between points tends to increase as the object points are farther away from the hologram surface. Therefore, a grid may be formed so that the size of the grid increases as the distance from the hologram surface increases.

[0035] One example of how this can be achieved is to consider the arrangement of object point light sources in a 3D polar coordinate system expressed by the distance l from the center of the hologram surface, the elevation angle θ, and the azimuth angle Φ, and form a grid by dividing each axis into fixed lengths or angles.

[0036] Alternatively, the grid may be represented as a viewing cone with a base parallel to the hologram plane, in which case the space may be divided by creating a grid of smaller viewing cones with a fixed length in the z-axis direction (the z-axis is perpendicular to the hologram plane) and a fixed angle of view within the viewing cone that contains the entire 3D region in which the points reside.

[0037] As will be described in detail later, in order to reserve in advance a memory size required for registering object points used in light wave propagation calculations and enable parallel processing of object point registration, the post-integration object point number estimation unit 30 estimates the post-integration object point number of the current frame based on the post-integration object point number in the past frame in advance for each frame before the object point integrating unit 50 integrates identical or adjacent object points to reduce the number of object points in the point cloud.

[0038] In this embodiment, the number of integrated object points in the current frame (f-th frame) is N ~ f The number of object points after integration in the previous frame (f-1th frame) is N' f-1 Multiplied by a certain percentage α, αN' f-1 It is estimated as follows.

[0039] In a CGH video, the 3D model does not move significantly between frames, so it is empirically recognized that the number of object points integrated by the object point integration unit 50 does not change significantly. Therefore, for example, the number of object points N' after integration in the previous frame f-1 By multiplying by a certain percentage α, the object point N in the current frame after integration can be obtained. ~ f can be estimated with high accuracy.

[0040] The ratio α may be a predetermined fixed value, or may be changed for each frame depending on the change in the model arrangement or the amount of movement of the model between frames, or a combination of both may be used, where the ratio α is a fixed value for frame sections where the amount of movement of the 3D model is small, and the ratio α is adaptively changed depending on the amount of movement for frame sections where the amount of movement is large.

[0041] It is desirable to set the ratio α to α>1 so that a larger memory area than the previous frame can be secured, taking into account errors in the estimated value. Also, since the object arrangement does not change significantly between frames, a value of α of at most about 2 is sufficient. For example, when displaying the upper body of a person in a remote conference using holography, the arrangement of the 3D captured person in the virtual space does not change significantly, so it is effective to set an appropriate fixed value for the ratio α based on past experience.

[0042] Before the object point integrating unit 50 starts the integration process, the memory area allocating unit 40 reserves a memory area of ​​a size necessary for the integration process of the object points on the global memory 90 based on the estimation result of the number of object points after integration. In the reserved memory area, a point cloud area M1 (hereinafter, may be referred to as a point cloud integration memory area), an occlusion table M2, and a grid number holding array M3 are reserved.

[0043] The point cloud area M1 registers the position coordinates and color information for each integrated object point. The occlusion table M2 registers occlusion information for each integrated object point, indicating from which element hologram the object point is visible and from which element hologram it is invisible. The grid number storage array M3 registers the number of the grid in which the object point exists.

[0044] In this embodiment, by reserving memory areas M1, M2, and M3 for the integrated point clouds in advance, as described in detail below, object points are acquired for each element hologram through cooperative operation of object point acquisition unit 20 and object point integration unit 50, and object points located within the same grid are integrated and retained, and the process of generating an occlusion table can be performed in parallel for each element hologram and between element holograms.

[0045] In this embodiment, when multiple object points are stored in the same grid, they are integrated. A consecutive grid number is assigned to each grid in advance as an identifier, and an occlusion table M2 is also created at the time of integration to manage from which element hologram each object point after integration can be observed.

[0046] When calculating the CGH for the first frame, it is not possible to estimate the number of object points after integration in previous frames. Therefore, as in the conventional technology, the process of acquiring object points, integrating nearby object points, and registering them in the point cloud area M1 and the occlusion table M2 is repeated serially for each object point.

[0047] From the second frame onwards, a memory area of ​​a size corresponding to the number of integrated object points in the current frame estimated based on the number of integrated object points in the previous frame is secured in global memory 90, and the acquisition of object points, integration of nearby object points, and registration in point cloud area M1 and occlusion table M2 are processed in parallel up to the number of element holograms E × number of rays N.

[0048] The memory area reservation unit 40 basically reuses the memory area used in the previous frame. If the memory area is insufficient, it may reserve additional memory area to make up for the shortage, or it may repeat releasing and reserving memory area for each frame.

[0049] When the memory area is secured, the object point acquisition unit 20 virtually irradiates light rays from the center point of each element hologram simultaneously using a ray tracing method, and places object points at the intersections of each light ray and the 3D model by parallel processing.

[0050] The object point integration unit 50 calculates the storage destination grid number for each placed object point based on its position information by parallel processing. If the storage destination grid number does not exist in the grid number holding array M3, it is added, and the position information and color information of the object point are newly registered and recorded in the point cloud area M1. Furthermore, for each object point, an occlusion determination unit 501 determines from which element hologram the object point is visible and from which element hologram the object point is invisible, as shown in Fig. 5, and the determination result is registered in the occlusion table M2.

[0051] In the occlusion table M2, occlusion information is managed in table format for each combination of object point and element hologram, as shown typically in Fig. 6. In this embodiment, the object point is used as an index, and a "1" is set in the corresponding cell of the element hologram where the object point can be observed, and a "0" is set in the corresponding cell of the element hologram where the object point cannot be observed.

[0052] On the other hand, if the storage destination grid number exists in the grid number holding array M3, the object point already registered in the point cloud area M1 is integrated with the current object point, and the point cloud area M1 and the occlusion table M2 are updated.

[0053] In this embodiment, the same grid number is calculated at the same timing as the storage destination of the object point, and it is possible that the same grid number and object point are stored in duplicate, though this is very rare. To prevent this, a function may be provided to check for overlapping grid numbers after the integration process and delete the duplicated data. Specifically, the point cloud region M1, occlusion table M2, and grid number storage array M3 are sorted by grid number, and if adjacent data have the same grid number, one of them is deleted.

[0054] In addition, if the actual number of integrated object points exceeds the estimated value and the memory allocated in advance is insufficient, the number of integrated object points may be reduced to less than the estimated value by randomly deleting some of the integrated object points. Since the optically reconstructed image of a CGH can be considered as a collection of many point light sources, the shape of the reconstructed image does not change even if some of them are randomly deleted. This makes it possible to perform processes related to object point registration in parallel within the range of the allocated memory size without significantly deteriorating the image quality.

[0055] Instead of randomly deleting some of the post-integration object points, the insufficient memory size may be sequentially secured by additional memory, or when memory becomes insufficient, the post-integration point cloud may be discarded and a larger memory area may be re-secured, after which the object point acquisition unit 20 and the object point integration unit 50 may perform the above process again from the beginning.

[0056] In this embodiment, the number of integrated object points in the current frame is estimated from one past frame, but the present invention is not limited to this, and the estimation may be performed by referring to the number of integrated object points in multiple past frames. For example, by adding the standard deviation to the maximum value of the number of integrated object points in multiple past frames, it becomes possible to obtain a highly accurate estimation result from the tendency of change in the number of integrated object points in the past frames.

[0057] The light wave propagation calculation unit 60 performs light wave propagation calculation from each object point to an element hologram that can observe the object point, based on the information of the point cloud region M1 and the occlusion table M2. Note that the point source method disclosed in Non-Patent Document 2 can be adopted for the light wave propagation calculation, and the calculation formula is expressed by the following formulas (2) and (3).

[0058]

number

[0059]

number

[0060] Here, (x, y) indicates the pixel position on the hologram surface through which the light wave propagates, and si(x, y) is the light wave distribution on the hologram surface propagated from each object point pi. N' is the number of object points after integration, Ai is the brightness of each object point pi, ri is the distance between each object point pi and pixel (x, y) on the hologram surface, and O(i, e) is the value of the occlusion table corresponding to each object point pi and element hologram e, indicating whether the object point pi is observable in the element hologram e. k represents the wave number calculated from the wavelength of light, and u(x, y) is the calculated object light wave distribution.

[0061] The interference fringe calculation unit 70 has a function of performing interference calculation by inserting a reference light wave as a computer simulation for the object light wave u(x, y) on the hologram surface, and outputting it as an image. The reference light in this embodiment uses a convergent spherical reference light wave. The method disclosed in Non-Patent Document 4 can be applied to the interference calculation using the convergent spherical reference light wave.

[0062] The complex amplitude distribution R(x, y) of the light wave when this convergent spherical reference light wave is propagated onto the hologram surface is expressed by the following equation (4): Here, Ro is the intensity of the reference light, and r is the distance from the position of the reference light to the position (x, y) on the hologram surface.

[0063]

number

[0064] The reference light is not limited to the above formula (4), but may be a simple spherical wave reference light as expressed by the following formula (5), or a parallel wave reference light as expressed by the following formula (6), where β in formula (6) is the angle of incidence of the reference light to the hologram surface.

[0065]

number

[0066]

number

[0067] The interference between the reference light wave and the object light wave is expressed by the following equation (7): where I(x, y) is the luminance distribution of the CGH.

[0068]

number

[0069] Finally, this brightness distribution is output as an image. For example, the interference fringes are normalized to the range of 0-255 and output as an image.

[0070] According to this embodiment, when integrating identical or adjacent object points to reduce the number of object points in a point cloud, the number of object points after integration is calculated in advance, and a memory area of ​​a size required for registering object points used in light wave propagation calculations is reserved in advance, so that it is possible to prevent the occurrence of processing to expand the memory area during the process of performing the processes related to object point registration in parallel. As a result, it is possible to maximize the effect of speeding up CGH generation, which is obtained by performing the processes related to object point registration in parallel.

[0071] FIG. 7 is a functional block diagram showing the configuration of the main parts of a computer-generated hologram generating device 1 according to a second embodiment of the present invention. Since the same reference numerals as those described above denote the same or equivalent parts, their explanation will be omitted.

[0072] This embodiment is characterized in that the 3D model input unit 10 is equipped with a ray density distribution calculation unit 101, and the coefficient α used by the post-integration object point number estimation unit 30 when estimating the post-integration object point number of the current frame based on the post-integration object point number of the past frame is adaptively determined for each frame in accordance with the relationship between the density distribution of rays used in the ray tracing method and the arrangement of the 3D model.

[0073] Fig. 8 shows the distribution of light rays in the ray tracing method used to acquire object points in a virtual space in which a 3D model is placed, and it can be seen that the light rays are concentrated in an area close to the central axis of the hologram surface. Since the probability of integrating object points is higher near the central axis where the light ray density distribution is high than in other areas, it is desirable to make the ratio α relatively small for frames in which the 3D model is placed in an area with high light ray density and relatively large for frames in which the 3D model is placed in an area with low light ray density.

[0074] Therefore, in this embodiment, a light ray density distribution calculation unit 101 is provided in the 3D model input unit 10, and the coefficient α used by the post-integration object point number estimation unit 30 when estimating the post-integration object point number of the current frame based on the post-integration object point number of the past frame is adaptively set based on the calculation result of the light ray density distribution calculation unit 101.

[0075] Since the ray density distribution is an indicator of how many rays of the ray tracing method pass through each part of the 3D model, it can be calculated in advance by evenly dividing the virtual space in which the 3D model is placed into a voxel grid and determining the number of rays passing through each grid (number of ray passes).

[0076] In this embodiment, the number of passing rays is obtained for each grid in which a 3D model of the virtual space exists, and a one-dimensional light ray density distribution as shown in Fig. 9 is calculated. Note that this voxel grid is different from the voxel grid used by the object point integrating unit 50 to identify object points to be integrated, and is used only for calculating the light ray density distribution, so the voxel size does not need to be as small as the voxel size for identifying object points to be integrated. Therefore, the light ray density distribution can be calculated without requiring an additional large memory area.

[0077] In this embodiment, when the light ray distribution density is high, α can be set to 1.2, and when the light ray distribution density is low, α can be set to 2.0. On the other hand, if it is known by some method that the number of points after reduction will be smaller than that of the previous frame, the ratio α may be set without being limited to the above range. When comparing the magnitude of light ray density between frames, a representative value such as the median or average value of the light ray density distribution may be compared, or an integral value of the distribution may be compared.

[0078] Note that the light ray density distribution may be replaced by the distribution of the model arrangement without calculating it precisely. The model arrangement distribution can be found by, for example, calculating the two-dimensional model arrangement distribution by labeling each 3D model that intersects with the light rays of the ray tracing method from the central coordinates of the hologram, as shown in Fig. 10. This is equivalent to roughly estimating the light ray density distribution based on how much the model arrangement distribution depends on the central axis, since the light rays of the ray tracing method used to acquire the point cloud are concentrated in the area close to the central axis of the hologram surface in virtual space, as shown in Fig. 11.

[0079] When replacing the light density distribution with the model arrangement distribution, the central region of the virtual space is defined by the following equation (8) as shown in FIG. 12, and the existence probability of the 3D model in the central region is represented by the maximum value, average value, or integral value.

[0080]

number

[0081] By decreasing the ratio α when the representative value is higher than in the previous frame, and increasing the ratio α when the probability of the surrounding presence increases, the optimal ratio α can be determined in the same way as when the light density distribution is used.

[0082] As shown in an example in FIG. 13, the similarity of the probability distribution with a filter designed so that the values ​​near the center are large may be calculated, and the model arrangement distribution may be obtained based on the calculation results.

[0083] According to this embodiment, the ratio α for calculating the number of integrated object points in the current frame based on the number of integrated object points in the previous frame can be optimized based on the light density of the area in which the 3D model is located, making it possible to estimate the number of integrated object points more accurately.

[0084] FIG. 14 is a functional block diagram showing the configuration of a computer-generated hologram generating device 1 according to a third embodiment of the present invention. Since the same reference numerals as those described above denote the same or equivalent parts, their explanation will be omitted.

[0085] This embodiment is characterized in that the 3D model input unit 10 is equipped with a movement amount calculation unit 102, and the coefficient α used by the post-integration object point number estimation unit 30 when estimating the post-integration object point number of the current frame based on the post-integration object point number of the past frame is adaptively set for each frame based on the movement amount of the 3D model.

[0086] When arranging a 3D model in a virtual space, the movement amount calculation unit 102 calculates the change in the center of gravity position of the 3D model from the previous frame as the movement amount. Since the change amount between frames in the number of integrated object points differs depending on the movement amount between frames, the estimation accuracy can be improved by using the movement amount of the model to estimate the number of integrated object points.

[0087] Generally, the greater the movement amount of the 3D model between frames, the greater the change in the 3D model arrangement between frames, and therefore the greater the change in the number of object points after integration. Therefore, in this embodiment, the ratio α is optimized by increasing the ratio α as the movement amount of the 3D model between frames increases.

[0088] According to this embodiment, the ratio α for calculating the number of integrated object points in the current frame based on the number of integrated object points in the previous frame is obtained based on the amount of movement of the model, so that the number of integrated object points can be estimated more accurately.

[0089] According to each of the above embodiments, high-quality CGH can be generated in a short time and can be provided in real time via communication infrastructure, making it possible to provide a variety of entertainment to many people regardless of geographic or economic disparity. As a result, it is possible to contribute to Goal 9 "Build resilient infrastructure and promote inclusive and sustainable industrialization" and Goal 11 "Make cities inclusive, safe, resilient and sustainable" of the Sustainable Development Goals (SDGs) led by the United Nations. [Explanation of symbols]

[0090] 10...3D model input unit, 20...object point acquisition unit, 30...post-integration object point number estimation unit, 40...memory area allocation unit, 50...object point integration unit, 60...light wave propagation calculation unit, 70...interference fringe calculation unit, 80...CGH output unit, 90...global memory, 101...light ray density distribution calculation unit, 102...movement amount calculation unit, 501...occlusion table determination unit

Claims

1. 1. An apparatus for generating a computer-generated hologram for each frame based on interference calculation between an object beam and a reference beam on a hologram plane, comprising: A means for placing a 3D model used for hologram calculation in a virtual space; and A means for obtaining object points of a 3D model for each element hologram of the hologram surface; a means for registering the acquired object points in a memory area, and at that time, integrating object points having the same position or close positions into a single object point in advance; A means for estimating the number of object points after integration for each frame; a means for reserving in advance the memory area having a size corresponding to the number of object points after integration; a means for calculating light wave propagation to each element hologram for each object point registered in the memory area, A computer generated hologram generating device, characterized in that the acquisition, integration and registration of the object points in the memory area are performed in parallel between object points.

2. The memory area is a memory area for integrating point clouds in which position information and color information of each object point are registered; 2. The computer-generated hologram generating device according to claim 1, further comprising an occlusion table in which occlusion information is registered for each combination of an object point and an element hologram.

3. 3. The computer-generated hologram generating device according to claim 1, wherein the means for estimating the number of integrated object points estimates the number of integrated object points in the current frame based on the number of integrated object points in a previous frame.

4. 4. The computer-generated hologram generating device according to claim 3, wherein the means for estimating the number of integrated object points estimates the number of integrated object points in the current frame based on the number of integrated object points in the previous frame or a statistical value of the number of integrated object points in the past few frames.

5. 5. The computer-generated hologram generating device according to claim 3, wherein the means for estimating the number of integrated object points is a value obtained by multiplying the number of integrated object points of a past frame by a predetermined ratio, the number of integrated object points of a current frame being the number of integrated object points of the past frame.

6. the means for acquiring the object point acquires an intersection point between a light ray and a 3D model as an object point by ray tracing from each element hologram; A means for calculating a density of a light ray passing through an area in which the 3D model of the virtual space is placed, 6. The computer-generated hologram generating device according to claim 5, wherein the means for estimating the number of object points after integration sets the ratio to a smaller value for a frame having a higher light ray density in an area in which a 3D model is placed.

7. the means for calculating the light ray density divides the virtual space into a number of small regions and calculates the number of light rays passing through each of the small regions; 7. The computer-generated hologram generating device according to claim 6, wherein the ratio is determined based on the number of light rays in a small region including a 3D model.

8. means for calculating frame-to-frame displacement of a 3D model; 6. The computer-generated hologram generating device according to claim 5, wherein the means for estimating the number of object points after integration sets the ratio to a larger value as the amount of movement of the 3D model increases.

9. a means for calculating a similarity between a placement probability distribution of the 3D model in the virtual space and a filter designed in advance so that the probability distribution near the center of the virtual space is large; 6. The computer-generated hologram generating device according to claim 5, wherein the means for estimating the number of object points after integration sets the ratio to a smaller value as the degree of similarity increases.

10. 10. The computer-generated hologram generating device according to claim 2, wherein when the secured memory area for point cloud integration becomes insufficient, a part of the registered object points is randomly discarded.

11. 11. The computer-generated hologram generating device according to claim 1, wherein the integrating means divides the virtual space into a number of small regions and integrates object points located within the same small region.

12. A method for generating a computer-generated hologram for each frame by a computer based on interference calculation between an object beam and a reference beam on a hologram plane, comprising: The 3D model used for hologram calculation is placed in the virtual space, Obtain the object points of the 3D model for each element hologram on the hologram surface, The acquired object points are registered in a memory area, and object points that are in the same or close positions are integrated into a single object point in advance. Estimate the number of object points after integration for each frame. The memory area is allocated in advance with a size corresponding to the number of object points after integration; Calculating light wave propagation to each element hologram for each object point registered in the memory area; A computer-generated hologram generating method, comprising the steps of: acquiring and integrating the object points and registering them in the memory area in parallel between the object points.

13. 13. The method of claim 12, wherein the number of integrated object points in the current frame is estimated based on the number of integrated object points in the previous frame.

14. A program for generating a computer-generated hologram for each frame based on interference calculation between an object beam and a reference beam on a hologram plane, The procedure for placing the 3D model used for hologram calculation in the virtual space, A procedure for obtaining object points of a 3D model for each element hologram on the hologram surface; A step of registering the acquired object points in a memory area, and at that time, integrating object points having the same position or close positions into a single object point in advance; A step of estimating the number of object points after integration for each frame; a step of allocating in advance the memory area having a size according to the number of object points after integration; a step of calculating light wave propagation to each element hologram for each object point registered in the memory area; A computer-generated hologram generating program, characterized in that the acquisition, integration and registration of each object point in the memory area are performed in parallel between object points.

15. 15. The computer-generated hologram generating program according to claim 14, wherein in the step of estimating the number of object points after integration, the number of object points after integration in a current frame is estimated based on the number of object points after integration in a previous frame.

Images