Computer-generated hologram generation device and program

Abstract

To provide a computer-generated hologram generation device capable of minimizing the amount of computation even when a drawing range is wide, such as in the case of a spherical image.SOLUTION: A computer-generated hologram generation device is provided, for generating a computer-generated hologram by performing interference computations between object light from point light sources on an object surface and reference light on a hologram plane, the device being configured to perform the interference computations by allocating denser point light sources to regions expected to attract more attention of users.SELECTED DRAWING: Figure 5

Description

Technical Field

[0001] The present invention relates to a computer-generated hologram generation device and a program capable of suppressing the amount of calculation even when the drawing range is wide as in the case of a full-sphere image, for example.

Background Art

[0002] Holography is a three-dimensional display technology that records and reproduces interference fringes between light from an object (object light) and reference light from the outside based on the interference and diffraction phenomena of light. In holography technology, the wave of light emitted from an object is interfered with reference light irradiated from a light source such as a laser, and the object light is recorded as interference fringes (hologram) on the hologram surface. By applying reproduction illumination light to these interference fringes, the light at the time of recording can be reproduced. Since it can faithfully reproduce the light emitted from an object, it is regarded as an ideal three-dimensional display technology that satisfies all the physiological factors of human three-dimensional perception.

[0003] Among this holography technology, computer-generated hologram (CGH) is a technology that simulates the calculation of light wave propagation and interference required for hologram calculation inside a computer and outputs interference fringes as electronic data represented by an image. Compared with analog holograms recorded on photographic plates, etc., it has advantages such as not requiring a complicated optical system for photography and being able to easily create a moving image by switching CGHs to be displayed on a spatial light modulator (SLM) one after another, and thus is expected to be applied to next-generation TVs and XR (a general term for VR, AR, MR (mixed reality), etc.) devices such as VR / AR (virtual reality / augmented reality).

[0004] On the one hand, as an issue of CGH, there exists a problem that the processing time for calculation is extremely long. In particular, as a calculation method of CGH, the "point light source method" is well-known, in which an object to be recorded is defined by a set of a large number of point light sources (3D point cloud data), and the propagation of light emitted from each of these point light sources is calculated to record an object light wave. However, since it is necessary to perform light wave propagation simulations from a large number of point clouds, this calculation processing time is extremely long. Also, in order to obtain a sufficient viewing field during hologram playback, an SLM with a pixel pitch of about 1 μm close to the wavelength of visible light is required. For example, the number of pixels required to realize a liquid crystal of 1 cm × 1 cm on the order of 1 μm is 10,000 × 10,000 pixels. Furthermore, assuming that the size of the liquid crystal is increased, in the future, liquid crystals in the hundreds of K to thousands of K level will be required. Considering that the calculation time of the point light source method is generally "the number of point light sources × the number of pixels on the hologram surface", it is difficult to perform real-time calculations with a huge number of points as input.

[0005] There are various conventional techniques for dealing with the issue of CGH where the calculation processing time becomes extremely long.

[0006] In Non-Patent Document 1, for the purpose of VR (XR) applications, etc., when a wearer slightly moves the position or orientation of the viewpoint, a method of calculating at high speed by applying a conversion to the object light wave distribution of the frame before movement for interference fringes corresponding to the situation is disclosed. The characteristics of this conversion are useful, for example, in an XR device capable of holographic viewing of an HMD (Head Mounted Display) type, when it is desired to display reproduced images from slightly different viewpoint positions for users with various interpupillary distances.

[0007] In addition, in the element hologram method used in Non-Patent Document 1 and the like, a single rectangular hologram surface is divided into rectangular small regions (element holograms), and a method has been proposed to calculate natural motion parallax at high speed by discretely measuring whether each point light source can be observed for each element hologram viewpoint. As described above, in the point light source method which is not the element hologram method, ray tracing is performed for each pixel of the hologram surface, visible point light sources are obtained therefrom, and the propagation calculation of object light for each pixel is performed only from the visible point light sources. As a result, interference fringe calculation considering the occlusion relationship of the object becomes possible, but this ray tracing for each pixel requires an extremely large amount of calculation. Therefore, in the element hologram method, ray tracing is roughly performed for each element hologram, and object light calculation is performed for each pixel from the visible point light sources for each element hologram. As schematically shown in FIG. 1 for the element hologram method, drawing results G1, ..., G6 (strictly speaking, not limited to the drawing results, but the results of ray tracing from the representative points of h1 to h6) of the object OB corresponding to six element hologram viewpoints h1, ..., h6 are obtained. For example, for the interference fringe calculation of h1, calculation is performed using the point light sources obtained in the result of G1 (and the occlusion relationships obtained thereby), and for h2 to h6, interference fringes are calculated in the same manner, and the interference fringes of the entire hologram surface can be obtained. From the viewpoint that the element hologram method also realizes high-speed calculation while maintaining the video quality, it is an effective light wave propagation calculation method.

Prior Art Documents

Non-Patent Documents

[0008]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0009] In the conventional CGH, no consideration has been given to coping with the increase in the amount of calculation when using CGH in a full - sphere image or a full - sphere video. Fig. 2 schematically shows the processing framework and problems when using CGH in a full - sphere video.

[0010] The VR video viewing using an HMD that is currently widely marketed is realized by receiving (or rendering) a 360 - degree full - sphere video viewed from a certain viewpoint on the device side, and cutting out and displaying a part of the video according to the user's posture such as the direction of the face. One reason why existing VR experiences take such a form is that "in order to realize natural video viewing without VR sickness, it is necessary to display a video adjusted to the position and orientation with a very short delay with respect to the movement of the face." Considering such requirements, even when using the high - speed light wave propagation calculation proposed in Non - Patent Document 1 and the like for an HMD, it is practically desirable to transmit the full - sphere object light wave data to the receiving device, and then cut out some object light waves according to the user's position and orientation on the device side and reproduce the holography. On the other hand, for the calculation of such a wide - range full - sphere object light wave, from the perspective that "CGH requires a much larger number of pixels (pixel density) than conventional displays," there is a problem of causing a further huge increase in the calculation time.

[0011] That is, as shown in Fig. 2, a 3DCG model that can operate in real - time is prepared (processing p1), the position and orientation information of the user's viewpoint that changes with time in real - time is obtained (processing p2), the interference fringes of the 3DCG model in the full - sphere are calculated at this position and orientation (processing p3), and the result is displayed in real - time (processing p4) to realize holography viewing with an HMD. However, there is a problem that the calculation in processing p3 becomes huge in the full - sphere.

[0012] Here, in existing VR-related holography technologies, the inventions mainly involve calculation theories such as how to approximately calculate the light wave propagation on a planar hologram surface (SLM) for a curved surface, and technologies targeting such all-spherical holography have not been proposed.

[0013] In view of the problems of the above prior art, an object of the present invention is to provide a computer-synthesized hologram generation device and a program capable of suppressing the amount of calculation even when the drawing range is wide, such as in the case of all-spherical images.

Means for Solving the Problems

[0014] To achieve the above object, the present invention provides a computer-synthesized hologram generation device that generates a computer-synthesized hologram by performing interference calculation between object light and reference light from point light sources on the object surface on a hologram surface, wherein the interference calculation is performed by assigning denser point light sources to regions determined to have a greater degree of user fixation. Also, it is characterized by being a program that causes a computer to function as the computer-synthesized hologram generation device.

Effects of the Invention

[0015] According to the present invention, by setting point light sources with a density corresponding to the degree of user fixation, the amount of calculation when generating a computer-synthesized hologram can be suppressed.

Brief Description of the Drawings

[0016]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Mode for Carrying Out the Invention

[0017] FIG. 3 is a functional block diagram of a CGH generation device 10 according to an embodiment. As shown in the figure, the CGH generation device 10 includes a user position and orientation acquisition unit 1 that acquires the position and orientation of a user, a 3D model input unit 3 that acquires 3D model data, an adaptive element hologram division unit 2 that determines a method for dividing element holograms based on the acquired position and orientation of the user, a point light source acquisition unit 4 that can adaptively acquire point light sources at intervals corresponding to the positions of each element hologram on the celestial sphere, a light wave propagation calculation unit 5 that calculates the object light wave of each element hologram surface based on the acquired point light sources, and an interference calculation unit 6 that calculates and reproduces the interference fringes of the object light wave and the reference light wave. Note that, in another embodiment, the point light source acquisition unit 4 may perform uniform point light source acquisition similar to existing methods instead of the above-described adaptive point light source acquisition.

[0018] As schematically shown in FIG. 4, which shows an aspect of calculation speedup realized by the CGH generation device 10 of the present embodiment, in the present embodiment, based on the concept of the element hologram method, by utilizing the characteristics of the user experience in a full-sphere type HMD, the amount of target data required for calculation can be reduced, thereby achieving calculation speedup.

[0019] That is, regarding the elemental hologram, the elemental hologram assigned to an area important for the user experience is made to cover a narrow range densely, and the elemental hologram assigned to an area not so important for the user experience is made to cover a wide range sparsely, so that the degree of impairment of the user experience can be minimized, the quality of the user experience can be maintained, and the calculation speed can be increased. (Basically, the elemental hologram division is performed by dividing the hologram plane composed of a certain fixed pixel density into regions and acquiring point light sources in units of those regions, and the pixel density does not change. However, according to this embodiment, the number of pixels for each elemental hologram can be made different.)

[0020] Specifically, in video viewing using an HMD, the quality of the content mainly corresponding to the line-of-sight movement in the directly front and horizontal directions greatly affects the user experience quality, and the quality of the content at the very top (ceiling) and very bottom (foot area) of the celestial sphere has little effect on the user experience quality. This is due to the fact that it is rare for the user to carefully view the images of the ceiling and the foot area. Also, compared to the front of the content, the quality on the back side also less affects the experience quality for the same reason. By taking advantage of this characteristic of the user experience unique to the HMD, as schematically shown as "Embodiment 1" in FIG. 4, by allowing deterioration of the quality of the content around the ceiling or the foot area, the calculation of light wave propagation can be accelerated.

[0021] Similarly, as shown as "Embodiment 2" in FIG. 4, when performing CG rendering, by densely allocating elemental holograms to the area where the foreground 3D model exists and sparsely allocating elemental holograms to the area where it does not exist, it is possible to achieve accelerated calculation while maintaining the quality of the user experience.

[0022] Incidentally, hereinafter, for convenience, on the all-sky spherical hologram plane centered on the user's viewpoint, the displacement angle in the vertical direction is expressed as latitude and the displacement angle in the horizontal direction is expressed as longitude, similar to a globe. Hereinafter, the details of each part of the CGH generation device 10 will be described.

[0023] [User position and orientation acquisition unit 1] The user position and orientation acquisition unit 1 acquires the position and orientation of the user, and outputs three-dimensional information of the position coordinates (x, y, z) in the world coordinate system, or six-dimensional information (6DoF: Degrees of Freedom) obtained by combining the orientation (rotation angle) of the face (viewpoint) with the position coordinates, or three-dimensional information (3DoF) of only the orientation (rotation angle) of the face (viewpoint). The CGH generation device 10 can be applied to the holographic viewing application in an HMD as described in the framework in FIG. 2. The acquisition of the six-dimensional or three-dimensional information in the user position and orientation acquisition unit 1 can be realized by using the position and orientation acquisition function of sensors and the like provided in a normal HMD device.

[0024] [3D model input unit 3] The 3D model input unit 3 has a function of inputting a 3D model for generating an omnidirectional hologram to the adaptive element hologram division unit 2 and the point light source acquisition unit 4. Here, the 3D model takes the form of a collection of points and polygons used in general computer graphics. Since this embodiment assumes application to the use of viewing omnidirectional holography, hereinafter, a scene in which a 3D model is arranged so as to surround the viewpoint position of the user at the position and orientation of the user at a certain time t is assumed. Of course, a scene in which a small number of 3D models are arranged only in the front can also be applied without loss of generality.

[0025] Note that the 3D model in the 3D model input unit 3 may be prepared in advance as predetermined content in holographic viewing provided by the CGH generation device 10 as something that changes according to the time t (it operates and changes in shape or changes in surface texture, including the special case where it is stationary or the texture does not change).

[0026] [Adaptive element hologram division unit 2] In the adaptive element hologram division unit 2, similar to the existing element hologram method described in Non-Patent Document 1 and the like, the entire hologram plane (in this embodiment, the entire spherical surface at a distance from the viewpoint position to the SLM is the entire hologram plane) is divided into element holograms. These element holograms are used as units for subsequent point cloud acquisition, occlusion information acquisition, and object light wave calculation.

[0027] In the element hologram method of Non-Patent Document 1, the entire hologram plane is divided into rectangular element holograms of equal size, and in each of them, point light source acquisition and light wave propagation calculation are performed, thereby realizing natural motion parallax in a holographic reproduction image with a relatively light computational load. The smaller the size of this element hologram, the more natural motion parallax can be realized. On the other hand, due to the nature of requiring point light source acquisition by ray tracing for each element hologram, an increase in the number of element holograms leads to an increase in calculation time.

[0028] Therefore, in the adaptive element hologram division unit 2 of this embodiment, as schematically described above with reference to FIG. 4, the calculation speed is increased by adaptively controlling the size of the element hologram based on the characteristics of the user's XR experience.

[0029] Specifically, as "Embodiment 1", as shown in the description column (A) of FIG. 5, the element hologram close to the horizontal position of the user's viewpoint (close to latitude 0°, low latitude) is divided into a size that can realize sufficient motion parallax (the minimum element hologram size s min =(a,b) [degrees]. a and b respectively represent the sizes in the latitude and longitude directions), and as the latitude increases in the middle and high latitudes, as shown in the description columns (B) and (C) of FIG. 5, the size in the latitude direction is increased at a certain rate according to the displacement amount y of the latitude of the element hologram, and it is divided according to the following formula (1). a' = a + k * y …(1)

[0030] In formula (1), k>0 is a user-set parameter indicating the increment of the size in the latitude direction according to the latitude, and a' is the size in the latitude direction of the element hologram at the corresponding position output by the adaptive element hologram division unit 2.

[0031] Note that FIG. 5 is an explanatory diagram of the adaptive element hologram division unit 2 (and the point light source acquisition unit 4). As already described, the explanatory columns (A), (B), and (C) are the explanatory columns of the adaptive element hologram division unit 2. The explanatory columns (a), (b), and (c) in FIG. 5 are the explanatory columns of the point light source acquisition unit 4, which will be described later.

[0032] Equation (1) was an example of setting the size a' in the latitude direction (vertical direction). Alternatively, regarding the longitude direction (horizontal direction), as in the following equation (2), the size in the longitude direction may be corrected as follows for the displacement amount x of the longitude from the previously assumed front (longitude 0°), and the front may be divided into a fine size and the back may be made larger and larger as b' towards the back. b' = b + l * x …(2)

[0033] In equation (2), l > 0 is a user-set parameter that indicates the increment of the size in the longitude direction according to the longitude, similar to k.

[0034] Note that the previously assumed front longitude 0° and / or latitude 0° in the horizontal direction may be, for example, the line-of-sight direction acquired by the user position and attitude acquisition unit 1, or longitude 0° and / or latitude 0° may be defined in advance in the virtual space of the 3DCG content for holographic viewing as corresponding to a predetermined front and / or horizontal direction. For example, regarding the longitude of the front, the 3DCG content is a live video by singing of a CG character in a 3D live house in the virtual space. On the premise that the user is placed in the audience seat and watches the live, the stage side where the CG character exists may be defined as the corresponding longitude 0°. Also, regarding the latitude in the horizontal direction, in 3DCG content premised on viewing while lying down (or looking up) like a planetarium, instead of the horizontal direction (ground plane direction), the diagonal upward direction may be defined as latitude 0°.

[0035] Note that the division result may be obtained using both of the formulas (1) and (2) for the element hologram size s = (a', b'), or only one of them may be used. When only formula (1) is used, the division result can be obtained with the element hologram size s = (a', b), and when only formula (2) is used, the division result can be obtained with the element hologram size s = (a, b').

[0036] The adaptive element hologram division unit 2 may also use "Embodiment 2" outlined above in Fig. 4.

[0037] That is, the 3DCG model is roughly rendered on the hologram plane in advance (calculation of interference fringes is not required, only normal 3DCG rendering is performed), and the area on the celestial sphere where the 3D model is distributed is estimated in advance. Then, the element holograms in the surrounding area corresponding to important 3D model objects such as the foreground are finely divided.

[0038] Specifically, the area where the 3D model object is rendered is divided so as to be included in a set of element holograms with the size of s min The other areas are divided into coarser element hologram sizes. Also, in combination with "Embodiment 1", the areas other than the element holograms of size s min may have their sizes changed according to latitude and longitude.

[0039] Here, specific examples of important 3D model objects include, for example, the following (1) to (4). When preparing a 3D model as 3DCG content in advance in the 3D model input unit 3, the information in the following (2) to (4) can be defined as attributes or properties. The information in the following (1) can be obtained by comparing the acquisition position in the user position and orientation acquisition unit 1 with the position of the 3DCG model in the virtual space. (1) Objects close to the user (2) Moving objects, especially those moving at high speed (3) Objects with high opacity or saturation (Regarding opacity, the closer an object is to being transparent, the lower its importance.) (4) Objects with pre-registered attributes such as people and animals In addition to the above (1) to (4), content producers may also manually set the importance for each object in advance so that users can view and use the content based on the set importance.

[0040] [Point light source acquisition unit 4] The point light source acquisition unit 4 acquires a point light source for each element hologram output from the adaptive element hologram division unit 2. Specifically, ray tracing is performed at a certain ray interval from the representative point (such as the centroid) of the element hologram, and point light sources are arranged at the position coordinates determined by the intersection judgment with the 3DCG model. As described above in the section of "Prior Art", since the calculation time of the object light wave in the subsequent stage is determined by "the number of point light sources × the number of pixels on the hologram surface", it is also desirable to sparsely acquire point light sources that do not significantly contribute to the user experience quality here.

[0041] Therefore, in the point light source acquisition unit 4, as shown in the description columns (a) and (b) of FIG. 5, for the three-dimensional object OB1 of one 3D model, the central part of the 3D model has a great influence on the user experience quality, so the rays are dense, and the peripheral part has a small influence, so the rays are sparse. (Also, due to the addition of the processing of the adaptive element hologram division unit 2 on the front stage side, as shown in the description column (c) of FIG. 5, for the three-dimensional object OB2 on the higher latitude side than the three-dimensional object OB1, the rays are acquired more sparsely than the three-dimensional object OB1.)

[0042] Specifically, in the point light source acquisition unit 4, adaptive ray acquisition considering the influence on the user experience can be performed according to "Embodiment 1" shown in FIG. 6 or "Embodiment 2" shown in FIG. 7.

[0043] In "Embodiment 1", as shown in FIG. 6, the interval between the light rays emitted from each element hologram is adaptively changed. Consider a normal vector v passing through the centroid of the element hologram with respect to the tangent plane λ of the element hologram. (Here, the element hologram surface is assumed to be spherical in order to cover the entire celestial sphere. Based on the premise of this embodiment where a hologram (interference fringes) for all directions is calculated in advance before reflecting the position and orientation of the user, the hologram on the spherical surface is calculated, and efficiency can be achieved by performing processes such as approximately mapping it to a plane when displaying it on the SLM while reflecting the position and orientation of the user.) With a light ray r0 having the same direction as this v as a reference, the minimum point light source acquisition angle θ set by the user min is used to set the next light ray r1. Further, light rays are set in such a form that the light ray interval spreads outward, for example, by setting r_2 forming an angle θ min +α with respect to the light ray r1. Here, α is the increment set by the user, and in accordance with the above-mentioned principle, the angle θ n formed by the nth light ray r n-1 and the (n - 1)th light ray r n can be set, for example, as in the following formula (3). θ n = θ min +(n - 1)α …(3)

[0044] Such settings of the light rays r n (n = 0, 1, 2, …) are performed until light rays come out outside the viewing angle calculated from the pixel density of the hologram.

[0045] Alternatively, as in "Embodiment 2" of FIG. 7, for the light rays forming an angle of θ th or less with respect to the normal v (light ray r0), they uniformly form an angle θ min with the adjacent light rays, and in other cases, the light rays may be set to form a larger θ max . In FIG. 7, the light rays r1, r2, etc. are set to be within the threshold θ th and form an angle θ min with the adjacent light rays, and the light rays r 10 , r 11 etc. are set to be outside the threshold θ th range and form an angle θ maxIt is set to form. The points where these set light ray groups first intersect the 3D model are obtained as point light sources.

[0046] Here, in FIGS. 6 and 7, the light rays are set in a form with a one-dimensional spread, but it should be noted that in reality, the light rays have a two-dimensional spread along a plane parallel to the tangent plane λ. (That is, in FIGS. 6 and 7, each light ray r n From the perspective of the visibility on the drawing plane of the angle formed by, each light ray r n is drawn only when it lies on one plane, but in reality, each light ray r n (n ≧ 1) can be set to be distributed so as to lie on the side surface of a cone and spread radially from the normal light ray r0. In this way, when setting each light ray r n in three-dimensional space, the angle can also be set according to the same principle as in Equation (3) and the like.)

[0047] Note that since it is guaranteed that the point light sources obtained by the point light source acquisition unit 4 can be observed from the representative points of each element hologram, it can be regarded as retaining the occlusion information of the 3D model in the dark. Also, θ min may be set to increase (become light rays with sparse intervals) according to the latitude of the element hologram based on the same concept as in Equation (1) (in the description column (c) of FIG. 5).

[0048] Note that in the point light source acquisition unit 4, instead of adaptively acquiring point light sources as in the methods of FIGS. 6 and 7 above, a point group may be acquired using light rays emitted at equal intervals from the center of the element hologram in the same manner as in the prior art. (In this case, by the processing of the adaptive hologram division unit 2 in the CGH generation device 10, the effect of ensuring the user experience quality and suppressing the calculation amount can be obtained.)

[0049] Similarly, in the adaptive hologram division unit 2, the element holograms may be set by uniformly dividing the entire sphere without performing adaptive division considering the position and orientation in the entire sphere, and in the point light source acquisition unit 4, the point light sources may be adaptively acquired by the methods of FIGS. 6 and 7 above.

[0050] In both the case where the point light source acquisition unit 4 adaptively acquires a point light source instead of uniformly and the case where the adaptive hologram division unit 2 performs element hologram division by distinguishing between large and small sizes, in the CGH generation apparatus 10 of the present embodiment, by the processing of the adaptive hologram division unit 2 and / or the point light source acquisition unit 4, the density of the point light source corresponding to the region where it is assumed in advance by prior knowledge that the degree of user fixation is high in the 3D CG content of the entire sphere is increased, and the density of the point light source corresponding to the region where it is assumed that the degree is low is decreased, thereby obtaining the effect of ensuring the user experience quality and suppressing the calculation amount.

[0051] That is, in Embodiment 1 of the adaptive hologram division unit 2, it is assumed that the degree of fixation is predetermined according to the position of the entire sphere, and for the region with a high degree of fixation, by setting the size of the element hologram surface to be smaller, the density of the corresponding point light source can be made higher.

[0052] FIG. 8 is a diagram schematically showing the state of change in the density of the point light source by changing and setting the size of the element hologram surface in the adaptive hologram division unit 2. In a situation where there is a partial region HLP of the element hologram surface distribution and an object OB3 to be drawn in the same positional relationship, the state of the point light source set in the case of performing sparse element hologram division (when setting a large size for the element hologram) C1 and the case of performing dense element hologram division (when setting a small size for the element hologram) C2 are schematically compared.

[0053] That is, when the size of the element hologram is set large, in case C1, the partial region HLP is divided into two large element holograms hA and hB (representative points A and B), and when the size is set small, in case C2, the same partial region HLP is divided into four small element holograms ha, hb, hc, and hd (representative points a, b, c, and d). For example, as can be seen from the state of the point light source (and the corresponding light ray) that becomes visible from the representative point A in case C1, when the point light source is uniformly set by the point light source acquisition unit 4, generally, regardless of the size of the element hologram, at a position close to the representative point (such as the front surface of the object OB3 near the representative point A, where the longitude / latitude is close), the point light source is densely acquired, and at a position far from the representative point, the point light source tends to be sparsely acquired.

[0054] Under the above tendency, when the size of the element hologram is made smaller, as can be seen by comparing cases C1 and C2, the range hA covered by the representative point A is ultimately densely covered by the ranges ha and hb by the two representative points a and b. As a result, in case C2 where the size of the element hologram is small, high-quality holography can be reproduced even when the viewpoint to the object OB3 changes within the range hA. (However, here, for example, when the viewpoint corresponds to the representative point b, only the corresponding ◆ (black-filled diamond) point light source is used, and it should be noted that the point light sources of other representative points a, c, d, etc. indicated by ● (black circle), ◇ (white-filled diamond), 〇 (white circle), etc. in the vicinity are not used simultaneously for the representative point b.)

[0055] Also, similarly to Embodiment 2 of the adaptive hologram division unit 2, regarding the same elemental hologram plane, for the point light sources of the observable object light, for the point light sources corresponding to the region that corresponds to the front side of the object and is determined to have a high degree of attentiveness, the density can be increased more. In this way, in the adaptive hologram division unit 2, differences are provided in the coarseness and fineness of the corresponding point light sources according to the degree of attentiveness between different elemental holograms according to Embodiments 1 and 2. Similarly, when the point light source acquisition unit 4 adaptively acquires point light sources according to Embodiments 1 and 2, within the same elemental hologram, differences are provided in the coarseness and fineness of the corresponding point light sources according to the degree of attentiveness (so as to be more prominent compared to the coarseness and fineness when setting light rays at uniform angular intervals).

[0056] Incidentally, as a suitable example of using the CGH generation device 10 of the present embodiment, the case of handling 3DCG content of the entire celestial sphere with a wide drawing range has been described. However, even if it is not in the form of the entire celestial sphere, for other 3DCG content with a wide drawing range, similarly, the effect of ensuring the user experience quality and suppressing the calculation amount can be obtained by the method of the CGH generation device 10 of the present embodiment.

[0057] [Optical wave propagation calculation unit 5] In the optical wave propagation calculation unit 5, based on the point light source method which is a conventional method, the propagation calculation of the object light wave from each point light source to the corresponding elemental hologram plane is performed. This calculation is represented by the following formulas (4) and (5).

[0058] [Equation]

[0059] Here, (x, y) indicates the pixel position on the hologram plane where the light wave propagates, and s i (x, y) is the light wave distribution on the hologram plane propagated from each point light source c i to. A i is the luminance of c i , and r i is the distance from the point light source c irepresents the distance to the pixel (x, y) on the hologram plane. Also, k represents the wave number calculated from the wavelength of light.

[0060] [Interference calculation unit 6] The interference calculation unit 6 performs interference calculation by inserting a reference light wave as a simulation on a computer with respect to the object light wave u(x, y) on the hologram plane output by the light wave propagation calculation unit 5. As the reference light in this embodiment, a converging spherical reference light wave that converges at the position of the focal length f of the lens of the reproduction optical system disclosed in Non-Patent Document 1 can be used. The complex amplitude distribution R(x, y) of the light wave when this converging spherical reference light wave is propagated on the hologram plane is represented by the following equation (6).

[0061] [Number]

[0062] Here, R o is the intensity of the reference light, and r represents the distance from the position of the reference light to the position (x, y) on the hologram plane. Note that the position of the reference light may be arranged at the focal position (0, 0, f) of the lens constituting the lens magnification optical system in this embodiment as well as in Non-Patent Document 1. Here, f is the focal length of the lens. Note that the reference light of the present invention is not limited to Equation (6), and may be a simple spherical wave reference light such as Equation (7) or a parallel light reference light such as Equation (8). However, φ in Equation (8) is the incident angle of the reference light to the hologram plane.

[0063] [Number]

[0064] Finally, the equation showing the interference between this reference light wave and the object light wave is the following equation (9), where I(x, y) is the luminance distribution of the CGH. In the interference calculation unit 6 of this embodiment, this I(x, y) is normalized to the range of 0 - 255 and output as an image to the SLM mounted on the HMD.

[0065]

Number

[0066] As described above, according to the CGH generation device 10 of the present embodiment, in the all-spherical holographic reproduction applicable to holographic viewing and the like with an HMD, it becomes possible to calculate the light wave propagation at high speed without degrading the quality of the reproduced image as compared with the conventional light wave propagation calculation. Hereinafter, various supplementary examples, additional examples, alternative examples, etc. will be described.

[0067] (1) According to the embodiment of the present invention, as an application example, by holographic reproduction of a three-dimensional model generated in real time, it becomes possible to view a target existing in a remote location (for example, a player during a sports game) as a free-viewpoint video with a sense of presence. As a result, it is not always necessary to actually move to a remote location to view content such as a sports game, or to use it as a display interface for remote communication to give advice on a target in a remote location (for example, advice on sports improvement). Since it is possible to suppress carbon dioxide emissions by saving the energy resources required for user movement, it becomes possible to contribute to Goal 13 of the Sustainable Development Goals (SDGs) led by the United Nations, "Take urgent action to combat climate change and its impacts."

[0068] (2) FIG. 9 is a diagram showing an example of the hardware configuration in a general computer device 70. The CGH generation device 10 can be realized as one or more computer devices 70 having such a configuration. When realizing the computer synthesized hologram generation device 10 with two or more computer devices 70, information necessary for processing may be transmitted and received via a network. The computer device 70 includes a CPU (Central Processing Unit) 71 that executes a predetermined instruction, a GPU (Graphics Processing Unit) 72 as a dedicated processor that executes part or all of the execution instructions of the CPU 71 instead of or in cooperation with the CPU 71, a RAM 73 as a main storage device that provides a work area for the CPU 71 (and the GPU 72), a ROM 74 as an auxiliary storage device, a communication interface 75, a display 76, a mouse, a keyboard, an input interface 77 that receives user input by a touch panel or the like, a camera 78, and a bus BS for exchanging data among these components.

[0069] Each functional unit of the CGH generation device 10 can be realized by the CPU 71 and / or the GPU 72 that reads and executes a predetermined program corresponding to the function of each unit from the ROM 74. Note that both the CPU 71 and the GPU 72 are a kind of arithmetic unit (processor). Here, when display-related processing is performed, the display 76 further operates in conjunction, and when communication-related processing regarding data transmission and reception is performed, the communication interface 75 further operates in conjunction. The display 76 is configured as an SLM (Spatial Light Modulator) to realize the 3D content display calculated by the interference calculation unit 6. In addition, since there is also an additional one configured as a normal liquid crystal display or the like, the intermediate result of the processing in the CGH generation device 10 may be displayed via the liquid crystal display or the like.

Explanation of Reference Numerals

[0070] 10…CGH generation device (computer synthesized hologram generation device), 1…user position and orientation acquisition unit, 2…adaptive element hologram division unit, 3…3D model input unit, 4…point light source acquisition unit, 5…light wave propagation calculation unit, 6…interference calculation unit

Claims

1. In a computer-generated hologram generation device that generates a computer-generated hologram by performing interference calculation between object light from a point light source on the surface of an object and reference light on a hologram plane, the interference calculation is performed by allocating denser point light sources to a region determined to have a greater degree of user fixation, the hologram plane corresponds to the entire sphere, and after dividing the hologram plane into elemental hologram planes, the interference calculation is performed for the point light sources of the three-dimensional model object observable for each elemental hologram plane, the point light sources to be the subject of the interference calculation are set by examining the observable point light sources for light rays whose angles are gradually changed from the front direction of the object on the elemental hologram plane, in the same elemental hologram plane, assuming that the front direction has the highest degree of fixation, the change width of the gradually changing angle is set to be smaller for light rays closer to the front direction, so that the density of the corresponding observable point light sources becomes denser as it gets closer to the front direction. A computer-generated hologram generation device characterized by this.

2. A program characterized by causing a computer to function as the computer-generated hologram generation device according to Claim 1.

Images