Information processing device, information processing method, program, and hologram display system

Abstract

The present invention improves user experience. This information processing device comprises: a grouping unit (21) which groups, into one or more groups, a plurality of pixels that compose one or more objects included in one piece of image data; a phase adjustment unit (22) which allocates phase patterns, which have different phase differences, to the plurality of pixels for each of the one or more groups; and a calculation unit (30) which generates hologram data from the image data to which the phase patterns are imparted.

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device, an information processing method, a program, and a hologram display system. [Background technology]

[0002] Computer-generated holograms (CGHs) are a technique for generating holograms solely through computations. Such techniques are necessary because it is often difficult to directly store a hologram of a scene.

[0003] Conventionally, a random phase is applied to the input pixel or voxel prior to the wave propagation process, which spreads the information carried by the pixel or voxel over a larger area on the hologram. Spreading the information over a larger area on the hologram plane results in a reconstructed image with a shallower depth of field, making it more robust to artifacts and dust on the optical system's lenses.

[0004] However, adding a random phase to the input image induces another type of noise in the replay field, called speckle noise, which appears approximately randomly in the replay field. Most techniques for reducing speckle noise are based on time-consuming iterative algorithms that have a real-time disadvantage.

[0005] CGH calculation algorithms that do not use random phases have also been developed, and the reconstructed images using these methods can achieve very high image quality without speckle noise. Without random phases, the holograms containing information from sampled points in space are concentrated in a small area. This results in a narrow beam of light in the reconstructed image, which in turn results in a large depth of field in the reconstructed image. Therefore, conventional techniques have been developed to mitigate these two phenomena. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Application Laid-Open No. 2005-195803 Summary of the Invention [Problem to be solved by the invention]

[0007] However, there is always a trade-off between the two phenomena mentioned above, and therefore, conventional technologies cannot simultaneously alleviate the two issues of poor image resolution and deep depth of field for the same image, making it difficult to improve the user experience.

[0008] Therefore, the present disclosure proposes an information processing device, an information processing method, and a program that enable an improvement in user experience. [Means for solving the problem]

[0009] In order to solve the above problem, an information processing device according to an embodiment of the present disclosure is configured to: The first group requires high resolution, and the second group requires shallow depth of field. a grouping unit for grouping the plurality of pixels into a first phase pattern is assigned to the first group, and a second phase pattern having a phase difference larger than that of the first phase pattern is assigned to the second group; a phase adjustment unit that assigns The first phase pattern and the second phase pattern are assigned and a calculation unit that generates hologram data from the image data. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a schematic diagram for explaining an overview of a holographic display system according to a first embodiment. [Figure 2] 1 is a block diagram showing a schematic configuration example of an information processing device according to a first embodiment. [Figure 3] 5 is a flowchart showing an example of the operation of the information processing device according to the first embodiment. [Figure 4]FIG. 2 is a diagram showing an example of an object image constituting input image data according to the first embodiment. [Figure 5] FIG. 2 is a diagram showing an example of a depth image constituting input image data according to the first embodiment. [Figure 6] FIG. 2 is a diagram showing an example of a label image constituting input image data according to the first embodiment. [Figure 7] FIG. 3 is a diagram illustrating an example of a random phase patch according to the first embodiment. [Figure 8] FIG. 2 is a diagram showing an example of a random phase patch whose frequency has been reduced according to the first embodiment. [Figure 9] FIG. 2 is a diagram showing the positional relationship between a reproduced hologram, a hologram display system, and a viewer (user) in the first embodiment. [Figure 10] FIG. 2 is a diagram showing an example of a hologram displayed to a user according to the first embodiment (when the user is looking at a baby in the foreground). [Figure 11] FIG. 2 is a diagram showing an example of a hologram displayed to a user according to the first embodiment (when looking at the boy in the back). [Figure 12] FIG. 10 is a block diagram showing a schematic configuration example of an information processing device according to a second embodiment. [Figure 13] FIG. 10 is a block diagram showing a schematic configuration example of an information processing device according to a third embodiment. [Figure 14] 10A and 10B are diagrams for explaining the direction of a user's line of sight detected by an eye-gaze tracking unit according to the third embodiment. [Figure 15] 15 is a diagram for explaining an example of grouping of objects based on the line of sight shown in FIG. 14. FIG. [Figure 16] FIG. 10 is a diagram showing an example of a hologram displayed to a user according to the third embodiment (when the user is looking at a baby in the foreground). [Figure 17] FIG. 10 is a diagram showing an example of a hologram displayed to a user according to the third embodiment (when looking at the boy in the back). [Figure 18] FIG. 10 is a block diagram showing a schematic configuration example of an information processing device according to a fourth embodiment. [Figure 19] FIG. 10 is a diagram showing an example of a surrounding image acquired by a camera according to a fourth embodiment. [Figure 20] FIG. 10 is a diagram showing an example of a characteristic map generated by a characteristic detection unit according to the fourth embodiment based on a surrounding image. [Figure 21] FIG. 13 is a diagram illustrating an example of grouping of objects according to the fourth embodiment. [Figure 22] FIG. 10 is a diagram showing an example of a hologram displayed to a user according to the fourth embodiment (when the user is looking at a baby in the foreground). [Figure 23] FIG. 13 is a diagram showing an example of a hologram displayed to a user according to the fourth embodiment (when looking at the boy in the back). [Figure 24] 13A and 13B are diagrams for explaining the direction of a user's line of sight detected by an eye-tracking unit according to the fifth embodiment. [Figure 25] FIG. 25 is a diagram for explaining an example of grouping of objects based on the line of sight shown in FIG. 24. [Figure 26] FIG. 13 is a diagram showing an example of a hologram displayed to a user according to the fifth embodiment (when the user is looking at a baby in the foreground). [Figure 27] FIG. 13 is a diagram showing an example of a hologram displayed to a user according to the fifth embodiment (when looking at the boy in the back). [Figure 28] FIG. 1 is a block diagram illustrating an example of a hardware configuration of an information processing apparatus according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In the following embodiments, the same components are designated by the same reference numerals, and redundant description will be omitted.

[0012] The present disclosure will be described in the following order. 1. First embodiment 1.1 Overview 1.2 Example of a general configuration of an information processing device 1.3 Example of operation of information processing device 1.4 Actions and Effects 2. Second embodiment 3. Third embodiment 4. Fourth Embodiment 5. Fifth Embodiment 6. Hardware configuration

[0013] 1. First embodiment An information processing device, an information processing method, and a program according to a first embodiment will be described in detail below with reference to the drawings.

[0014] 1.1 Overview First, an overview of this embodiment will be described. Fig. 1 is a schematic diagram for explaining the overview of a holographic display system according to this embodiment. In Fig. 1 and the following figures, (A) to (F) indicate respective schematic positions.

[0015] As shown in FIG. 1, this embodiment illustrates a hologram display system including a light source 101, an expanding optical system composed of a plurality of lenses 102 and 103, a beam splitter 104, and a spatial light modulator 105.

[0016] 1, laser light L1 from a light source 101 is converted into coherent light L2, the beam diameter of which is expanded, by an expanding optical system made up of multiple lenses 102 and 103. The coherent light L2 passes through a beam splitter 104 and enters a spatial light modulator 105.

[0017] In this embodiment, a reflective spatial light modulator is exemplified as the spatial light modulator 105. The coherent light L2 is modulated by the spatial light modulator 105 so as to form a hologram at a point in a predetermined region in space.

[0018] Beam splitter 104 projects hologram 106 reproduced by spatial light modulator 105, for example, at position E in space so that it can be observed by user 107. User 107, who is at position F, can view hologram 106 superimposed on real space visible through beam splitter 104 by observing the direction of position E.

[0019] In this way, the hologram display system according to this embodiment can provide user experiences such as making virtual objects appear in real space, using special effects to present objects in real space, and presenting specific information to the user.

[0020] In the above configuration, the display device 50 may be, for example, an optical see-through head-mounted display (hereinafter referred to as an AR HMD). The pre-processing unit 20, the CGH calculation unit 30, and the post-processing unit 40 may constitute an information processing device in a hologram display system. A part or all of this information processing device may be disposed within the display device 50, i.e., within the AR HMD, or may be disposed in a server (including a cloud server) connected to the display device 50 via a predetermined network (e.g., a local area network (LAN), the Internet, a mobile communication network including LTE (Long Term Evolution), WiFi (Wireless Fidelity), 4G, 5G, etc.).

[0021] 1.2 Example of a general configuration of an information processing device Next, a schematic configuration example of an information processing device provided in the hologram display system according to this embodiment will be described in detail with reference to the drawings. Fig. 2 is a block diagram showing a schematic configuration example of the information processing device according to this embodiment. As shown in Fig. 2, the information processing device 1 includes a pre-processing unit 20, a CGH calculation unit 30, and a post-processing unit 40.

[0022] (Pre-processing unit 20) The preprocessing unit 20 includes a grouping unit 21 and a phase adjustment unit 22, and performs preprocessing (described later) on input image data (input image data 10 (described later)).

[0023] (CGH calculation department 30) The CGH calculation unit 30 generates hologram data to be input onto the SLM plane by calculation from the input image data that has been preprocessed by the preprocessing unit 20.

[0024] (Post-processing unit 40) The post-processing unit 40 converts the hologram data generated by the CGH calculation unit 30 into a hologram signal that can be displayed on the display device 50 .

[0025] (Display device 50) The display device 50 outputs the hologram signal converted by the post-processing unit 40, thereby displaying a hologram of the object reproduced from the input image data to the user in a three-dimensional manner.

[0026] 1.3 Example of operation of information processing device Next, an example of the operation of the information processing device 1 according to this embodiment will be described in detail with reference to the drawings. Fig. 3 is a flowchart showing an example of the operation of the information processing device according to this embodiment.

[0027] As shown in FIG. 3, in this operation, in step S101, first, the information processing device 1 inputs information (hereinafter referred to as input image data) relating to an image to be displayed as a hologram. In this embodiment, the input image data 10 may be, for example, 2D image data, 2.5D image data, 3D image data, etc. Note that the 2.5D image data may be, for example, image data composed of color information of the three RGB primary colors and depth information (also referred to as distance information) for each pixel or voxel (hereinafter simply referred to as pixel). Also, the 3D image data may be, for example, image data composed of color information of the three RGB primary colors and 3D information.

[0028] In the following description, an example will be given in which 2.5-dimensional image data is input as input image data 10. Figures 4 and 5 are examples of input image data according to this embodiment, with Figure 4 showing image data (referred to as an object image) consisting of RGB color information for each pixel out of the input image data 10, and Figure 5 showing image data (hereinafter referred to as a depth image) consisting of depth information for each pixel.

[0029] The object image G40 shown in Fig. 4 includes a boy object C41 located in the upper left, a baby object C42 located in the lower right, and a clock object C43 located in the lower left. In the following description, when distinguishing between objects displayed in relation to real-space objects (in this example, objects C41 and C42) and objects displayed on a virtually set screen (in this example, object C43), objects displayed in relation to real-space objects will be referred to as virtual objects (including effects for presentation, etc.; hereinafter, referred to as AR objects), and objects displayed on a virtually set screen on a predetermined plane in real space will be referred to as OSD objects. Note that in Fig. 4, the white background region R41 indicates a transparent point with no color information, i.e., an area where no object exists.

[0030] 5 includes an area C51 corresponding to a boy AR object C41 located in the upper left, an area C52 corresponding to a baby AR object C42 located in the lower right, and an area D23 corresponding to a clock OSD object C43 located in the lower left. In Fig. 5, the blacked-out area R51 indicates an area to which no depth information is assigned or to which the furthest depth information is assigned.

[0031] In a depth image, the depth information assigned to each point represented as a pixel may be depth information corresponding to the actual distance from the real-space position of the user 107 to the real-space position to which each pixel corresponds.

[0032] In this example, depth information corresponding to a distance of 500 mm from the user 107 is assigned to an area C53 corresponding to a clock OSD object C43 located at the bottom left. Depth information corresponding to a distance of 1000 mm from the user 107 is assigned to an area C52 corresponding to a baby AR object C42 located at the bottom right, and depth information corresponding to a distance of 2000 mm from the user 107 is assigned to an area C51 corresponding to a boy AR object C41 located at the top left.

[0033] The input image data 10 may include map information regarding the type of image to be reproduced as a hologram.

[0034] Furthermore, in this embodiment, the input image data 10 may include label information related to objects included in the object image. In the example shown in Fig. 4 and Fig. 5, label information may be assigned to regions C61 to C63 corresponding to a boy AR object C41, a baby AR object C42, and a clock OSD object C43, respectively, as shown in Fig. 6. Hereinafter, an image onto which the regions C61 to C63 to which label information is assigned are mapped will be referred to as a labeled image G60. Note that in Fig. 6, a white region R61 indicates a region to which no label information is assigned.

[0035] 6, area C61 is an area to which label information indicating a boy AR object C41 is assigned, area C62 is information to which label information indicating a baby AR object C42 is assigned, and area C63 is information to which label information indicating a clock OSD object C43 is assigned. Note that in FIG. 6, areas C61 and C62 corresponding to objects C41 and C42 to be superimposed on objects in real space are indicated by diagonal hatching, and area C63 corresponding to on-screen object C43 is indicated by black.

[0036] In step S102 of FIG. 3, points (corresponding to pixels, hereinafter referred to as pixels) constituting the input image data 10 input in step S101 are grouped into one or more groups. The algorithm for grouping the pixels may be changed in various ways depending on the actual use case. In this embodiment, since it is difficult to achieve high resolution and shallow depth of field simultaneously, pixels requiring high resolution and pixels requiring shallow depth of field are grouped into different groups and treated differently.

[0037] For example, it is considered that a user would want an on-screen object such as the OSD object C43, which is a clock, to be clearly visible regardless of the distance from which the viewpoint is adjusted, and therefore it is desirable for the on-screen object to have high resolution and a large depth of field.

[0038] On the other hand, it is desirable that AR objects such as AR objects C41 and C42 be displayed at positions close to their corresponding real-space objects, and for this purpose, they need to have a depth of field similar to that of the real-space objects. In other words, a shallow depth of field is important for AR objects.

[0039] Thus, high resolution and a deep depth of field are required for OSD objects, while shallow depth of field is required for AR objects even at the expense of resolution. Therefore, in this embodiment, each pixel of the input image data 10 is grouped into one or more groups based on label information. For example, the input image data 10 is classified into a group of OSD objects (hereinafter referred to as an OSD object group) and a group of AR objects (hereinafter referred to as an AR object group). Note that transparent areas where no objects exist do not need to be grouped, and therefore can be omitted.

[0040] 3, the phase adjustment unit 22 of the pre-processing unit 20 assigns phase values ​​to all pixels of the input image data 10. At this time, different characteristic phase values ​​are assigned to pixels belonging to an AR object group and pixels belonging to an OSD object group so that the reproduced pixels have different characteristics. Note that no phase value may be assigned to pixels that do not belong to either an AR object group or an OSD object group.

[0041] As a method for assigning a phase value to each pixel, for example, a recursive random phase (RRP) method can be used. In the RRP method, first, a patch of a random phase pattern (hereinafter referred to as a random phase patch) to be assigned to an area called a cell of a predetermined size (for example, m pixels x n pixels) is generated. The random phase pattern may be, for example, a pattern in which the difference between the phase values ​​assigned to adjacent pixels is a random value that is not a fixed value.

[0042] In this example, random phase patches in which the maximum phase difference within the patch is π are assigned to multiple pixels included in one cell. Such allocation of random phase patches on a cell-by-cell basis is performed for the entire range of the input image data 10. FIG. 7 is a diagram showing an example of a random phase patch assigned to one cell when the cell size is 1 pixel x 32 pixels. In FIG. 7, the horizontal axis indicates the number (sample number) of pixels arranged horizontally in the input image data, and the vertical axis indicates the phase value assigned to each pixel.

[0043] 7 is repeated for a horizontal row of pixels in the input image data 10. This repetition is applied to all rows of the input image data 10.

[0044] By using the square root of the color intensity of each pixel in the object image G40 shown in Figure 4, the input image data 10 is converted from a real-valued image to a complex-valued image having an amplitude that conforms to the newly added phase information.

[0045] The random phase pattern can scatter light so as to widen the area that the light forms on the SLM plane after wave propagation: high-frequency phase information in the input image space scatters light so as to widen the area of ​​light formed on the SLM plane, and low-frequency phase information scatters light so as to narrow the area of ​​light formed on the SLM plane.

[0046] High frequency phase contributes to a shallower depth of field by scattering the light pattern on the SLM plane more, so high frequency phase patches are more suitable for AR object groups.

[0047] On the other hand, low-frequency phase contributes to a larger depth of field by shrinking the light pattern on the SLM plane, so low-frequency phase patches are more suitable for OSD object groups.

[0048] As can be seen from the random phase patch shown in Fig. 7, there are various methods for reducing or increasing the frequency of the phase component. Examples of methods for reducing the frequency of the phase component include a method of multiplying the phase component by a coefficient (hereinafter referred to as a phase coefficient) smaller than 1.0 that reduces the amplitude of the random phase patch, and a method of filtering the phase component with a low-pass filter. In this embodiment, as shown in Fig. 8, a case is exemplified in which a random phase patch having a lower frequency than the random phase patch shown in Fig. 7 is newly generated by multiplying the random phase patch by a phase coefficient of 0.25.

[0049] By assigning a phase value to each pixel belonging to the OSD object group using a low frequency random phase patch as illustrated in FIG. 8, the OSD object C43 can be reproduced with greater clarity and a greater depth of field.

[0050] Similarly, by assigning a phase value to each pixel belonging to the AR object group using a high-frequency random phase patch as illustrated in Figure 7, AR objects C41 and C42 can be reproduced with a shallower depth of field and lower resolution.

[0051] In step S104 of Figure 3, the complex field generated by the phase adjustment unit 22 of the pre-processing unit 20 using the above-mentioned rules is propagated to the SLM plane based on the depth information of the regions C51 to C34 corresponding to each object described using Figure 5.

[0052] 9 is a diagram showing the positional relationship between the reproduced holograms, the hologram display system, and the viewer (user) in this embodiment. In Fig. 9, of the three reproduced holograms 106a to 106c, hologram 106a located at position E1 corresponds to the OSD object C43 of a clock, hologram 106b located at position E2 corresponds to the AR object C42 of a baby, and hologram 106c located at position E3 corresponds to the AR object C41 of a boy.

[0053] Each point appearing as a pixel on the SLM plane is found by the CGH calculation unit 30 using a wave propagation equation. For example, the Fresnel diffraction equation shown in the following equation (1) can be used as this wave propagation equation. In equation (1), z represents the distance from each of the reconstructed image (hologram) planes to the spatial light modulator 105.

number

[0054] For example, in Figure 9, if the distance from spatial light modulator 105 located at position A to hologram 106c located at position E3 is 1000 mm, the value of distance z for the image to display the hologram to propagate from position E3 to position A is 1000 mm.

[0055] The wave propagation function for generating a hologram on the SLM plane is not limited to the wave propagation formula shown in equation (1). For example, it is also possible to use the Rayleigh-Sommerfeld diffraction formula or the Fraunhofer diffraction formula, which are based on other hypotheses.

[0056] Pixels at different depths propagate to the SLM plane separately and are accumulated into the same field on the SLM plane. The simplest way to combine fields at different distances into one is to use the field merging method. However, more sophisticated methods for merging fields at different depths can also be applied to this embodiment based on actual use cases.

[0057] 3, the integrated complex field generated in step S104 is subjected to the post-processing described above. Note that the post-processing varies depending on the type of spatial light modulator 105. In this embodiment, which uses a reflective spatial light modulator 105, a process for directly displaying the complex field can be used.

[0058] In a complex-valued SLM, the amplitude component is quantized in the range of 0 to 255, and the phase component in the range of 0 to 2π is quantized to a discrete value in 255 steps. In the post-processing according to this embodiment, the complex field is mapped to a signal that can be displayed by the SLM and then quantized.

[0059] The amplitude information can be mapped using the following equation (2): According to equation (2), the field U generated by the CGH calculation unit 30 is converted into a signal that can be displayed by the SLM.

number

[0060] 3, the signal that has passed through the spatial light modulator 105 is displayed. In this example, as illustrated in FIG. 9, a clock OSD object C43 is displayed as a hologram 106a at a position E1 that is 500 mm away from the user 107 at a position F, a baby AR object C42 is displayed as a hologram 106b at a position E2 that is 1000 mm away from the user 107, and a boy AR object C41 is displayed as a hologram 106c at a position E3 that is 2000 mm away from the user 107.

[0061] 10 and 11 are diagrams showing examples of scenes that the user can see depending on where the user is looking, i.e., where the user's focus position is, Fig. 10 is a diagram showing a scene that the user can see when the user is looking at a baby in the foreground, and Fig. 11 is a diagram showing a scene that the user can see when the user is looking at a boy in the background. Note that in Fig. 10 and Fig. 11, for areas R101 and R111 that correspond to the background other than objects C41 to C43 illustrated in Fig. 4, it is assumed that the user is looking directly at the real space through the optical see-through AR HMD.

[0062] As described above, in this embodiment, a shallow depth of field is set by assigning a high-frequency phase value to the AR objects C41 and C42. Therefore, as in the image G100 illustrated in Fig. 10, when the user 107 is looking at the baby in the foreground, the baby's AR object C102 is displayed clearly, while the boy's AR object C101 located in the background is displayed blurred. In contrast, as in the image G110 illustrated in Fig. 11, when the user 107 is looking at the boy in the background, the boy's AR object C111 is displayed clearly, while the baby's AR object C112 located in the foreground is displayed blurred.

[0063] As described above, according to this embodiment, the user 107 can view the AR object with the same visual effect (for example, distance perception) as an object existing in real space. This makes it possible to enhance the reality of the AR object, thereby improving the user experience.

[0064] On the other hand, in this embodiment, a low-frequency phase value is assigned to the OSD object C43, thereby setting a deep depth of field. Therefore, as illustrated in Figures 10 and 11, the clock OSD objects C103 and C113 are clearly displayed to the user 107 no matter where the user 107 is looking.

[0065] 1.4 Actions and Effects As described above, according to this embodiment, it is possible to set appropriate parameters for different points on the same image, thereby improving the user experience.

[0066] Specifically, since it is possible to freely set the resolution and depth of field for each object included in the same image, it is possible to simultaneously present high-resolution objects and objects with shallow depth of field to the user. This allows for the reproduction of natural depth blur for objects with shallow depth of field, and for high-resolution objects to be reproduced clearly and easily visible, thereby improving the user experience.

[0067] It should be noted that the above-described embodiment is a flexible framework, and it goes without saying that an AR developer can freely adjust the resolution and depth of field of an object to suit the use case of the system they are developing. Furthermore, in the above-described embodiment, an example has been given in which the depth of field for each object is adjusted by assigning phase patches with different maximum phase differences to each group into which pixels are allocated, but this is not limited to this. For example, various modifications are possible, such as assigning phase patches with phase patterns with different phase differences to each group, or assigning phase patches with different phase patterns.

[0068] 2. Second embodiment Next, an information processing device, an information processing method, and a program according to a second embodiment will be described in detail with reference to the drawings. In the following description, the same components as those in the above-described embodiment will be denoted by the same reference numerals, and redundant description thereof will be omitted.

[0069] Fig. 12 is a block diagram showing a schematic configuration example of an information processing device according to this embodiment. As shown in Fig. 12, the information processing device 2 according to this embodiment has a configuration similar to that of the information processing device 1 described in the first embodiment with reference to Fig. 2, except that the preprocessing unit 20 further includes an image analysis unit 23.

[0070] In this embodiment, unlike the first embodiment, the input image data 10 does not include a label image (see FIG. 6). Instead, in this embodiment, the preprocessing unit 20 includes an image analysis unit .

[0071] The image analysis unit 23 analyzes the input image data 10 by machine learning using a neural network such as a convolutional neural network (CNN). For example, the image analysis unit 23 classifies each pixel of the input image data 10 into a plurality of classes by classification, and labels the pixels classified into each class. The labels assigned to the pixels in this way can be used in place of the label information in the first embodiment.

[0072] As a machine learning technique using a neural network such as a convolutional neural network (CNN), for example, PSPNet (Pyramid Scene Parsing Network) can be used. PSPNet can classify each pixel of the input image data 10 into multiple classes using the method described in the first embodiment with reference to FIG.

[0073] The other configurations, operations, and effects may be the same as those of the above-described embodiment, and therefore detailed description thereof will be omitted here.

[0074] 3. Third embodiment Next, an information processing device, an information processing method, and a program according to a third embodiment will be described in detail with reference to the drawings. In the following description, the same components as those in the above-described embodiment will be denoted by the same reference numerals, and redundant description thereof will be omitted.

[0075] Fig. 13 is a block diagram showing a schematic configuration example of an information processing device according to this embodiment. As shown in Fig. 13, the information processing device 3 according to this embodiment has the same configuration as the information processing device 1 described with reference to Fig. 2 in the first embodiment, except that it further includes a camera (imaging unit) 60 and the preprocessing unit 20 further includes an eye tracking unit 24. Note that in this embodiment, as in the second embodiment, the input image data 10 does not include a label image (see Fig. 6).

[0076] The camera 60 captures, for example, a user 107 who is a viewer. Image data of the captured user 107 (hereinafter referred to as a user image) is input to the gaze tracking unit 24 of the pre-processing unit 20. In response to this, the gaze tracking unit 24 analyzes the input user image to detect the user's gaze direction.

[0077] In this embodiment, the grouping unit 21 groups each pixel of the input image data 10 based on the gaze information input from the gaze tracking unit 24. For example, the grouping unit 21 according to this embodiment groups each pixel of the input image data 10 into two groups based on whether the pixel corresponds to an object currently being viewed by the user 107. One group is a group of objects currently being viewed by the user 107 (hereinafter referred to as the focus group), and the other group is a group of objects not included in the objects currently being viewed by the user 107 (hereinafter referred to as the out-of-focus group).

[0078] FIG. 14 is a diagram showing the direction in which the user is looking in the input image data, and FIG. 15 is a diagram showing an example of an image (hereinafter referred to as a grouped image) corresponding to the label image illustrated in FIG. 6, and is a diagram for explaining an example of grouping of objects based on the gaze direction shown in FIG. 14.

[0079] As shown in Figure 14, when the user 107's line of sight V141 is directed toward the baby AR object C142, as shown in Figure 15, pixels belonging to area C152 corresponding to AR object C142 are grouped into a focus group, and AR objects other than AR object C142, in this example, pixels belonging to area C151 corresponding to AR object C141, and pixels belonging to area C153 corresponding to OSD object C143, are grouped into an out-of-focus group.

[0080] In this way, when the regions corresponding to each object are grouped based on the gaze direction of the user 107, phase values ​​are assigned to the pixels belonging to each group. In this example, since the user 107 is looking at the baby in front, low-frequency random phase patches are assigned to the pixels corresponding to the AR object C142 (i.e., pixels belonging to the focus group) in order to display the AR object C142 clearly and with high resolution.

[0081] Pixels corresponding to AR object C141 and OSD object C143, which are grouped in the out-of-focus group, are assigned high-frequency random phase patches that achieve a shallow depth of field so that they are displayed blurred to the same extent as real-world objects.

[0082] Note that the method of assigning phase values ​​to pixels belonging to the focus group and the method of assigning phase values ​​to pixels belonging to the out-of-focus group may be similar to the method described above using Figure 8, for example, and therefore detailed explanation will be omitted here.

[0083] 16 and 17 are diagrams showing examples of scenes that the user can see depending on the user's line of sight, with Fig. 16 being a diagram showing a scene that the user can see when the user's line of sight is directed toward the baby in the foreground, and Fig. 17 being a diagram showing a scene that the user can see when the user's line of sight is directed toward the boy in the background. Note that in Fig. 16 and 17, for areas R161 and R171 that correspond to the background other than objects C141 to C143 shown in Fig. 14, it is assumed that the user is looking directly at the real space through the optical see-through AR HMD.

[0084] 16, when the user 107 is looking at the baby in the foreground, low-frequency phase information is assigned to pixels corresponding to the baby AR object C142 grouped in the focus group, so that the baby AR object C162 that the user 107 is looking at is displayed clearly with a large depth of field.

[0085] In contrast, high-frequency phase information is assigned to the pixels corresponding to the boy's AR object C141 and the clock OSD object C143, which are grouped in the out-of-focus group, and a shallow depth of field is set for them, causing the boy's AR object C161 and the clock OSD object C163 to appear blurred.

[0086] 17, when the user 107 is looking at the boy in the background, low-frequency phase information is assigned to the pixels corresponding to the AR object C141 of the boy grouped in the focus group. As a result, the AR object C171 of the boy that the user 107 is looking at is displayed clearly with a large depth of field.

[0087] In contrast, high-frequency phase information is assigned to the pixels corresponding to the baby AR object C142 and the clock OSD object C143, which are grouped in the out-of-focus group, and a shallow depth of field is set for them, causing the baby AR object C171 and the clock OSD object C173 to appear blurred.

[0088] Note that the OSD object C143 may always be displayed with high resolution and a large depth of field regardless of the line of sight of the user 107. In this case, the area C153 corresponding to the OSD object C143 may be grouped into a group (hereinafter referred to as an OSD group) different from the focus group and the out-of-focus group.

[0089] The other configurations, operations, and effects may be the same as those of the above-described embodiment, and therefore detailed description thereof will be omitted here.

[0090] 4. Fourth Embodiment Next, an information processing device, an information processing method, and a program according to a fourth embodiment will be described in detail with reference to the drawings. In the following description, the same components as those in the above-described embodiment will be denoted by the same reference numerals, and redundant description thereof will be omitted.

[0091] Fig. 18 is a block diagram showing a schematic configuration example of an information processing device according to this embodiment. As shown in Fig. 18, the information processing device 4 according to this embodiment has the same configuration as the information processing device 1 described with reference to Fig. 2 in the first embodiment, except that it further includes a camera 60 and the preprocessing unit 20 further includes a characteristic detection unit 25. Note that in this embodiment, as in the second embodiment, the input image data 10 does not include a label image (see Fig. 6).

[0092] The camera 60 captures an image of the surroundings of the user 107, including the visual angle of the user 107, for example. The characteristic detection unit 25 generates a characteristic map indicating which area in real space the user 107 is likely to look at, based on the image of the surroundings of the user 107 captured by the camera 60 (hereinafter referred to as the surrounding image). An area that the user 107 is likely to look at is, for example, an area that is likely to attract the attention of the user 107, and in the characteristic map generated by the characteristic detection unit 25, a high characteristic value can be set for this area.

[0093] FIG. 19 is a diagram showing an example of a surrounding image acquired by a camera, and FIG. 20 is a diagram showing an example of a characteristic map generated by a characteristic detection unit according to this embodiment based on the surrounding image.

[0094] 19, the angle of view of the camera 60 includes, for example, the angle of view of the user 107. The angle of view of the camera 60 does not necessarily have to be wider than the angle of view of the user 107. The characteristic detection unit 25 generates a characteristic map as shown in FIG. 20 based on the surrounding image input from the camera 60.

[0095] In this example, as shown in the characteristic map G200 illustrated in FIG. 20, for example, the highest characteristic value is set for the area C202 corresponding to the baby area C192 in the surrounding image G190 shown in FIG. 19, the next highest characteristic value is set for the area C204 corresponding to the woman area C194, and the next highest characteristic value is set for the area C201 corresponding to the boy and man area C191.

[0096] Based on the characteristic map G200 generated as described above, the grouping unit 21 according to this embodiment groups the AR objects based on the positional relationship between each of the regions C201, C202, and C204 and each of the AR objects in the input image data 10. For example, as shown in Fig. 21 , the grouping unit 21 may group pixels corresponding to a baby AR object C212 corresponding to the region C202 of the characteristic map G200 into a group with the highest attention level (hereinafter referred to as a high attention group), and group pixels corresponding to other objects, in this example, a boy AR object C211 and a clock OSD object C213, into a group with a low attention level (hereinafter referred to as a low attention group).

[0097] In order to ensure that the highly visible AR object C202 is always displayed clearly with high resolution, the phase adjustment unit 22 assigns low-frequency phase values ​​to the pixels grouped into the highly visible group using the method described with reference to Figure 8.

[0098] On the other hand, in the normal state, in order to display the AR object C201 with a low attention level in a blurred manner without causing discomfort, the phase adjustment unit 22 assigns a high-frequency phase value to the pixels grouped in the low attention group by the method described with reference to Fig. 7. Note that the non-normal state, which is the opposite of the normal state, may be a state in which the user 107 is gazing at an object in real space corresponding to an AR object different from the AR objects grouped in the high attention group.

[0099] 22 and 23 are diagrams showing examples of scenes that the user can see depending on the user's line of sight, with Fig. 22 being a diagram showing a scene that the user can see when the user's line of sight is directed toward the baby in the foreground, and Fig. 23 being a diagram showing a scene that the user can see when the user's line of sight is directed toward the boy in the background. Note that in Fig. 22 and 23, for areas R221 and R231 that correspond to the background other than objects C211 to C213 shown in Fig. 21, it is assumed that the user is looking directly at the real space through the optical see-through AR HMD.

[0100] In Figures 22 and 23, the highly visible baby AR objects C222 and C232 are assigned low-frequency random phase patches, so that the baby AR objects C222 and C232 are displayed clearly with a large depth of field no matter where the user 107 is looking.

[0101] On the other hand, because high-frequency random phase patches are assigned to the boy's AR objects C221 and C231, which are less popular, the boy's AR objects C222 and C232 are displayed with a shallow depth of field. As a result, when the user 107 is looking at the boy, the boy's AR object C231 is displayed clearly as shown in Fig. 23, but when the user 107 is not looking at the boy, for example, when the user 107 is looking at a baby, the boy's AR object C231 is displayed blurry as shown in Fig. 22.

[0102] This embodiment is considered to be particularly effective when, for example, the user 107 always pays close attention to a specific object in real space or when it is necessary to pay close attention to it. Even in this case, an AR object for an object to which the user 107 is not paying attention is displayed with a clarity according to the focal position of the user 107, so that a natural user experience can be provided.

[0103] The other configurations, operations, and effects may be the same as those of the above-described embodiment, and therefore detailed description thereof will be omitted here.

[0104] 5. Fifth Embodiment Next, an information processing device, an information processing method, and a program according to a fifth embodiment will be described in detail with reference to the drawings. In the following description, the same components as those in the above-described embodiments will be denoted by the same reference numerals, and redundant description thereof will be omitted.

[0105] The information processing device according to this embodiment may have the same configuration as the information processing device 3 according to the third embodiment described above, for example. However, the information processing device according to this embodiment executes the following operations.

[0106] 24 and 25 are diagrams for explaining the operation performed by the information processing device according to this embodiment, where FIG. 24 is a diagram showing the direction in which the user is looking in the input image data, and FIG. 25 is a diagram for explaining an example of grouping objects based on the gaze direction shown in FIG. 24.

[0107] As shown in FIG. 24, in this embodiment, the gaze tracking unit 24 detects the gaze direction of the user 107 based on an image acquired by the camera 60, as in the third embodiment.

[0108] On the other hand, the grouping unit 21 according to this embodiment groups depth images in the input image data 10 based on depth information for each pixel, as shown in Fig. 25. In this example, similar to the example described above, an object image G240 includes a boy AR object C241, a baby AR object C242, and a clock OSD object C243, and different depth information is assigned to each object in the depth image F160. For example, a region C251 corresponding to the boy AR object C241 is assigned with depth information indicating that the distance from the user 107 is 2000 mm, a region C252 corresponding to the baby AR object C242 is assigned with depth information indicating that the distance from the user 107 is 1000 mm, and a region C253 corresponding to the clock OSD object C243 is assigned with depth information indicating that the distance from the user 107 is 500 mm.

[0109] In such a case, the grouping unit 21 groups the regions of each object in the input image data 10 based on the region C251 corresponding to the boy AR object C241, the region C252 corresponding to the baby AR object C242, and the region C253 corresponding to the clock OSD object C243 in the depth image G250.

[0110] In this way, by using the depth information included in the input image data 10, the grouping unit 21 can easily group the pixels of the input image data 10.

[0111] Similar to the above-described embodiment, the phase adjustment unit 22 assigns a phase value according to the line of sight direction V241 of the user 107 to pixels belonging to each group based on the groups for each distance generated as described above.

[0112] In this embodiment, a lookup table such as that shown in Table 1 below is used to set the phase value for each of the grouped pixels. [Table 1]

[0113] As shown in Table 1, in the lookup table, the maximum phase difference in the patch is defined for each distance from an object located in the line of sight V241 of the user 107, i.e., the object that the user 107 is looking at, as a reference. Note that the random phase patch assigned to a cell of a predetermined size may be the random phase patch described above with reference to FIG. 8 or a random phase patch obtained by adjusting this random phase patch based on the maximum phase difference identified from the lookup table (see, for example, FIG. 7).

[0114] Therefore, as shown in Figure 24, when user 107 is looking at a baby located 1000 mm away from him / her, a random phase patch (see, for example, Figure 7) in which the maximum phase difference within the patch is set to 1π is assigned to pixels belonging to a group corresponding to a boy AR object C241 located 1000 mm away from the baby, and a random phase patch (see, for example, Figure 8) in which the maximum phase difference within the patch is set to 1 / 4π is assigned to pixels belonging to a group corresponding to a clock OSD object C243 located 500 mm closer to the baby.

[0115] Note that the maximum phase difference within the patch is 0π, that is, a constant phase value with no phase difference is assigned to pixels belonging to the group corresponding to the baby AR object C242 that the user 107 is looking at.

[0116] In this way, by configuring each object to have a phase difference according to the distance from the object that the user 107 is looking at based on a lookup table, it is possible to provide the following user experience.

[0117] 26 and 27 are diagrams for explaining the user experience provided to the user by this embodiment, Fig. 26 is a diagram showing a scene seen by the user when the user is looking at the baby in the foreground, and Fig. 27 is a diagram showing a scene seen by the user when the user is looking at the boy in the background. Note that in Fig. 26 and Fig. 27, for areas R261 and R271 corresponding to the background other than objects C241 to C243 illustrated in Fig. 24, it is assumed that the user is looking directly at the real space via the optical see-through AR HMD.

[0118] 26, when the user 107 is looking at the baby in the foreground, a random phase patch with a very low frequency (no phase difference in this example) is assigned to the pixel corresponding to the baby's AR object C242. As a result, the baby's AR object C262 that the user 107 is looking at is displayed clearly with a large depth of field.

[0119] On the other hand, pixels corresponding to objects C241 and C243 (in this example, a boy and a clock) whose distances from the user 107 are different from that of the baby are assigned high-frequency random phase patches whose maximum phase difference in the patch corresponds to the distance of each object from the baby (or the user 107), thereby setting a shallow depth of field. For example, a high-frequency random phase patch with a maximum phase difference of 1π corresponding to a range of +1000 mm from the baby is assigned based on the lookup table of Table 1 to pixels corresponding to the boy's AR object C241, and a high-frequency random phase patch with a maximum phase difference of 1 / 4π corresponding to a range of -500 mm from the baby is assigned based on the lookup table of Table 1 to pixels corresponding to the clock's OSD object C243. As a result, the objects corresponding to these objects (the boy's AR object C261 and the clock's OSD object C263) are displayed blurred.

[0120] 27, when the user 107 is looking at the boy in the background, a random phase patch with a very low frequency (no phase difference in this example) is assigned to the pixel corresponding to the boy's AR object C242. As a result, the boy's AR object C272 that the user 107 is looking at is displayed clearly with a large depth of field.

[0121] On the other hand, pixels corresponding to objects C241 and C243 (in this example, a baby and a clock) whose distances from the user 107 are different from that of the boy are assigned high-frequency random phase patches whose maximum phase difference in the patch corresponds to the distance of each object from the boy (or user 107), thereby setting a shallow depth of field. For example, a high-frequency random phase patch with a maximum phase difference of 1π corresponding to a range in which a distance of −1000 mm from the boy belongs is assigned to pixels corresponding to the baby AR object C242 based on the lookup table of Table 1, and a high-frequency random phase patch with a maximum phase difference of 2 / 3π corresponding to a range in which a distance of −1500 mm from the boy belongs is assigned to pixels corresponding to the clock OSD object C243 based on the lookup table of Table 1. As a result, the objects corresponding to these objects (the baby AR object C272 and the clock OSD object C273) are displayed blurred.

[0122] The other configurations, operations, and effects may be the same as those of the above-described embodiment, and therefore detailed description thereof will be omitted here.

[0123] 6. Hardware configuration The information processing devices according to the above-described embodiments, modifications, and applications can be realized by, for example, a computer 1000 configured as shown in Fig. 28. Fig. 28 is a hardware configuration diagram showing an example of the computer 1000 that realizes the functions of the information processing device according to the above-described embodiments. The computer 1000 has a CPU 1100, a RAM 1200, a ROM (Read Only Memory) 1300, an HDD (Hard Disk Drive) 1400, a communication interface 1500, and an input / output interface 1600. The components of the computer 1000 are connected by a bus 1050.

[0124] The CPU 1100 operates and controls each unit based on programs stored in the ROM 1300 or the HDD 1400. For example, the CPU 1100 loads the programs stored in the ROM 1300 or the HDD 1400 into the RAM 1200 and executes processing corresponding to the various programs.

[0125] The ROM 1300 stores boot programs such as a BIOS (Basic Input Output System) that is executed by the CPU 1100 when the computer 1000 is started up, and programs that depend on the hardware of the computer 1000 .

[0126] HDD 1400 is a computer-readable recording medium that non-temporarily records programs executed by CPU 1100 and data used by such programs. Specifically, HDD 1400 is a recording medium that records a projection control program according to the present disclosure, which is an example of program data 1450.

[0127] The communication interface 1500 is an interface for connecting the computer 1000 to an external network 1550 (e.g., the Internet). For example, the CPU 1100 receives data from other devices and transmits data generated by the CPU 1100 to other devices via the communication interface 1500.

[0128] The input / output interface 1600 is an interface for connecting the input / output device 1650 and the computer 1000. For example, the CPU 1100 receives data from an input device such as a keyboard or a mouse via the input / output interface 1600. The CPU 1100 also transmits data to an output device such as a display, a speaker, or a printer via the input / output interface 1600. The input / output interface 1600 may also function as a media interface for reading programs and the like recorded on a predetermined recording medium. Examples of media include optical recording media such as a DVD (Digital Versatile Disc) or a PD (Phase Change Rewritable Disk), magneto-optical recording media such as an MO (Magneto-Optical disk), tape media, magnetic recording media, and semiconductor memories.

[0129] For example, when the computer 1000 functions as the information processing device according to the above-described embodiment, the CPU 1100 of the computer 1000 executes programs loaded onto the RAM 1200 to realize the functions of the pre-processing unit 20, the CGH calculation unit 30, and the post-processing unit 40. The HDD 1400 stores programs and the like according to the present disclosure. The CPU 1100 reads and executes program data 1450 from the HDD 1400, but as another example, the CPU 1100 may obtain these programs from another device via an external network 1550.

[0130] Although the embodiments of the present disclosure have been described above, the technical scope of the present disclosure is not limited to the above-described embodiments, and various modifications are possible within the scope of the gist of the present disclosure. Furthermore, components of different embodiments and modifications may be combined as appropriate.

[0131] Furthermore, the effects of each embodiment described in this specification are merely examples and are not intended to be limiting, and other effects may also be obtained.

[0132] The present technology can also be configured as follows. (1) a grouping unit that groups a plurality of pixels that constitute one or more objects included in one image data into one or more groups; a phase adjustment unit that assigns a phase pattern with a different phase difference to the plurality of pixels for each of the one or more groups; a calculation unit that generates hologram data from the image data to which the phase pattern is added; An information processing device comprising: (2) The information processing device according to (1), wherein the phase adjustment unit assigns the phase patterns with different maximum phase differences to the plurality of pixels for each of the one or more groups. (3) the image data includes label information for the one or more objects; The grouping unit groups the plurality of pixels into the one or more groups based on the label information. The information processing device according to (1) or (2). (4) An image analysis unit that analyzes the image data and assigns a label to each pixel that constitutes the image data, The grouping unit groups the plurality of pixels into the one or more groups based on the labels assigned by the image analysis unit. The information processing device according to (1) above. (5) Further comprising an eye tracking unit that detects the eye direction of the user, The grouping unit groups, into a first group, pixels constituting an object associated with an object located in the line of sight direction in real space, among the one or more objects, and groups, into a second group, pixels constituting an object associated with an object not located in the line of sight direction. The information processing device according to (1) above. (6) an imaging unit that captures an image of the user's surroundings; a characteristic detection unit that detects characteristics of an object present around the user by analyzing image data acquired by the imaging unit; Furthermore, The grouping unit groups the plurality of pixels constituting the one or more objects associated with the object into the one or more groups based on the characteristics of the object detected by the characteristic detection unit. The information processing device according to (1) above. (7) the image data includes depth information for each pixel that constitutes the image data, The grouping unit groups the plurality of pixels into the one or more groups based on the depth information. The information processing device according to (1) above. (8) an eye tracking unit that detects the user's eye gaze direction; a storage unit that stores a lookup table that holds a correspondence relationship between distance and phase difference; Furthermore, The phase adjustment unit determines the phase difference of the phase pattern to be assigned to the pixels belonging to each of the one or more groups by referring to the lookup table based on a distance between an object to which an object constituted by the pixels grouped into each of the one or more groups is associated and an object located in the line of sight in real space. The information processing device according to (7) above. (9) The phase adjustment unit assigning a first phase pattern, in which the phase difference is a first phase difference, to the pixels grouped into the first group by the grouping unit; A second phase pattern, in which the phase difference is a second phase difference greater than the first phase difference, is assigned to the pixels grouped into the second group. The information processing device according to any one of (1) to (7) above. (10) the phase pattern has a predetermined pixel size; The phase adjustment unit assigns the phase pattern to each cell obtained by dividing the image data into the predetermined pixel size. The information processing device according to any one of (1) to (9) above. (11) The information processing device according to any one of (1) to (10), wherein the phase pattern is a random phase pattern in which the difference between phase values ​​assigned to adjacent pixels is a random value. (12) The information processing device described in any one of (1) to (11), wherein the one or more objects include a first object corresponding to an object in real space and a second object displayed on a virtual screen set in the real space. (13) The information processing device according to (12), wherein the phase adjustment unit assigns a low-frequency phase pattern with a small phase difference to pixels corresponding to the second object. (14) Grouping a plurality of pixels constituting one or more objects included in one image data into one or more groups; assigning a phase pattern having a different phase difference to each of the one or more groups to the plurality of pixels; generating hologram data from the image data to which the phase pattern has been added; An information processing method including: (15) A program for causing a computer to function to generate hologram data for displaying a hologram from image data, A step of grouping a plurality of pixels constituting one or more objects included in one image data into one or more groups; assigning a phase pattern with a different phase difference to each of the one or more groups to the plurality of pixels; generating hologram data from the image data to which the phase pattern has been added; A program for causing the computer to execute the above. (16) The information processing device according to any one of (1) to (13), a display device that displays a hologram to a user based on the hologram data generated by the information processing device; and A hologram display system comprising: (17) the information processing device further includes a post-processing unit that converts the hologram data into a hologram signal that can be stereoscopically displayed on the display device; The display device includes a spatial light modulator that modulates light output based on the hologram signal to display the hologram to a user. The hologram display system according to (16) above. (18) The hologram display system according to (17), wherein the spatial light modulator is a reflective spatial light modulator. [Explanation of symbols]

[0133] 1, 2, 3, 4 Information processing device 10 Input image data 20 Pretreatment section 21 Grouping section 22 Phase adjustment section 23 Image analysis unit 24 Eye tracking unit 25 Characteristics detection unit 30 CGH calculation section 40 Post-processing section 50 Display device 60 cameras 101 Light source 102, 103 lenses 104 Beam Splitter 105 Spatial Light Modulator (SLM) 106, 106a, 106b, 106c holograms 107 users

Claims

1. a grouping unit that groups a plurality of pixels that constitute one or more objects included in one image data into a first group that requires high resolution and a second group that requires shallow depth of field; a phase adjustment unit that assigns a first phase pattern to the first group and a second phase pattern having a larger phase difference than the first phase pattern to the second group of the plurality of pixels; a calculation unit that generates hologram data from the image data to which the first phase pattern and the second phase pattern are assigned; An information processing device comprising:

2. the image data includes label information for the one or more objects; The grouping unit groups the plurality of pixels into the first group and the second group based on the label information. The information processing device according to claim 1 .

3. An image analysis unit that analyzes the image data and assigns a label to each pixel that constitutes the image data, The grouping unit groups the plurality of pixels into the first group and the second group based on the labels assigned by the image analysis unit. The information processing device according to claim 1 .

4. Further comprising an eye tracking unit that detects the eye direction of the user, The grouping unit groups, into the first group, pixels constituting an object associated with an object located in the line of sight direction in real space, among the one or more objects, and groups, into the second group, pixels constituting an object associated with an object not located in the line of sight direction. The information processing device according to claim 1 .

5. an imaging unit that captures an image of the user's surroundings; a characteristic detection unit that detects characteristics of an object present around the user by analyzing image data acquired by the imaging unit; Furthermore, The grouping unit groups the plurality of pixels constituting the one or more objects associated with the object into the first group and the second group based on the characteristics of the object detected by the characteristic detection unit. The information processing device according to claim 1 .

6. the image data includes depth information for each pixel that constitutes the image data, The grouping unit groups the plurality of pixels into the first group and the second group based on the depth information. The information processing device according to claim 1 .

7. an eye tracking unit that detects the user's eye gaze direction; a storage unit that stores a lookup table that holds a correspondence relationship between distance and phase difference; Furthermore, The phase adjustment unit determines the phase differences of the first phase pattern and the second phase pattern to be applied to the pixels belonging to the first group and the second group, respectively, by referring to the lookup table based on a distance between an object to which an object constituted by the pixels grouped into each of the one or more groups is associated and an object located in the line of sight in real space. The information processing device according to claim 6 .

8. The phase adjustment unit assigning the first phase pattern, in which the phase difference is a first phase difference, to the pixels grouped into the first group by the grouping unit; The second phase pattern, in which the phase difference is a second phase difference greater than the first phase difference, is assigned to the pixels grouped into the second group. The information processing device according to claim 1 .

9. the first phase pattern and the second phase pattern have a predetermined pixel size; The phase adjustment unit assigns the first phase pattern and the second phase pattern to each cell obtained by dividing the image data into the predetermined pixel size. The information processing device according to claim 1 .

10. The information processing apparatus according to claim 1 , wherein the first phase pattern and the second phase pattern are random phase patterns in which the difference between the phase values ​​assigned to adjacent pixels is a random value.

11. The information processing device according to claim 1 , wherein the one or more objects include a first object associated with an object in a real space, and a second object displayed on a virtual screen set in the real space.

12. The information processing apparatus according to claim 11 , wherein the phase adjustment unit assigns the first phase pattern to pixels corresponding to the second object.

13. A plurality of pixels constituting one or more objects included in one image data are grouped into a first group requiring high resolution and a second group requiring shallow depth of field; assigning a first phase pattern to the first group and a second phase pattern having a phase difference larger than that of the first phase pattern to the second group for the plurality of pixels; generating hologram data from the image data to which the first phase pattern and the second phase pattern are assigned; An information processing method including:

14. A program for causing a computer to function to generate hologram data for displaying a hologram from image data, A step of grouping a plurality of pixels constituting one or more objects included in one image data into a first group requiring high resolution and a second group requiring shallow depth of field; assigning a first phase pattern to the first group and a second phase pattern having a phase difference larger than that of the first phase pattern to the second group for the plurality of pixels; generating hologram data from the image data to which the first phase pattern and the second phase pattern are assigned; A program for causing the computer to execute the above.

15. The information processing device according to claim 1 ; a display device that displays a hologram to a user based on the hologram data generated by the information processing device; and A hologram display system comprising:

16. the information processing device further includes a post-processing unit that converts the hologram data into a hologram signal that can be stereoscopically displayed on the display device; The display device includes a spatial light modulator that modulates light output based on the hologram signal to display the hologram to a user.

16. The holographic display system of claim 15.

17. 17. The holographic display system of claim 16, wherein the spatial light modulator is a reflective spatial light modulator.

Images