Display device, display system, and information processing device
Patent Information
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- SONY SEMICON SOLUTIONS CORP
- Filing Date
- 2026-01-21
- Publication Date
- 2026-08-06
Smart Images
Figure JP2026001872_06082026_PF_FP_ABST
Abstract
Description
Display device, display system, and information processing device
[0001] The present disclosure relates to a display device, a display system, and an information processing device.
[0002] Spatial light modulators (SLMs) are widely used to dynamically output hologram reproduction images. There are also display devices that increase the size of the reproduced image and / or widen the diffraction angle of the reproduced image by arranging a plurality of SLM panels. The image reproduced by an SLM must consider phase information, and the influence of misalignment of at least one of the position or orientation of the display panel on the reproduced image is significant, compared to a display that displays general video information.
[0003] To suppress misalignment of the SLM panel, for example, a method of adjusting the position and orientation for each SLM module integrated with an optical system is known. However, according to this method, it is necessary to attach an optical system to the SLM panel, and this optical system becomes very large. Furthermore, in the control of lenses and the like that form the optical system, it is necessary to control distortion of the lenses and the like, and when trying to widen the viewing angle, the diffraction angle of the SLM module itself must be widened, resulting in problems such as both the SLM panel and the optical system of lenses and the like becoming larger.
[0004] Japanese Patent Application Laid-Open No. 2017-058584
[0005] Therefore, one of the non-limiting problems to be solved by the embodiments of the present disclosure is to improve the image quality of the formed three-dimensional reproduction image. The problems to be solved by the embodiments of the present disclosure can also be, as some further non-limiting examples, problems corresponding to the effects described in the embodiments. That is, the problems corresponding to any at least one of the effects described in the description of the embodiments of the present disclosure can be the problems to be solved in the present disclosure.
[0006] According to one embodiment, the display device comprises a memory circuit, a processing circuit, and a spatial light modulator. The memory circuit stores error data indicating the deviation of the spatial light modulator from a reference. The processing circuit acquires information of the input wavefront input to the spatial light modulator based on the error data stored in the memory circuit, and generates a pattern to be displayed on the spatial light modulator based on the input wavefront information.
[0007] The error data may include at least one of the positional error of the spatial light modulator from a reference position, or the angular error of the spatial light modulator from a reference orientation.
[0008] The processing circuit may acquire reference data relating to the reference wavefront input to the spatial light modulator when there is no deviation, or it may acquire information about the input wavefront input to the spatial light modulator that takes the deviation into account, based on the reference data and the error data.
[0009] The information about the input wavefront may be expressed as a complex amplitude.
[0010] The information about the input wavefront may include at least one of the following: amplitude or phase.
[0011] The processing circuit may acquire information about the input wavefront having a size that can cover the entire effective display area of the spatial light modulator.
[0012] The processing circuit may acquire information about the input wavefront that is the same size as the effective display area of the spatial light modulator.
[0013] The memory circuit may acquire the error data along with the information of the reference wavefront.
[0014] The system may further include an input interface, through which information on the position or orientation of the spatial light modulator may be obtained from the user, and the processing circuit may obtain information on the input wavefront based on at least one of the position or orientation information of the spatial light modulator obtained through the input interface, and generate the pattern.
[0015] The processing circuit may generate the pattern for a predetermined display pattern, apply the pattern to the spatial light modulator, or obtain input from a user who has observed the displayed image via the input interface.
[0016] According to one embodiment, the display system comprises a plurality of the display devices described above. Each of the plurality of display devices acquires information on the input wavefront that is input to it, and generates the pattern to be displayed on each of the spatial light modulators based on the acquired information on the input wavefront.
[0017] The plurality of display devices may be arranged in a tile-like pattern.
[0018] According to one embodiment, the display device comprises a memory circuit, a processing circuit, and a plurality of spatial light modulation panels. The memory circuit stores error data indicating the deviation of each of the spatial light modulation panels from a reference. The processing circuit acquires information on the input wavefront input to each of the spatial light modulation panels based on the error data stored in the memory circuit, and generates a pattern to be displayed on each of the spatial light modulation panels based on the input wavefront information.
[0019] The spatial light modulation panels may be arranged in a tile-like pattern.
[0020] According to one embodiment, the information processing device comprises a memory circuit and a processing circuit, and generates a pattern to be displayed on a spatial light modulator. The memory circuit stores error data indicating the deviation of the spatial light modulator from a reference. The processing circuit acquires information of the input wavefront input to the spatial light modulator based on the error data stored in the memory circuit, and generates a pattern to be displayed on the spatial light modulator based on the input wavefront information.
[0021] A schematic diagram showing an example of a display device according to one embodiment. A schematic diagram showing an example of a display device according to one embodiment. A schematic diagram showing an example of a display system according to one embodiment. A schematic block diagram showing an example of a display device according to one embodiment. A flowchart showing an example of processing circuit processing according to one embodiment. A diagram showing an example of a reconstructed image by a spatial light modulator according to one embodiment. A diagram showing an example of a reconstructed image by a spatial light modulator according to one embodiment. A diagram showing an example of a reconstructed image by a spatial light modulator according to one embodiment. A diagram showing an example of a reconstructed image by a spatial light modulator according to one embodiment. A diagram showing an example of a reconstructed image by a spatial light modulator according to one embodiment. A diagram showing an example of a wavefront input to a spatial light modulator according to one embodiment. A diagram showing an example of a wavefront input to a spatial light modulator according to one embodiment. A front view showing an example of a spatial light modulator according to one embodiment. A front view showing an example of a spatial light modulator according to one embodiment. A front view showing an example of a spatial light modulator according to one embodiment. A front view showing an example of a spatial light modulator according to one embodiment. A front view showing an example of a spatial light modulator according to one embodiment. A front view showing an example of a spatial light modulator according to one embodiment. A schematic block diagram showing an example of a display device according to one embodiment. A flowchart showing an example of processing circuit processing according to one embodiment.
[0022] The embodiments of this disclosure will now be described with reference to the drawings. The drawings are for illustrative purposes only, and the shape, size, or size ratio of each component in the actual device does not need to be exactly as shown in the drawings. Furthermore, the drawings are simplified, so any other components necessary for implementation should be appropriately provided in addition to those shown in the drawings.
[0023] This disclosure will describe embodiments in the following order: 1. Display device and display system 2. Hologram generation and correction 3. Input wavefront 4. Other examples of device configuration
[0024] <1. Display devices and display systems>
[0025] Figure 1 is a schematic diagram showing an example, not limited to, a display device according to one embodiment. The display device 1 includes a spatial light modulator. The display device 1, for example, displays a dot pattern on the spatial light modulator and generates a three-dimensional image perceptible to an observer by the transmitted wave of light irradiated from the back of the displayed dot pattern, or the reflected wave of light irradiated from the front of the dot pattern. The dot pattern displayed on the spatial light modulator may be formed, for example, by a general computer-generated hologram (CGH) generation method.
[0026] By observing this image, users can view the content as three-dimensional information. For example, as shown in the diagram, users in different locations can view the content from different angles by looking at this holographic image.
[0027] In the embodiment shown in Figure 1, if at least one of the position or orientation is not properly set, the display device 1 will output an image observed from a different position or angle to each user than the intended observation result. For this reason, it is desirable that appropriate calibration be performed on at least one of the position or orientation of the display device 1.
[0028] Furthermore, the position and / or orientation of the spatial light modulator within the display device 1 may shift. In this case, irradiating the shifted spatial light modulator with light that would produce a suitable image if it were positioned in the desired location and / or orientation may cause the image to shift in position and / or angle, or to become blurred.
[0029] Figure 2 is a schematic diagram showing another, not limited, example of a display device 1 according to one embodiment. The display device 1 may have a configuration in which panels of multiple spatial light modulators 10 are arranged in a tile-like manner. That is, a single display device 1 may be equipped with multiple spatial light modulators 10, and the multiple spatial light modulators 10 may each form a panel and be arranged in a tile-like manner.
[0030] The display device 1 can form a larger, unified three-dimensional image of the multiple spatial light modulators 10 by displaying information corresponding to the position of each of the multiple spatial light modulators 10. In this embodiment, each of the spatial light modulators 10 must be positioned in an appropriate location and orientation. If at least one of the spatial light modulators 10 is not positioned in an appropriate location or orientation, it may not be possible to observe a proper three-dimensional image in the range of light diffracted by that panel.
[0031] Figure 3 is a schematic diagram illustrating an unspecified example of a display system according to one embodiment. The display system 2 is configured to include a plurality of display devices 1. Thus, the display system 2, which is formed as one large display device, may be configured to include a plurality of display devices 1.
[0032] In this case as well, as in Figure 2, it is desirable that the spatial light modulators 10 of each display device 1 be arranged in a tile-like pattern at appropriate positions and orientations. If at least one panel of the display device 1 is not positioned or oriented appropriately, it may not be possible to observe a proper stereoscopic image in the range of light diffracted by that panel.
[0033] In the display device 1 according to this disclosure, wavefront information input to each spatial light modulator 10 is acquired based on the position and / or orientation error of the spatial light modulator 10, and a display pattern is generated based on this input wavefront information to achieve appropriate imaging.
[0034] <2. Hologram Generation>
[0035] (First Embodiment)
[0036] Figure 4 is a schematic block diagram showing an example of a display device 1 according to one embodiment. The display device 1 includes, for example, a spatial light modulator 10, a drive circuit 12, a processing circuit 14, and a memory circuit 16. Based on position and / or orientation deviation information (error information) of the spatial light modulator 10, the display device 1 acquires what kind of wavefront is actually incident on the spatial light modulator 10 due to the effect of this deviation, compared to when the spatial light modulator 10 is in an ideal position and orientation, and generates a pattern to be displayed on the spatial light modulator 10 based on this incident wavefront information.
[0037] In the following, the wavefront incident on the spatial light modulator 10 at an ideal position and orientation is referred to as the reference wavefront. The wavefront actually incident on the spatial light modulator 10, which has a position and / or orientation deviation, when illuminated by light from the reference wavefront, is referred to as the input wavefront. That is, by displaying an appropriate display pattern on the spatial light modulator 10 with respect to the input wavefront, it is possible to form an image that corrects for the position and / or orientation deviation.
[0038] The spatial light modulator 10 is a modulator for displaying patterns. The display device 1 can, for example, display a dot pattern created using the CGH method on the spatial light modulator 10 and form a three-dimensional image by irradiating it with light having an appropriate wavefront.
[0039] The drive circuit 12 is a circuit for driving the spatial light modulator 10. For example, the drive circuit 12 drives the spatial light modulator 10 to display a pattern based on data output from the processing circuit 14. Alternatively, the drive circuit 12 may be configured to read pattern data stored in the memory circuit 16 and drive the spatial light modulator 10 to display it. The drive circuit 12 can display any pattern by driving the spatial light modulator 10 to form a pattern in each region (e.g., a pixel).
[0040] The processing circuit 14 is a circuit that executes processing related to the display of the pattern of the spatial light modulator 10. This processing circuit 14 may be configured to include a dedicated circuit such as an application-specific integrated circuit (ASIC), or may be configured to include a general-purpose circuit such as a central processing unit (CPU) or a graphics processing unit (GPU).
[0041] For example, the processing circuit 14 acquires information on the incident wavefront based on information on the reference wavefront and error data related to the position and orientation of the spatial light modulator 10, and generates a pattern to be displayed on the spatial light modulator 10 based on the acquired information on the incident wavefront.
[0042] The storage circuit 16 includes an area for storing data necessary for display in the spatial light modulator 10. The storage circuit 16 stores, for example, data related to at least one of the deviations in the position or orientation of the spatial light modulator 10. The storage circuit 16 stores, for example, data related to the error in position from the reference position of the spatial light modulator 10 and / or the error in orientation from the reference orientation.
[0043] That is, the storage circuit 16 stores data including at least one of the error in position from the reference position of the spatial light modulator 10 or the error in angle from the reference orientation of the spatial light modulator 10. The position information may be, for example, information on three axes in a Cartesian coordinate system. The angle information may be, for example, information on the angle around each axis in the same coordinate system as the position information.
[0044] However, the data format is not limited to these, and for example, other expressions suitable for calculations, such as coordinate systems such as spherical coordinate systems and cylindrical coordinate systems, or expressions of angles such as direction cosines and Euler angles with respect to the reference orientation, can also be used.
[0045] The processing circuit 14 calculates information on the input wavefront based on the error data related to the position and / or orientation stored in the storage circuit 16 and the information on the reference wavefront. The processing circuit 14 generates a pattern to be displayed on the spatial light modulator 10 based on the information on the input wavefront.
[0046] Further, the memory circuit 16 may store information such as the wavelength of the input light, pixel pitch, and number of pixels, which are information used in the calculations described later, together with the position information of the spatial light modulator 10 and the attitude information of the spatial light modulator 10. The processing circuit 14 can appropriately obtain necessary information from the memory circuit 16 and execute calculations.
[0047] For example, the processing circuit 14 may generate a pattern based on a stereoscopic image to be observed by the user. As another example, the processing circuit 14 may generate a pattern to be displayed on the spatial light modulator 10 by correcting the information of the pattern based on the already generated stereoscopic image based on the information of the input wavefront with respect to the reference wavefront.
[0048] FIG. 5 is a flowchart showing an example of the processing of the processing circuit 14 according to an embodiment. First, the processing circuit 14 acquires reference data, which is information on the reference wavefront (S100). The reference data is the information on the wavefront input to the spatial light modulator 10 when the spatial light modulator 10 is arranged at an appropriate position (reference position) and in an appropriate attitude (reference attitude) as described above.
[0049] For example, the processing circuit 14 acquires the reference data related to the reference wavefront via an interface. When the reference wavefront is a predetermined wavefront, the information on this reference wavefront may be stored in the memory circuit 16 in advance, and the processing circuit 14 may acquire the reference data stored in the memory circuit 16.
[0050] The processing circuit 14 acquires error data of the spatial light modulator 10, which has been measured and stored in advance, from the memory circuit 16 (S102). The error data is information including at least one of the position error from the appropriate position of the spatial light modulator 10 or the angular error from the appropriate attitude of the spatial light modulator 10.
[0051] The processing circuit 14 acquires input wavefront information based on the acquired reference data and error data related to the reference wavefront (S104). For example, the processing circuit 14 calculates what shape of light arrives at the same time in a spatial light modulator 10 that is positioned and / or positioned in an orientation with an error, and acquires input wavefront information for the spatial light modulator 10 that has errors in position and / or orientation based on this calculation result. If multiple display devices 1 or multiple spatial light modulators 10 are provided, the processing circuit 14 can also acquire input wavefront information that takes into account the time delay from the reference wavefront.
[0052] The input wavefront information may be represented as a complex amplitude, for example, without limiting it. Alternatively, the input wavefront information may include at least one of either amplitude information or phase information, for example, without limiting it.
[0053] The processing circuit 14 generates a pattern to be displayed on the spatial light modulator 10 based on the acquired input wavefront information (S106). As described above, the pattern can be obtained by performing calculations based on the input stereoscopic image information and the input wavefront information, or by correcting a pattern previously created by CGH or the like based on the input wavefront information.
[0054] Next, we will explain the error data in more detail using diagrams.
[0055] Figure 6 shows the reference position and orientation, and Figures 7 to 10 show spatial light modulators 10 with errors (deviations) as some non-limiting examples. Note that although these figures use a transmissive hologram as an example, errors can be similarly defined for reflective holograms. Also, as shown in these figures, the x, y, and z axes of the Cartesian coordinate system may be defined as non-limiting representations.
[0056] As shown in Figure 6, the spatial light modulator 10 is positioned at a reference position and in a reference orientation, and displays a display pattern to form an appropriate three-dimensional image when the reference wavefront is input from the back (or the front in the case of a reflected wave; although this is omitted below, the case of a reflected wave is also considered). The reference wavefront has a spherical distortion, but is not limited to this; it may be a plane wave or have other appropriate wavefront shapes for illuminating the hologram.
[0057] By positioning the spatial light modulator 10 in an appropriate location and orientation, and irradiating the spatial light modulator 10 with transmitted or reflected light having a reference wavefront, a three-dimensional image can be formed at an appropriate position.
[0058] Figure 7 shows an example of a shift in the z-direction. The spatial light modulator 10 shifts in the z-direction, causing a shift in the z-direction of the stereoscopic image. As a result, observers may perceive the image at a position shifted in the z-direction when observing the stereoscopic image, or they may perceive a blurred image where the focus is on a different point when trying to focus at the position where the original stereoscopic image should be reproduced.
[0059] Figure 8 shows an example of a shift in the x and y directions as an example of a misalignment. In such a case, the observer will observe a three-dimensional image at a position shifted in the x and y directions, similar to the direction in which the spatial light modulator 10 is shifted, in addition to the shift in the z direction mentioned above. If there are multiple spatial light modulators 10, the observer will observe a three-dimensional image at a position where the shifted spatial light modulator 10 does not align with the other spatial light modulators 10. As a result, a problem occurs in which an unnatural three-dimensional image is observed within the range in which the diffracted light from the spatial light modulator 10 is observed.
[0060] Figure 9 shows an example where the position is correct but the orientation is misaligned. The orientation misalignment may be expressed, for example, as roll (angular misalignment around the x-axis), pitch (angular misalignment around the y-axis), and yaw (angular misalignment around the z-axis), with respect to the plane over which the spatial light modulator 10 extends. When there is an angular misalignment, the position at which the stereoscopic image is formed is shifted, or the angle at which the stereoscopic image is formed is shifted.
[0061] Figure 10 shows an example of a discrepancy in position and orientation. This discrepancy is a combination of the positional and orientation discrepancies described above, and when there are discrepancies in both position and orientation, positional, focal, and angular discrepancies in the image may occur simultaneously.
[0062] In processing S104, the processing circuit 14 corrects the various deviations shown in Figures 7 to 10 by modifying the display pattern without correcting the position and orientation of the spatial light modulator 10, thereby enabling the observation of an appropriate reconstructed image. The wavefront information input to the spatial light modulator 10 can be obtained from the information of each deviation.
[0063] The processing circuit 14 can perform the above calculations by obtaining the necessary data from the various deviation data stored in the memory circuit 16. The deviation data stored in the memory circuit 16 may be data that has been measured in advance using various methods.
[0064] Figures 11 and 12 show non-limiting examples of how a reference wavefront input to a spatial light modulator 10 having positional and orientation deviations propagates to form an input wavefront when it is in an ideal position. Figure 11 shows the input wavefront when the spatial light modulator 10 is a transmission type, and Figure 12 shows a hypothetical representation of the input wavefront from a reflected state when the spatial light modulator 10 is a reflection type.
[0065] As shown in Figure 11, in a transmissive device, light with the appropriate wavefront is input from the side of the spatial light modulator 10 opposite to the side the user observes (the back side), and this light forms a three-dimensional reconstructed image. As shown in Figure 12, in a reflective device, light with the appropriate wavefront is input from the side of the spatial light modulator 10 that the user observes (the front side), and this light forms a three-dimensional reconstructed image.
[0066] The processing circuit 14 calculates how the reference wavefront is actually input to the spatial light modulator 10 in each case and obtains information about the input wavefront. Then, based on this information about the input wavefront, the processing circuit 14 calculates a pattern to be displayed on the spatial light modulator 10 so that an appropriate reconstructed image (for example, the reconstructed image shown in Figure 6) can be observed when an appropriate wavefront is input.
[0067] For example, in response to a shift in the z-direction, the processing circuit 14 acquires information about the input wavefront by accelerating or delaying the wavefront of the light reaching the spatial light modulator 10 in time, that is, by appropriately changing the propagation distance. The processing circuit 14 takes the right side of the axis shown in the figure as the positive direction, and for example, in the case of a transmissive type, it can generate the input wavefront by shifting the wavefront by z, and in the case of a reflective type, it can generate the input wavefront by shifting the wavefront by -z.
[0068] For example, in response to deviations in the x and y directions, the processing circuit 14 acquires information about the input wavefront by shifting the center position of the wavefront of the light reaching the spatial light modulator 10. The processing circuit 14 can generate the input wavefront, for example, by shifting the reference wavefront by (-x, -y).
[0069] For example, in response to a yaw angle shift, the processing circuit 14 acquires information about the input wavefront by rotating the wavefront of the light reaching the spatial light modulator 10. If the yaw angle shift is γ, the rotation angle of the reconstructed image will also be γ, so the processing circuit 14 can acquire information about the input wavefront by, for example, rotating the reference wavefront by -γ.
[0070] For example, in the case of deviations in pitch angle and roll angle, that is, the inclination (α, β) of the plane itself formed by the spatial light modulator 10 from the reference attitude, the processing circuit 14 can generate an input wavefront by appropriately shifting the phase of the wavefront to match the inclination of the spatial light modulator 10.
[0071] The processing circuit 14 calculates the pattern to be displayed on the spatial light modulator 10 based on the input wavefront information acquired as described above. It is not necessary to separate the generation of input wavefront information from the calculation of the display pattern; a method may be used to directly calculate the pattern for deviations in the z, x, and y directions, yaw angle, or pitch and roll angles.
[0072] The processing circuit 14 can correct the displayed pattern by performing diffraction calculations at the positions shifted by the z (transmission type) or -z (reflection type) values described above if there is a shift in the z direction. The processing circuit 14 can correct the displayed pattern by performing shift analysis calculations to have shifts in the x and / or y directions, respectively, if there are shifts in the x and / or y directions.
[0073] The following describes methods for generating or correcting display patterns as examples, not limited to, each type of discrepancy. However, the forms of this disclosure are not limited to these examples, and CGH can also be generated or corrected using other methods.
[0074] For example, using shift Fresnel diffraction calculations, the calculation can be performed according to the following formula. In this method, the calculation formula applied differs depending on the propagation distance z.
[0075] Here, u1 is the source wavefront (reference wavefront), u2 is the destination wavefront (input wavefront), λ is the wavelength, p is the pixel pitch, N is the number of samples in the x and y axes, ox and oy are the parallel shift amounts in the destination, and s is the scale factor of the sampling interval of the source wavefront. Also, the script forms F and F -1The terms ∫ and ∫ represent the Fourier transform and inverse Fourier transform, respectively. Functions shown in uppercase represent the frequency-space functions of the functions shown in lowercase. z >= zc (= 2Nsp 2 In the case of / λ), equations (1) to (4) apply.
[0076] Here, the following equation may also be satisfied.
[0077] z < zc (= 2Np 2 In the case of / λ), equations (9) and (10) can be applied to the processing circuit 14.
[0078] Next, we will describe the correction using the Shifted Band Expanded Angular Spectrum Method (Shift-BEASM) as an example that is not limited to this method. According to the Shifted Band Expanded Angular Spectrum Method, display patterns can be generated or corrected as follows.
[0079] The processing circuit 14 can correct the displayed pattern by rotating it by -γ if there is a deviation in the yaw angle. If there is a deviation in the pitch angle and / or roll angle, the correction method for the processing circuit 14 differs depending on whether it is a transmissive or reflective type.
[0080] When the processing circuit 14 is a transmissive type, it can correct the displayed pattern by rotating the wavefront so that the reproduced image can obtain appropriate angle correction. On the other hand, when the processing circuit 14 is a reflective type, it can correct the displayed pattern by adding a grating.
[0081] In the case of a reflective grating, the phase of the grating can be expressed by the following equation (21), where α is the pitch angle adjustment angle and β is the roll angle adjustment angle.
[0082] In the case of a reflective type, the processing circuit 14 can apply a grating to the CGH by multiplying it by the pattern that displays the intensity information shown in equation (21).
[0083] Wavefront rotation can be expressed by the following formula, where θx, θy, and θz are the rotation angles with respect to the x, y, and z axes, respectively. Let f(x, y) be the wavefront before the transformation, and f^(x^, y^) be the wavefront after the rotation transformation, with (u, v, w) representing the coordinates of (x, y, z) in frequency space. The Fourier spectra of f(x, y) and f^(x^, y^) can be expressed as F(u, v) and F^(u^, v^), respectively.
[0084] Given the above setting, the vector f = (u, v, w) in frequency space satisfies the following for f^(u^, v^, w^):
[0085] Therefore, the relationship before and after wavefront rotation in frequency space is expressed by the following equation, where J represents the Jacobian.
[0086] The wavefront that has undergone a rotational transformation in frequency space is transformed again by the inverse Fourier transform as shown in equation (31).
[0087] The processing circuit 14 can generate, recalculate, or correct the patterns to be displayed on the spatial light modulator 10 according to the respective formulas described above.
[0088] Furthermore, this generation, recalculation, or correction is not limited to the above, and any method capable of correcting appropriate spatial positional and orientational deviations can be used. For example, the processing circuit 14 can process the above calculations using a method described in a different format, based on the representation of positional and / or orientational deviations stored in the memory circuit 16 described above.
[0089] The processing circuit 14 performs at least one of the above necessary calculations based on the input reference plane data to acquire the input wavefront and / or display pattern.
[0090] The processing circuit 14 can generate amplitude information using methods such as the Burch encoding method, Single sideband encoding, and Error diffusion method, as well as phase information using methods such as the Double phase method, Complex amplitude modulation method, and Error diffusion method, as examples of non-limited encoding of display patterns.
[0091] As described above, the processing circuit 14 can, for example, acquire input wavefront information as complex amplitude information, or as amplitude only or phase only information. The processing circuit 14 can also, for example, use input wavefront information as divided within a plane. Furthermore, information from the memory circuit 16 (such as information regarding deviations) can be stored outside the display device 1, and the processing circuit 14 can acquire this external information at the same time as acquiring the input wavefront.
[0092] The processing circuit 14 is a conversion formula that converts the adjusted wavefront into discretized grayscale data that can be displayed on the spatial light modulator 10, and may encode the complex amplitude into phase (including phase only) and / or amplitude (including amplitude only) information using the conversion formula.
[0093] <3. Input Wavefront>
[0094] Next, we will explain some examples of input wavefronts used to generate display patterns.
[0095] (Second Embodiment)
[0096] Figure 13 shows the relationship between a spatial light modulator 10 and a reference wavefront according to one embodiment. This figure is, for example, a view of the front of the spatial light modulator 10 from the positive z-axis direction shown in Figure 12. In the figure, the dashed lines indicate the arrangement of the spatial light modulator 10 in relation to its reference position and reference orientation. The spatial light modulator 10 is positioned with a deviation from this reference position and reference orientation.
[0097] In such cases, even if information about the range related to the reference position and reference orientation is obtained as the reference wavefront, it is possible that information about the reference wavefront and / or input wavefront does not exist in the range indicated by the shaded area, for example. For example, in Figure 12, the input wavefront does not reach a part of the area below the spatial light modulator 10. Under such circumstances, the accuracy of generating an appropriate display pattern across the entire surface of the spatial light modulator 10 may decrease.
[0098] Therefore, the processing circuit 14 can widen the range of the reference wavefront and / or input wavefront to avoid such a decrease in accuracy and perform the analysis. In other words, the processing circuit 14 may acquire information on the reference wavefront and / or input wavefront that is large enough to cover the entire effective display area of the spatial light modulator 10 and perform various calculations.
[0099] Figure 14 shows the relationship between a spatial light modulator 10 and the calculated input wavefront according to one embodiment. Within the range indicated by the dashed line, the processing circuit 14 acquires information about the input wavefront. The processing circuit 14 may, for example, acquire information about a range that is sufficiently wider than the range that takes into account the shift of the spatial light modulator 10, as information about a virtual input wavefront. By acquiring information about the input wavefront within such a range, the processing circuit 14 can appropriately calculate what kind of pattern to display on the entire surface of the spatial light modulator 10.
[0100] The processing circuit 14 may, for example, determine the range in which to acquire the input wavefront based on information about position and orientation deviations. Alternatively, the processing circuit 14 may pre-determine the range in which to acquire the input wavefront based on information such as wavelength, pixel pitch, and number of pixels stored in the memory circuit 16. Furthermore, the processing circuit 14 may pre-determine the range in which to acquire the input wavefront using information on wavelength, pixel pitch, number of pixels, position of the spatial light modulator 10, and orientation of the spatial light modulator 10.
[0101] The processing circuit 14 can, for example, acquire information about the input wavefront based on the light output from the light source corresponding to the spatial light modulator 10.
[0102] (Third embodiment)
[0103] In contrast to the above, as another example, when a reconstructed image is formed by arranging panels of multiple spatial light modulators 10 in a tile-like manner, the processing circuit 14 can also acquire input wavefronts using information from at least a portion of the wavefronts input to adjacent spatial light modulators 10. Figure 14 shows an example of the range in which wavefront information is acquired in, for example, the display device 1 shown in Figure 2 or the display system 2 shown in Figure 3.
[0104] As shown in Figure 15, when the panels of the spatial light modulator 10 are arranged in a tiled pattern, information on overlapping wavefronts can be obtained by performing calculations such as propagation and shift diffraction from the reference wavefront information for each panel. The shaded areas are regions where information on adjacent wavefronts is obtained in overlapping manner. In these regions, the processing circuit 14 can estimate the input wavefront by superimposing the wavefronts, for example, using Huygens' principle.
[0105] Information about the input wavefront can be obtained by extending the input wavefront to adjacent panels in this manner. This extension allows for the generation of display patterns that form a more accurate reconstructed image in spatial light modulators 10 arranged in a tile-like pattern.
[0106] (Fourth Embodiment)
[0107] When the spatial light modulators 10 are arranged in a tiled pattern, the case where a light source is prepared for each spatial light modulator 10 was explained using Figure 15. However, it is possible that multiple spatial light modulators 10 are illuminated by a single light source, or more precisely, by light having a single wavefront for multiple spatial light modulators 10.
[0108] Figure 16 shows the case where a continuous wavefront is irradiated from a single light source (or a single wavefront) to multiple spatial light modulators 10. In this case, the processing circuit 14 can obtain information about the input wavefront covering the range of each spatial light modulator 10 by acquiring information about the input wavefront from the reference wavefront of the light source.
[0109] (Fifth embodiment)
[0110] In the third and fourth embodiments, the input wavefront was acquired over a wider area than the effective pixel area of the spatial light modulator 10, but this is not limited to that. For example, if there is a light source for each of the spatial light modulators 10 shown in Figure 15, or if there are light sources for multiple spatial light modulators 10 shown in Figure 16, calculating the wavefront from each light source within the area where the spatial light modulators 10 may exist may incur excessive computational and time costs.
[0111] Figure 17 shows an example of a spatial light modulator and input wavefront according to one embodiment. As shown in Figure 17, the processing circuit 14 may, for example, propagate light from each light source in the situations shown in Figure 14, Figure 15, or Figure 16, and reconstruct the input wavefront over a sufficient area to appropriately generate a display pattern in the effective pixels of the spatial light modulator 10.
[0112] By limiting the area in this way, it becomes possible to reduce the cost of the display device 1 while forming a highly accurate reproduced image. By reducing the time cost, the display device 1 can achieve faster processing.
[0113] In Figure 17, the area is shown to be wider than the area of the spatial light modulator 10. However, the input wavefront information may be acquired in an area having the same area as the spatial light modulator 10, or in the same area as the effective pixels among the pixels that display the spatial light modulator 10 (i.e., an area where a pattern can be displayed).
[0114] (Sixth Embodiment)
[0115] Figure 18 shows yet another example of the region for acquiring the input wavefront. In Figure 14, the range is expanded, and in Figure 15, the input wavefront is acquired in overlapping regions, but the method is not limited to these. The processing circuit 14 can also acquire appropriate input wavefront information across the entire spatial light modulator 10 by interpolating the region of the input wavefront in which the spatial light modulator 10 may exist with input wavefronts from adjacent light sources, based on information such as the reference wavefront and the wavelength of the input light source in adjacent regions.
[0116] In other words, the processing circuit 14 can obtain information about the input wavefront by simulating propagation from the light source corresponding to the spatial light modulator 10 within the range of the input wavefront that can be obtained from the said light source, and in other regions, it can obtain information about the input wavefront in regions where information about the input wavefront has not been obtained, for example, based on information from an adjacent light source.
[0117] In this case as well, similar to the fifth embodiment described above, the input wavefront information can be acquired in the region where the spatial light modulator 10 is located. The processing circuit 14 can also determine the region from which to acquire the input wavefront by pre-acquiring the region from which to acquire the input wavefront from adjacent light sources, based on the wavelength of the light source, the pixel pitch, the number of pixels, and the position and attitude information of the spatial light modulator 10.
[0118] <4. Other examples of device configurations>
[0119] (Seventh Embodiment)
[0120] In the embodiments described above, examples were given in which a display pattern is generated based on information stored in the memory circuit 16 or stored externally, but the embodiments in this disclosure are not limited thereto.
[0121] As shown in Figure 19, the display device 1 may further be equipped with an interface (I / F 18). The display device 1 can receive various data from the user via this I / F 18. The processing circuit 14 can use this received data to perform the various processes described above.
[0122] The display device 1 can acquire information such as mechanical stress and thermal adaptability that may deviate from design values in the operating environment via the I / F 18. Based on the strain information acquired via the I / F 18, the display device 1 can appropriately calculate the deviation related to the position and orientation of the spatial light modulator 10 and reflect it in the above calculation. In other words, the display device 1 can acquire at least one of the position or orientation information of the spatial light modulator 10 from the user via the I / F 18 and reflect it in the generation of the display pattern.
[0123] Furthermore, this design makes it possible to reflect in the calculations values that have been appropriately corrected for any deviations in the spatial light modulator 10 that may occur due to changes over time.
[0124] The user input interface may be located outside of the display device 1. In this case, various corrections can be performed at the timing of the wavefront input.
[0125] Furthermore, correction may be performed at the timing when wavefront information is input to the display device 1. In this case, the user input to the display device 1 may be controlled to include corrections based on changes from the position and attitude information of the spatial light modulator 10 used for input wavefront generation.
[0126] (Eighth embodiment)
[0127] If the I / F 18 shown in Figure 19 is present, the display device 1 can also make further corrections based on user feedback. Figure 20 is a flowchart of the processing of the processing circuit 14 according to one embodiment.
[0128] As shown in this figure, the processing circuit 14 can acquire feedback information from the user via the I / F 18. Based on the feedback information from the user, the processing circuit 14 determines whether the display confirmation is appropriate (S108).
[0129] When the user inputs an NG (S108: NG), the processing circuit 14 repeats the processing from, for example, S104. Alternatively, the processing circuit 14 may repeat the processing from S106 or S102 instead of S104. If the processing from S102 is repeated, the processing circuit 14 may further prompt the user via the I / F 18 for input on how the discrepancy is occurring, acquire the data received from the user, and reflect it in the generation of the display pattern.
[0130] Display verification can be achieved by displaying the generated pattern on the spatial light modulator 10, and then the user confirms the image reconstructed by irradiating the displayed pattern with the input wavefront from the light source.
[0131] In this case, the reproduced image may be, for example, a grid of lines, a grid of circles, or a checkerboard pattern. The user can send feedback to the display device 1 via I / F 18 by inputting sensing information such as whether the angle is appropriate or whether blurring has occurred regarding the reproduced image of the generated display pattern.
[0132] The processing circuit 14 may generate the adjustment pattern using either a hologram pattern or a brightness distribution pattern.
[0133] The processing circuit 14 may select the spatial light modulator 10 plane as the plane for generating the luminance distribution, or it may select any other suitable plane.
[0134] Furthermore, while the feedback is provided by the user, it is not limited to this. For example, an external information processing device of the display device 1 may acquire a reconstructed image using an image sensor such as a camera, and based on this reconstructed image, transmit data to the display device 1 via the I / F 18 to obtain an ideal reconstructed image. Similarly, an image sensor may be provided outside the display device 1, and data from this image sensor may be input to the display device 1 via the I / F 18, and the processing circuit 14 inside the display device 1 may automatically correct the image to obtain an appropriate reconstructed image based on the data from the image sensor. This correction may be performed repeatedly.
[0135] Furthermore, when displaying patterns such as the grid lines described above on the surface of the spatial light modulator 10, it is also possible to acquire positional and / or orientation deviation information of the spatial light modulator 10 using various camera calibration methods.
[0136] (Ninth Embodiment)
[0137] In all of the above, the display device 1 is provided with a processing circuit 14, but it is not limited to this configuration. The display device 1 may also be equipped with an external information processing device, and this information processing device may transmit signals to the display device 1 to drive the spatial light modulator 10. In other words, an information processing device that generates a similar display pattern outside of the display device 1, rather than inside the display device 1, is also naturally included as an example of this disclosure.
[0138] Similarly, in the case where the display system 2 comprises multiple display devices 1, the display pattern may be generated by an information processing device provided outside of the display device 1.
[0139] The embodiments described above may also take the following forms.
[0140] (1) A display device comprising a memory circuit, a processing circuit, and a spatial light modulator, wherein the memory circuit stores error data indicating the deviation of the spatial light modulator from a reference, the processing circuit acquires information of the input wavefront input to the spatial light modulator based on the error data stored in the memory circuit, and generates a pattern to be displayed on the spatial light modulator based on the information of the input wavefront.
[0141] (2) The display device according to (1), wherein the error data includes at least one of the positional error of the spatial light modulator from a reference position, or the angular error of the spatial light modulator from a reference orientation.
[0142] (3) The display device according to (1) or (2), wherein the processing circuit acquires reference data relating to the reference wavefront input to the spatial light modulator when there is no deviation, and acquires information of the input wavefront input to the spatial light modulator that takes the deviation into account, based on the reference data and the error data.
[0143] (4) The display device according to (3), wherein the information of the input wavefront is expressed in terms of complex amplitude.
[0144] (5) The display device according to (3), wherein the information of the input wavefront includes information of at least one of amplitude or phase.
[0145] (6) The display device according to any one of (3) to (5), wherein the processing circuit acquires information of the input wavefront having a size that can cover the entire effective display area of the spatial light modulator.
[0146] (7) The display device according to (6), wherein the processing circuit acquires information of the input wavefront having a size equivalent to the effective display area of the spatial light modulator.
[0147] (8) The display device according to any one of (3) to (7), wherein the memory circuit acquires the error data together with the reference wavefront information.
[0148] (9) An input interface, further comprising, via the input interface, information on the position or orientation of the spatial light modulator is obtained from a user, and the processing circuit obtains information on the input wavefront based on at least one of the information on the position or orientation of the spatial light modulator obtained via the input interface, and generates the pattern, according to any one of (1) to (8).
[0149] (10) The display device according to (9), wherein the processing circuit generates the pattern for a predetermined display pattern, applies the pattern to the spatial light modulator, and obtains input from a user observing the displayed image via the input interface.
[0150] (11) A display system comprising a plurality of display devices described in any of (1) to (10), wherein each of the plurality of display devices acquires information of the input wavefront input to it, and generates the pattern to be displayed on each of the spatial light modulators based on the acquired information of the input wavefront.
[0151] (12) The display system according to (11), wherein the plurality of display devices are arranged in a tile-like manner.
[0152] (13) A display device comprising a memory circuit, a processing circuit, and a plurality of spatial light modulation panels, wherein the memory circuit stores error data indicating the deviation of each of the spatial light modulation panels from a reference, the processing circuit acquires information of the input wavefront input to each of the spatial light modulation panels based on the error data stored in the memory circuit, and generates a pattern to be displayed on each of the spatial light modulation panels based on the information of the input wavefront.
[0153] (14) The display device according to (13), wherein the spatial light modulation panels are arranged in a tile-like manner.
[0154] (15) An information processing device comprising a memory circuit and a processing circuit for generating a pattern to be displayed on a spatial light modulator, wherein the memory circuit stores error data indicating the deviation of the spatial light modulator from a reference, and the processing circuit acquires information of an input wavefront input to the spatial light modulator based on the error data stored in the memory circuit, and generates a pattern to be displayed on the spatial light modulator based on the information of the input wavefront.
[0155] The aspects of this disclosure are not limited to the embodiments described above, but include various conceivable variations, and the effects of this disclosure are not limited to those described above. The components in each embodiment may be appropriately combined and applied. That is, various additions, modifications, and partial deletions are possible, as long as they do not deviate from the conceptual idea and spirit of this disclosure derived from the claims and their equivalents.
[0156] 1: Display device, 10: Spatial light modulator, 12: Drive circuit, 14: Processing circuit, 16: Memory circuit, 18: I / F, 2: Display system
Claims
1. A display device comprising a memory circuit, a processing circuit, and a spatial light modulator, wherein the memory circuit stores error data indicating the deviation of the spatial light modulator from a reference, the processing circuit acquires information of the input wavefront input to the spatial light modulator based on the error data stored in the memory circuit, and generates a pattern to be displayed on the spatial light modulator based on the information of the input wavefront.
2. The display device according to claim 1, wherein the error data includes at least one of the positional error of the spatial light modulator from a reference position, or the angular error of the spatial light modulator from a reference orientation.
3. The display device according to claim 1, wherein the processing circuit acquires reference data relating to the reference wavefront input to the spatial light modulator when there is no deviation, and acquires information of the input wavefront input to the spatial light modulator that takes the deviation into account, based on the reference data and the error data.
4. The display device according to claim 3, wherein the information of the input wavefront is represented by a complex amplitude.
5. The display device according to claim 3, wherein the information of the input wavefront includes information of at least one of amplitude or phase.
6. The display device according to claim 3, wherein the processing circuit acquires information on the input wavefront having a size that can cover the entire effective display area of the spatial light modulator.
7. The display device according to claim 6, wherein the processing circuit acquires information of the input wavefront having a size equivalent to the effective display area of the spatial light modulator.
8. The display device according to claim 3, wherein the memory circuit acquires the error data together with the reference wavefront information.
9. The display device according to claim 1, further comprising an input interface, wherein at least one of the position or orientation information of the spatial light modulator is obtained from a user via the input interface, and the processing circuit obtains information of the input wavefront based on at least one of the position or orientation information of the spatial light modulator obtained via the input interface to generate the pattern.
10. The display device according to claim 9, wherein the processing circuit generates the pattern for a predetermined display pattern, applies the pattern to the spatial light modulator, and obtains input from a user observing the displayed image via the input interface.
11. A display system comprising a plurality of display devices according to claim 1, wherein each of the plurality of display devices acquires information on the input wavefront input to it, and generates the pattern to be displayed on each of the spatial light modulators based on the acquired information on the input wavefront.
12. The display system according to claim 11, wherein the plurality of display devices are arranged in a tile-like manner.
13. A display device comprising a memory circuit, a processing circuit, and a plurality of spatial light modulation panels, wherein the memory circuit stores error data indicating the deviation of each of the spatial light modulation panels from a reference, the processing circuit acquires information of the input wavefront input to each of the spatial light modulation panels based on the error data stored in the memory circuit, and generates a pattern to be displayed on each of the spatial light modulation panels based on the information of the input wavefront.
14. The display device according to claim 13, wherein the spatial light modulation panels are arranged in a tile-like manner.
15. An information processing device comprising a memory circuit and a processing circuit for generating a pattern to be displayed on a spatial light modulator, wherein the memory circuit stores error data indicating the deviation of the spatial light modulator from a reference, and the processing circuit acquires information of an input wavefront input to the spatial light modulator based on the error data stored in the memory circuit, and generates a pattern to be displayed on the spatial light modulator based on the information of the input wavefront.