Method for reducing holographic speckles and display device

Abstract

A method and display device for reducing holographic speckle when displaying images is described. A target image 10 is decomposed into input images 11. The first input image contains higher spatial frequency components of the target image and is captured using a first display method 12 to generate a first holographic display image. The second input image contains lower spatial frequency components of the target image and is captured using a second display method 12 to generate a second display image. The first and second display images are combined for display to a user. The second display method 12 is adapted to reduce or eliminate holographic speckle compared to the first holographic display method 12, thereby reducing holographic speckle in the combined display image 13.

Description

Technical Field

[0001] The present invention relates to a method for reducing holographic speckles and a display device.

Background Art

[0002] A computer-generated hologram (CGH) is a representation of a light field generated by a coherent or at least partially coherent laser that reflects objects in a scene, and stores depth information and focusing information of the scene. Although often used for three-dimensional scenes, CGH may also be used to display two-dimensional images. Each pixel in the target scene represents a point emitter of a given intensity at a given depth. Each pixel also has a target phase value that encodes the scattering characteristics of the surface.

[0003] In a display device, a hologram may be generated by reflecting laser light off a spatial light modulator, which modulates the incident light according to a computer-generated image to create an impression of the scene for the user. Digital holography is an excitation area for hologram display, and there is a problem in managing speckle noise that appears in a hologram, particularly in a computer-generated hologram or digital holography.

[0004] Some techniques for reducing noise in holograms are known. They include the use of iterative algorithms such as the Gerchberg-Saxton algorithm for reducing speckle noise. However, this technique is computationally expensive and difficult to implement for holograms of three-dimensional scenes. Another option is to introduce hardware into a holographic display device to reduce speckle noise. However, this approach has the disadvantage of tending to cost in terms of the resolution of the displayed holographic image.

Summary of the Invention

[0005] According to a first aspect of the present invention, a method for reducing holographic speckle when displaying an image is provided. The method includes displaying an image by combining a first holographic display image and a second display image. The first holographic display image includes higher spatial frequency components of the image and is generated using a first holographic display method. The second display image includes lower spatial frequency components of the image and is generated using a second display method. The second display method is adapted to reduce holographic speckle or not include holographic speckle as compared to the first holographic display method.

[0006] Generally, when the spatial frequency in the first holographic image is higher than the spatial frequency in the second display image, the first holographic display image may include higher frequency components than the second display image. There may be some overlap in the spatial frequencies present in the first holographic image and the second display image. For example, the average spatial frequency in the first holographic image may be higher than the average spatial frequency in the second display image. Other statistical measurements may be used. For example, the peak of the frequency distribution may be at a frequency in the first holographic display image that is higher than the peak in the second display image. Correspondingly, generally, when the spatial frequency in the second display image is lower than the spatial frequency in the first holographic image, the second display image may include lower frequency components than the first holographic display image. The average spatial frequency in the second display image may be lower than the average spatial frequency in the first holographic image. Other statistical measurements may be used. For example, the peak of the frequency distribution may be at a frequency in the second display image that is lower than the peak in the first holographic display image.

[0007] Embodiments of the invention may be display images that include a lower holographic speckle due to a reduction of the holographic speckle in a second display image. Additionally, embodiments may be able to maintain imaging resolution because the first holographic display method includes less speckle reduction or no speckle reduction compared to the first holographic display method.

[0008] In some embodiments, the displayed image is a three-dimensional image. In other embodiments, the displayed image is a two-dimensional image.

[0009] The first holographic display image may be generated based on a first input image, the second display image may be generated based on a second input image, and all intensity values of the first input image and the second input image are non-negative (a non-negative number is positive or zero). In this way, the first holographic display image may be additionally combined with the second display image. This may be useful when the images are combined incoherently.

[0010] The method may include decomposing a target image to generate a first input image and a second input image, and the first input image and the second input image are generated such that the displayed image perceived by the user is substantially the same as the target image.

[0011] The method may include decomposing a target image to generate a second input image that includes lower spatial frequencies of the target image, and generating a first input image based on the second input image and a target holographic image. The decomposition of the target image to generate the second input image may include applying a minimum filter to the target image. The first input image may be obtained by applying a blur function to the second input image and subtracting the blurred second input image from the target image. The blur function may be a blur function selected to be similar or identical to the blur associated with the second display method.

[0012] Other embodiments may include the step of decomposing a target image to generate a first input image, a second input image, and an intermediate input image, where the first input image includes a higher spatial frequency of the target image than the intermediate input image, and the intermediate input image includes a higher spatial frequency of the target image than the second input image, and displaying the image is performed by combining a first holographic display image, a second display image, and a third display image, and the third display image is generated using a third display method for displaying the intermediate input image. In some embodiments, the target image may be decomposed to generate a second input image, the first blurring function may be applied to the second input image, and the blurred second input image may be subtracted from the target image to generate a second target image. The second target image may be decomposed to generate an intermediate input image, the second blurring function may be applied to the intermediate input image, and the blurred intermediate image may be subtracted from the second target image to generate the first input image. The first blurring function may be selected to be similar or identical to the blurring associated with the second display method. The second blurring function may be selected to be similar or identical to the blurring associated with the third display method. In other embodiments, the target image may be decomposed to generate a second target image that includes a lower spatial frequency of the target image, the first blurring function may be applied to the second target image, and the blurred target image may be subtracted from the target image to generate the first input image. The second target image may be decomposed to generate a second input image, the second blurring function may be applied to the second input image, and the blurred second input image may be subtracted from the second target image to generate the intermediate input image.

[0013] The first display image and the second display image may be combined by displaying the first display image and the second display image in chronological order so that the viewer perceives the target image as a combination of the first display image and the second display image. Displaying in chronological order may be performed at a frequency that is preferably 30 Hz or higher, more preferably 60 Hz or higher. A higher frequency may be preferred so that the target image is perceived at a higher frame rate. For example, when there are two images to be combined into the target image, displaying in chronological order may operate at 120 Hz so that the target image is perceived at 60 Hz.

[0014] The first display image and the second display image may be combined by simultaneously displaying the first display image and the second display image so that the viewer simultaneously receives light from the first display image and the second display image.

[0015] The second display method may include a second holographic display method in which the second display image is generated with only a single depth. Such an embodiment may have a lower computational load due to the absence of varying depth values.

[0016] According to a second aspect of the present invention, a display device is provided that reduces holographic speckle when displaying an image. The display device is configured to display an image by combining a first holographic display image including higher spatial frequency components of the image and a second display image including lower spatial frequency components of the image. The display device has a first operation mode for generating the first holographic display image and a second operation mode for generating the second display image, and the second operation mode is configured to reduce holographic speckle or not include holographic speckle as compared with the first operation mode. The image may be a three-dimensional image or a two-dimensional image.

[0017] The second operation mode may be configured to generate an image with a greater blur than the first operation mode.

[0018] The display device may be configured to display the first holographic display image and the second display image in chronological order or simultaneously such that they are perceived as a combination of the first holographic display image and the second display image.

[0019] In some embodiments, the display device includes a holographic image generator, the holographic image generator includes an optical blur component, and the second operation mode uses the holographic image generator and the optical blur component to reduce the holographic speckles in the second display image.

[0020] In some embodiments, the display device includes a holographic image generator, the holographic image generator includes a processor and a memory configured to process an input image to reduce holographic speckles, and the second operation mode uses the holographic image generator.

[0021] The display device may include a non-holographic two-dimensional display. In such embodiments, the second operation mode may use the non-holographic two-dimensional display. In such embodiments, the fact that a lower frequency is included in the second display image may mean that the defocusing effect caused by using the non-holographic display may be small.

[0022] The display device may include a phase-limited holographic display. In such embodiments, the first operation mode may use the phase-limited holographic display.

[0023] The display device may include a holographic display and a variable diffuser provided in the optical path of the holographic display. In such an embodiment, the first operation mode and the second operation mode may use the holographic display and may have different blurring characteristics over time by varying the level of diffusion provided by the variable diffuser. The display of the first holographic display image and the second display image may be synchronized with the chronological order of the varying diffusion provided by the variable diffuser.

[0024] The display device may include a controller configured to separate the image data and associated diffusion data indicating the level of diffusion required in the image, and to control the display of the image by diffusion according to the diffusion data.

[0025] Further features and advantages of the present invention will become apparent from the following description of the preferred embodiments of the present invention, given by way of example only, created with reference to the accompanying drawings.

Brief Description of the Drawings

[0026]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Best Mode for Carrying Out the Invention

[0027] FIG. 1 gives an overview of a method of processing a target image 10 to generate a combined display image 13. The processing is performed by a digital holographic display device that generates a combined display image 13 that will be visually recognized by a user. The target image 10 is a three-dimensional image data set including image pixels, and each of the image pixels has an intensity value and a depth value. In monochromatic display, the target image 10 is the image to be displayed. In color holographic display, the target image 10 represents one color of the image to be displayed, such as one of a Red image, a Green image, or a Blue image. In this embodiment, the target image 10 includes a single pixel value and a single depth value at each spatial position within the image. More generally, the three-dimensional image data set includes a plurality of image pixels at a single spatial position, and each of the image pixels has a different depth value. It should be noted that the target image 10 is an intensity map and gamma correction is not applied to the target image 10.

[0028] The target image 10 is decomposed into a plurality of input images 11 by a holographic display device, and each of the plurality of input images 11 is an intensity map. The step of the decomposition process is executed by a processor and a memory in the holographic display device according to program instructions stored in the memory. The target image is decomposed into three input images I1 to I3 as shown in FIG. 1, but generally, two or more input images 11 may be formed. The input images 11 then undergo imaging by the imaging process 12 to generate a plurality of display images. The imaging process 12 is shown separately in FIG. 1 and may be executed by separate display systems. However, two or more of the imaging processes 12 may be different operating modes of a single display system, and this example is given below. The light from the display images is combined to generate a combined display image 13. The combination of the plurality of display images may be executed by optically combining the light from the plurality of display images or by sequentially displaying the plurality of images at a speed high enough for the viewer to perceive the images as combined. For example, the display images may be sequentially displayed at a frequency of 60 Hz or higher, which may be sufficient for the images to be perceived by the viewer as a single image.

[0029] Since the input images 11 are imaged by the imaging process 12 and incoherently combined to generate the combined display image 13, the decomposition technique may be selected such that each of the generated input images 11 has a non-negative intensity value. This is because when combining the display images, it is possible to add the images together, but it is difficult or impossible to remove light unless a coherent technique is used for the combination.

[0030] FIG. 2 is a chart of an exemplary method of decomposing a target image 10 that is to be presented to a user to generate two input images 22 and 24. The target image 10 first undergoes low-pass filtering. In an embodiment, this involves the use of an N×N minimum filter. For example, a 9×9 minimum filter may be used where an array of 9×9 pixels around the target pixel is inspected and the minimum pixel value within that array is selected as the value for the target pixel. Applying this filter across the entire target image 10 generates a low-pass image 22.

[0031] The low-pass image 22 undergoes convolution with a blur kernel. The blur kernel is preferably selected to approximate the blur characteristics for which the unblurred low-pass image 22 is targeted as the input image to be captured by the imaging process 12. Ideally, the blur kernel should sum to 1, never be negative, and be normalized to have a shape that can be easily processed in hardware. The blurred low-pass image 23 generated by applying the blur kernel approximates the first display image generated by the imaging process 12 that displays the unblurred low-pass image 22. A suitable blur kernel may be a flat-top function or a truncated Gaussian function. The flat-top function is expected to provide better speckle reduction but may be more difficult to reproduce in hardware.

[0032] The high-pass image is generated by subtracting the blurred low-pass image 23 from the target image 10. The high-pass image 24 is used as the first input image and is captured by the first imaging process 12, and the unblurred low-pass image 22 is used as the second input image and is captured by the second imaging process 12. In this way, the decomposition produces an input image 11 having minimum content on either side of the cut-off frequency of the blur kernel, such that no content is lost when the decomposed images are displayed by the imaging process 12.

[0033] As described above, the target image 10 includes depth values associated with pixels. The input image 11 generated during decomposition inherits the depth values from the target image 10 without the need to consider depth values during decomposition.

[0034] The method described above in connection with FIG. 2 includes a single decomposition to generate an unblurred low-pass image 22 and a high-pass image 24 as the input image 11. In a further embodiment, the method of generating multiple input images 11 by using different N×N minimum filters may be repeated. For example, the method shown in FIG. 2 may first be executed using the target image and a 27×27 minimum filter. As described above, this generates a first unblurred low-pass image and a first high-pass image. The unblurred low-pass image forms a third input image in this embodiment. The first high-pass image may then be considered as the target image for a second decomposition using a 9×9 minimum filter. The second decomposition uses the same method as the first decomposition, as illustrated in FIG. 2. The second unblurred low-pass image is generated by a 9×9 filter, and the 9×9 filter forms a second input image. The second unblurred low-pass image then receives a blur kernel to generate a second blurred low-pass image, and the second blurred low-pass image is subtracted from the first high-pass image (the second target image). The resulting second high-pass image forms a first input image. The blur kernel used to blur the third input image is preferably selected to correspond to the blur introduced by the imaging process 12 used to display the third input image. Correspondingly, the blur kernel used to blur the second input image is preferably selected to correspond to the blur introduced by the imaging process 12 used to display the second input image.

[0035] The method described in the foregoing paragraphs generates three input images 11, where the first input image includes a higher frequency of the target image 10 than the second input image, and the second input image includes a higher input frequency than the third input image. The reader will recognize that the process may be repeated any number of times by using a series of minimum value filters of different sizes to serially decompose the generated high-pass images to produce the desired number of input images 11. In this way, successive images corresponding to different frequency ranges within the target image are produced. First, an image with the lowest frequency is produced, and each iteration produces an image with successively higher frequencies.

[0036] Returning to FIG. 1, the first input image in each of the embodiments described above is a high-pass image and includes a higher frequency of the target image 10 than the unblurred low-pass image. In the embodiment illustrated by two decompositions, the first input image (the high-pass image from the second decomposition) has a higher frequency than the second input image (the unblurred low-pass image from the second decomposition). The second input image (the unblurred low-pass image from the second decomposition) has a higher frequency than the third input image (the unblurred low-pass image from the first decomposition).

[0037] The imaging process(es) 12 used to generate the second and subsequent display images from the second and subsequent input images 11 may be selected to have a greater speckle reduction ability compared to the imaging process 12 used to generate the first display image from the first input image. For example, the imaging process 12 used to display the first input image may be a phase-only holographic display that does not have a speckle removal hardware element such as a vibrating mirror. Alternatively, the imaging process 12 used to display the first input image may include a speckle removal hardware element such as a vibrating mirror, but the speckle removal hardware element may be deactivated. The second or subsequent imaging process 12 used to display the second or subsequent input image may be a phase-only holographic display that includes an active speckle removal hardware element such as a vibrating mirror. This imaging process 12 tends to generate a display image with better speckle noise characteristics at the expense of some loss of resolution. The second or subsequent imaging process 12 may further include a software processing step that applies the Gerchberg-Saxton algorithm to the input image to calculate the phase-only hologram. The use of the Gerchberg-Saxton algorithm with a phase-only hologram is known and discussed, for example, in ‘A Practical Algorithm for the Determination of Phase from Image and Diffraction Plane Pictures’, R.W. Gerchberg and W.O. Saxton published in OPTIK Vol.35(No.2) 237-246(1972) and ‘Speckle-suppressed phase-only holographic three-dimensional display based on double-constraint Gerchberg-Saxton algorithm’, by Chenliang Chang and Jun Xia, in Applied Optics, 54(23):6994-7001, which was published in September 2015.

[0038] In some embodiments, the imaging process 12 for the second or subsequent input image is selected to be a two-dimensional imaging process. In one example, the imaging process 12 may be a phase-limited holographic imaging process, in which the depth data in the second input image is replaced with a single depth value in order to reduce the computational processing load. In other embodiments, the two-dimensional imaging process 12 may be a non-holographic display such as a flat panel display. Due to the fact that the out-of-focus effect may be low for the lower frequency components of the target image 10, the use of a non-holographic display method or the removal of depth variations in a holographic display method may be acceptable for the display of the second or subsequent display image. In other words, it is the higher frequency components of the target image that carry most of the depth information, and thus the lost depth information in the lower frequency components may not be perceived by the viewer or may reduce the impact on depth perception when combined with the target image.

[0039] Image decomposition techniques that use an N×N minimum filter to generate low-pass and high-pass input images 11 have been described. However, it should be recognized that this is not the only possible image decomposition technique. For example, image decomposition techniques based on edge detection for identifying the high-frequency portion of an image or using frequency separation techniques are also possible.

[0040] The imaging process 12 described above is selected to receive a second and subsequent input image 11 including lower frequencies of the target image 10 for a level of speckle noise reduction. The reason for this is that speckle reduction techniques, such as the use of a speckle removal hardware element, tend to reduce the cost of image resolution while reducing speckle noise. Therefore, by using a phase-limited holographic method without a speckle removal hardware to display higher frequencies in the first image, the loss of resolution can be reduced, and the advantage of speckle noise reduction in the 3D image can also be obtained.

[0041] To illustrate the advantages of the embodiment, FIG. 3 consists of three images of the area around a horse's eye. The upper image 31 is a simulation of a phase-limited hologram including holographic speckles (represented two-dimensionally in FIG. 3). The middle image 32 is a simulation of a hologram generated after performing a single image decomposition of the type described in connection with FIG. 2 using a 9×9 minimum filter to generate a non-blurred low-pass image, and then blurring the high-pass image using a blur kernel. The middle image 32 shows significantly less speckle noise than the upper image 31.

[0042] The lower image 33 in FIG. 3 is a simulation of a holographic image after three image decompositions using the 27×27, 9×9, and 3×3 minimum filters in sequence as described above, and thus is a combination of four images. There is a further significant improvement in image quality, which can be most clearly seen in the area of the horse's eye in FIG. 3.

[0043] An example of the hardware that can be used to implement the imaging process 12 is described here. As previously mentioned, the imaging process 12 may be implemented as separate modes using different imaging systems or different operating modes of a single hardware configuration.

[0044] FIG. 4 is a diagram of a phase-limited holographic imaging apparatus. Imaging process 12 performed by the phase-limited holographic imaging apparatus may be suitable for displaying a first input image that includes higher frequencies of the target image 10 because it does not include any speckle removal hardware. Light source 40 generates a coherent laser beam. The laser beam is focused by a first lens 41. The dotted line 42 shown in FIG. 4 indicates the focal plane of the laser light. The laser beam is collimated by a second lens 43 before passing through the laser aperture 44. Beam splitter 45 directs the beam to spatial light modulator 46, which modulates light according to an input image such as the first input image. Light from the spatial light modulator returns to focusing element 47 through the beam splitter. Beam splitter 45 may be a polarization beam splitter, in which case an additional optical element (not shown) for adjusting the polarization of the light may be inserted in front of spatial light modulator 46. Light from the spatial light modulator is focused by focusing element 47 to form an image at imaging plane 48.

[0045] The laser light generated by laser light source 40 typically has a Gaussian distribution in the intensity profile. The image formed at imaging plane 48 is the Fourier transform of the phase profile in spatial light modulator 46 convolved with the laser intensity at the focal plane of laser light 42. The image is additionally convolved by a sinc function that is the Fourier transform of laser aperture 44. Typically, it is laser aperture 44 that is the limit to the overall resolution. Here, it should be noted that if the beam profile of the laser is not ideal, the blur kernel used in the decomposition algorithm may be adapted to account for the actual laser beam profile.

[0046] As described above, the holographic imaging apparatus of FIG. 4 tends to produce speckle noise in the displayed image. However, the background noise is generally acceptable because the intensity of the higher frequency components in the first input image tends to be relatively low.

[0047] FIG. 5 is a diagram of a phase-limited holographic imaging apparatus including an optical diffuser 54. This phase-limited holographic imaging apparatus is such that the diffuser 54 is selected to reduce speckle noise and may be suitable for displaying a second or subsequent input image in processing because it becomes a limiting factor for resolution.

[0048] The laser light source 50 generates a laser beam incident on the aperture 51. The aperture 51 serves to set the source profile of the laser light and is a hardware element that may be omitted in some embodiments. The laser beam is then focused by a first lens 52 in front of the optical diffuser 54 to a focal point 53, and the optical diffuser 54 takes the form of a rotating diffuser. When a static diffusing surface is used to diffuse the laser light, a rotating diffuser is used because the convolution kernel of the static diffuser has a fixed random phase and the image in the image plane 56 behind the spatial light modulator maintains speckles. However, when the diffusing surface is dynamic, the blur kernel has a dynamically changing phase profile and the speckles generated by the diffusing surface change dynamically. When time-averaged, the speckle noise is reduced and the blur kernel takes the profile 55 from the laser light source plane 53. As mentioned above, the profile of the laser light source may be set by the addition of the aperture 51.

[0049] The diffused light is collimated by a second lens and formed into an image in the image plane 56 as described in connection with FIG. 4. Therefore, the description of the corresponding elements in FIG. 5 is not repeated.

[0050] As described in connection with FIG. 1, the input image 11 is captured by the imaging process 12, and the display images are combined to generate the combined display image 13. This may be performed within a single hardware array in different operating modes by sequentially displaying the display images so as to give the viewer an impression of the combined display image 13. The level of blurring is synchronized with the display of different input images 11. The level of blurring preferably varies between no blurring (point source) and a level of blurring sufficient to generate a blurred low-pass image with minimal speckle.

[0051] An example of how to display a sequence of images with variable levels of optical blurring (different operating modes) is described here with reference to FIG. 6. FIG. 6 represents a part of the phase-limited holographic imaging apparatus according to FIG. 5, but in this case, instead of the rotating diffuser of FIG. 5, a vibrating mirror is used as the optical diffusing element. The corresponding elements in FIG. 6 are given the same reference numbers as in FIG. 5, incremented by 10 only. The rotating diffuser of FIG. 5 is replaced by an electrically controlled diffuser in the form of an ultrasonic MEMS mirror 64. Ultrasonic MEMS mirrors are commercially available, such as those manufactured by Dyoptyka. When active, the ultrasonic mirror 60 provides a time-varying diffusing surface as shown in the upper image of FIG. 6A.

[0052] FIG. 6 shows a technique for adapting a holographic imaging apparatus such that the amount of diffusion varies rapidly as different input images 11 are captured.

[0053] FIG. 6A shows the laser 60 focused by the first lens 62 immediately before the vibrating mirror 64. The vibrating mirror imparts blurring to the resulting image, reducing image speckle at the expense of resolution.

[0054] FIG. 6B shows that the first lens 62 may be adapted to be controllably driven to vary the focusing plane of the laser beam and thus vary the degree of diffusion by the vibrating mirror 64. Driving to control the first lens 62 may be performed by use of a voice coil, piezoelectric material, or a liquid lens, etc. As shown in FIG. 6B, the laser light may be focused on the vibrating mirror 64 to minimize the resulting blur and create a light source close to a point. The level of blur applied is lower than the level of blur in FIG. 6A.

[0055] The laser light may be focused by the driveable lens 62 at one or more positions in front of the vibrating mirror 64 to increase the blur, diffuse the light (as in FIG. 6A, etc.), or reduce the blur (as in FIG. 6B, etc.).

[0056] In an embodiment where the target image is decomposed into two images, the holographic imaging device may operate according to FIG. 6B for the first input image (with reduced blur) and according to FIG. 6A for the second input image (with increased blur). In this way, the level of blur is controlled by the focusing point of the first lens 62. In an embodiment where the target image is decomposed into three or more images, varying the focusing point of the first lens 62 is appropriate for displaying the second and subsequent images in which blur is introduced into the display image. Since the position of the focusing of the laser light can be controlled, it is possible to provide several different levels of diffusion according to the number of input images 11 (and thus image decomposition) as desired. The driveable lens 62 is immediately movable to provide different levels of diffusion in synchronization with the display of different input images by the spatial light modulator 46. By displaying a fast sequence of input images 11 with different amounts of diffusion, the viewer can perceive the combined display image 13.

[0057] FIG. 6C depicts the situation when power to the ultrasonic mirror is turned off so that it interrupts its vibration and a flat reflective surface is then provided. This arrangement enables two diffused states through control of the optical diffusing element. For example, the optical diffusing element may be selectively turned on and off in sequence when the target is decomposed into two images.

[0058] Some embodiments may combine the operations in FIGS. 6A, 6B, and 6C by providing optical diffusing elements that can be selected and driven in both drivable lenses. When the target image is decomposed into three images, the operation according to FIG. 6C can be applied to the first input image (the highest frequency image) to preserve the content of the image, the operation according to FIG. 6B can be applied to the intermediate image, and the operation according to FIG. 6A can be applied to the second image (the lowest frequency image).

[0059] Although the optical diffusing element 64 is described in conjunction with a vibrating mirror, the technique of FIG. 6 can be applied to other optical diffusing elements. For example, they can be applied using a rotating disk optical diffuser having a diffusing surface and a flat region within the diffusing surface. In that case, stopping the rotation of the rotating disk optical diffuser in the orientation in which the laser light is incident on the flat region of the disk optical diffuser has an effect similar to that discussed above with reference to FIG. 6C when the vibrating mirror is turned off.

[0060] FIG. 7 illustrates a further embodiment in which variable diffusion is provided by a step diffuser element 70 through which laser light for generating a holographic image is transmitted. This step diffuser arrangement may be introduced into the apparatus described in connection with FIG. 5 instead of the rotating diffuser 54. The step diffuser element 70 is horizontally translatable as indicated by the double arrow 72 in FIG. 7A. The horizontal translation has two components.

[0061] First, within the stage of the stepped diffuser, the stepped diffuser is oscillated to provide a time-varying diffusion kernel that has the effect of time-averaging the diffusion pattern and reducing speckle noise.

[0062] Second, when different amounts of diffusion are required (or when no diffusion is required), the stepped diffuser may translate horizontally between the stages, as shown between FIGS. 7A and 7B. In FIG. 7B, the stepped diffuser element 70 has translated to the left, such that the light from the laser is incident on stage 76 rather than stage 74. This changes the focusing point of the laser due to the different refractive indices of the stepped diffuser element with respect to air. As shown in FIG. 7B, the laser light focuses closer to exit the surface of the stepped diffuser element, and thus there is less resulting diffusion or blurring. They are graphically represented by the laser light distribution profile 78b in FIG. 7B, which is sharper than the distribution profile 78a in FIG. 7A.

[0063] In other words, while the position of the focusing lens does not change, stepped diffusers at different depths change the focusing point. As shown in FIG. 7A, the laser light is focused further from the exit surface or the diffuser surface of the stepped diffuser 70. FIG. 7B shows a situation where the laser light focuses closer to the exit surface or the diffuser surface of the stepped diffuser 70. This second configuration has an effect similar to that of a point light source configuration.

[0064] In some embodiments, when no diffusion is required, the stepped diffuser element 70 may be completely removed from the optical path, such as by continuing the translation movement.

[0065] A holographic display device has been described above that enables the display of a holographic image by introducing a time-varying amount of diffusion or blur to control speckle noise. However, there may be cases where only a first image containing higher frequency components of the target image 10 requires holographic display. The second and any subsequent input images that encompass the lower frequencies of the target image 10 may be displayed by other means. For example, an input image that encompasses sufficiently low frequencies may be displayed using a non-holographic two-dimensional display such as an LCD display, a DMD display, an OLED display, or a micro LED display. This is possible because any out-of-focus blur resulting from a loss of depth information is not perceivable due to the already blurred characteristics of the image decomposed as a result of the low-pass filter. The size of the blur kernel for which this is possible is approximately 4D 2 / Cλ pixels, where D is the pupil diameter, C is the near focal length, and λ is the wavelength. For typical values, this is a kernel size of about 100 pixels wide. Simultaneously displaying images from a non-holographic display and a holographic display is possible using various techniques.

[0066] One technique is by using a birdbath-style optical combiner known from head-mounted display technology. In this way, light from two images, a first holographic image and a second two-dimensional non-holographic image, can be combined and viewed simultaneously. In other embodiments, both displays may be holographic displays, where the first one includes an active speckle removal function for displaying the second and subsequent input images 11, and the other does not include an active speckle removal function for displaying the first input image. Again, those two displays may be configured with respect to a combined display using an optical combiner.

[0067] In an embodiment where decomposition is performed two or more times on a target image 10 to generate three or more input images 11, the images after the first input image and before the final input image include mid-spatial frequencies. Those frequencies tend not to be affected by the loss of resolution. Thus, it is typically preferred to display those input images 11 using a holographic display device including at least one hardware speckle removal element (rotating diffuser, ultrasonic mirror, or stepped diffuser) of the type described above.

[0068] In other embodiments, input images 11 having mid-frequency content (mid-blurring kernel) may be displayed using light field technology (multiple incoherent images are displayed on multiple sub-pupils). This is possible because the diffraction limit for the reduced pupil is acceptable for the reduced resolution of those decomposed input images 11.

[0069] The above techniques are for general use, and the display device can be implemented in a head-mounted display, a head-up display, a display panel, or other display types.

[0070] A phase-limited holographic imaging device including the apparatus described with reference to FIGS. 4 and 5 has been described above. When writing, many spatial light modulators are phase-limited. However, the techniques discussed herein are applicable in cases where amplitude-limited spatial light modulators or phase and amplitude spatial light modulators are used. The methods and apparatus described above do not change in this case, but the capabilities of the spatial light modulator are changed to include modulation of the light amplitude.

[0071] The above embodiments are to be understood as exemplary examples of the invention. Further embodiments of the invention are contemplated. For example, the embodiment of the display device described in connection with FIGS. 5 - 7 may be combined with a software application of the Gerchberg - Saxton algorithm to the second or subsequent input image 11 so as to reduce the speckle noise in the resulting display image. This may have the effect of further reducing the noise in the lower - frequency input image 11, although at the cost of additional computational overhead.

[0072] In the above embodiment where multiple decompositions are performed, the largest N×N minimum - value filter is first applied, and subsequently, the N×N minimum - value filter is applied in the order in which smaller N×N minimum - value filters are applied. For example, a 27×27 minimum - value filter may be applied, and then the high - pass image may be further decomposed using a 9×9 minimum - value filter. The high - pass image resulting from the 9×9 minimum - value filter decomposition may be further decomposed using a 3×3 minimum - value filter. In other embodiments, instead of further decomposing the high - pass image, the low - pass image may be further decomposed. For example, a 3×3 minimum - value filter may be first applied to generate high - pass and low - pass images. The low - pass image may be further decomposed using a 9×9 minimum - value filter to generate additional high - pass and low - pass images. Finally, the low - pass image may be further decomposed using a 27×27 minimum - value filter to generate another set of high - pass and low - pass images. Of course, 27×27, 9×9, and 3×3 are simply selected as examples, and any suitable set of N×N minimum - value filters may be used.

[0073] As described above, the final input image encompasses the lowest frequency of the target image 10. Since the resolution of this image is typically low, the display of the last input image or the last few images that encompass the lowest frequency may be performed with a reduced spatial resolution in order to reduce the computational load. The reduction in resolution may also be applied to the depth values and / or intensity values of the lower frequency images. Thus, the depth values and / or intensity values may be quantized to have a lower resolution in the mid-frequency and / or lower frequency images (the second or subsequent input images).

[0074] The above embodiments have discussed the display of a single target image. However, it will be recognized that video is a sequence of images and that the above techniques are equally applicable to three-dimensional video displays. Thus, the term "image" in the above description should be understood to include images that form part of a video sequence.

[0075] In the above-described embodiments, the step of decomposing the target image 10 to generate the input image 11 is performed by a processor and memory within the holographic display device. In further embodiments, the processing of the target image may be performed by an application-specific circuit. In other embodiments, the holographic display device may be connected to a separate information processing device, such as a PC, server, or cloud, and the processing of generating the input image 11 from the target image 10 may be performed by the information processing device.

[0076] Referring now to FIG. 8, an example of hardware control for synchronizing the display of the input image 11 with appropriate diffusion characteristics is discussed. FIG. 8 represents a schematic block diagram of an example apparatus for synchronizing the display of an image having selectable diffusion levels. As described above, for example, referring to FIGS. 6 and 7, the input image 11 may be displayed in chronological order by the same hardware operating in different diffusion modes over time. FIG. 8 is a block diagram of a system operable to synchronize the display of an image with the operation of a diffuser.

[0077] The controller 80 receives an input 82 of the image 11 for display according to diffusion data indicating the diffusion level required for application during display. For example, referring to FIG. 1, the controller 80 may receive images I1, I2, and I3 via the input interface 82. The diffusion data may be included as metadata associated with each image. In some embodiments, the diffusion data is a value corresponding to the relative or absolute level of diffusion to be applied. For example, a value of 0 indicates no diffusion (for an image with high spatial frequency components), and a value of 1 may indicate that diffusion is active (for an image with low spatial frequency components). Other embodiments of the diffusion data are possible.

[0078] The controller 80 is configured to separate the diffusion data from the input image and provide the input image to the SLM driver 84 for display on the SLM 86. The SLM driver 84 notifies the controller 80 via a signal 88 when an image is formed on the SLM. Next, the controller 80 provides a signal 90 for irradiating the SLM with the diffusion control element 92 simultaneously with activating the laser 94 or other at least partially coherent light source.

[0079] The diffusion control element 92 is configured to activate and deactivate the diffuser according to the diffusion data in response to the required speckle reduction in the output image. In some embodiments, the diffusion control element 92 is also configured to control the level of diffusion brought about by the diffuser. For example, when used with the embodiment of FIG. 6, the diffusion control element may control the level of diffusion by selectively activating the ultrasonic mirror and / or adjusting the focusing. Further, if the diffuser is binary or otherwise has discrete states (such as the steps in FIG. 7), the diffusion control element may selectively activate and deactivate the diffuser or switch between states during the display of one image according to the desired amount of diffusion, such as by using pulse width modulation. This can make it possible to reduce the hardware complexity and improve the image quality.

[0080] This structure of FIG. 8 enables a single SLM to display images 11 in sequence by controlling the diffusion synchronized with the image sequence. Through the use of the hardware control path, accurate and fast synchronization between the display of the image on the SLM and the appropriate amount of diffusion may be achieved.

[0081] The controller 80 is implemented by, but not limited to, the field programmable gate array (FPGA) in FIG. 8. Other embodiments may use application specific integrated circuits (ASICs) or appropriately programmed processing systems.

[0082] Some embodiments may combine the functional blocks shown in FIG. 8 into a single element. For example, any two or all of the controller 82, the SLM driver 84, and the diffusion control 92 may be combined.

[0083] The High Definition Multimedia interface (HDMI) is used to supply image data to the controller in FIG. 8 through the input interface 82. In other embodiments, other interfaces including DisplayPort, Thunderbolt, and USB may be used.

[0084] In some embodiments, the diffusion data is independent of the display hardware and the diffusion control 92, or the controller 82 converts the diffusion data into appropriate hardware operations. For example, the diffusion data may be set as a predetermined value according to the required level of diffusion, and this can be achieved using hardware via the controller 82 and / or the diffusion control 92.

[0085] In an alternative structure, the controller 80 may be used to direct the image to the appropriate hardware according to the diffusion data. For example, the display device may further include a non-holographic display having different, predetermined levels of diffusion, or an additional holographic display. The controller 80 is provided connected to all those displays, optionally via appropriate driver circuitry, and directs the received image data to the appropriate one according to the diffusion data. For example, if the embodiment of FIG. 8 further includes a non-holographic display, the image may be directed to the non-holographic display or the SLM according to the required level of diffusion (such that images containing low spatial frequencies, which require a higher level of diffusion, are directed to the non-holographic display).

[0086] Although the embodiments described herein have been applied to three-dimensional images, they are equally applicable to two-dimensional images displayed holographically. Such two-dimensional images may be associated with a depth, for example, the depth of the image plane for display.

[0087] The methods described herein may be embodied wholly in software, wholly in hardware, or in any combination of them. Where a software embodiment is used, the example may include a computer-readable medium, which may be a non-transitory computer-readable medium including computer-executable instructions that, when executed by a processor, instruct the processor to perform the method.

[0088] It should be understood that any feature described in connection with any one embodiment may be used alone or in combination with other features described, and may also be used in combination with one or more features of any other embodiment, or any combination of any other embodiments. Further, equivalents and modifications not described above may be employed without departing from the scope of the invention as defined in the appended claims.

Claims

Claim 1 A method for reducing holographic speckle when displaying an image, comprising: displaying the image by combining a first holographic display image and a second display image, wherein the first holographic display image includes higher spatial frequency components of the image and is generated using a first holographic display method, the second display image includes lower spatial frequency components of the image and is generated using a second display method, the second display image has an average spatial frequency lower than the average spatial frequency of the first holographic display image, and the second display method is adapted to reduce holographic speckle or be free of holographic speckle as compared to the first holographic display method; the first holographic display image and the second display image are displayed in time sequence the first holographic display image and the second display image such that a viewer perceives the image as a combination of the first holographic display image and the second display image, or displayed simultaneously and optically combined the first holographic display image and the second display image such that a viewer receives light from the first holographic display image and the second display image simultaneously; A method, which is combined thereby. Claim 2 The method according to claim 1, wherein the first holographic display image is generated based on a first input image, the second display image is generated based on a second input image, and all intensity values of the first input image and the second input image are non-negative. Claim 3 The method according to claim 2, comprising decomposing a target image to generate the first input image and the second input image, wherein the first input image and the second input image are generated such that the displayed image perceived by a user is substantially the same as the target image. Claim 4 The method according to claim 2, comprising generating the second input image including lower spatial frequencies of the target image by decomposing the target image, and generating the first input image based on the second input image and the target image. Claim 5 A step of decomposing a target image to generate the first input image, the second input image, and an intermediate input image, wherein the first input image includes a spatial frequency of the target image higher than that of the intermediate input image, the intermediate input image includes a spatial frequency of the target image higher than that of the second input image, and displaying the image is performed by combining the first holographic display image, the second display image, and a third display image, and the third display image is generated using a third display method for displaying the intermediate input image, the method according to claim 2.

6. The method according to any one of claims 1 to 5, wherein the second display method includes a second holographic display method in which the second display image is a two-dimensional image.

7. The method according to any one of claims 1 to 6, wherein the displayed image is three-dimensional.

8. A display device for reducing a holographic speckle when displaying an image, The display device is configured to display the image by combining a first holographic display image including a higher spatial frequency component of the image and a second display image including a lower spatial frequency component of the image, the second display image having an average spatial frequency lower than an average spatial frequency of the first holographic display image, the display device having a first operation mode for generating the first holographic display image and a second operation mode configured to generate the second display image, and the second operation mode is configured to reduce a holographic speckle or not include a holographic speckle as compared with the first operation mode.

9. The display device according to claim 8, wherein the second operation mode is configured to generate an image with a greater blur than the first operation mode.

10. The display device according to claim 8 or 9, configured to display the first holographic display image and the second display image in time sequence or simultaneously such that they are perceived as a combination of the first holographic display image and the second display image.

11. Comprising a holographic image generator, the holographic image generator including an optical blur component, The display device according to claim 8, 9 or 10, wherein the second operation mode uses the holographic image generator and the optical blurring component to reduce the holographic speckles in the second display image.

12. A holographic image generator, the holographic image generator including a processor and a memory configured to process an input image to reduce holographic speckles, The display device according to claim 8, 9 or 10, wherein the second operation mode uses the holographic image generator.

13. Comprising a non-holographic two-dimensional display, The display device according to claim 8, 9 or 10, wherein the second operation mode uses the non-holographic two-dimensional display.

14. Comprising a phase-limited holographic display, The display device according to any one of claims 8 to 13, wherein the first operation mode uses the phase-limited holographic display.

15. A holographic display, and A variable diffuser provided in the optical path of the holographic display, The display device according to claim 8 or 9, wherein the first operation mode and the second operation mode use the holographic display and have different blurring characteristics in time sequence by varying the level of diffusion provided by the variable diffuser.

16. The display device according to any one of claims 8 to 15, wherein the displayed image is a three-dimensional image.

17. The display device according to any one of claims 8 to 16, comprising a controller configured to separate image data and associated diffusion data indicating the required level in the image, and control the display of the image by diffusion according to the diffusion data.

Images