Hologram optimization method and device therefor

Abstract

A hologram optimization method and a device therefor are disclosed. The hologram optimization device inputs a phase pattern to a spatial light modulator to generate a reproduced image, obtains a region of interest from the reproduced image, obtains a value of a loss function defined by a sum of a first value indicating a difference between the reproduced image and a target image and a second value indicating a ratio between energy of the region of interest and incident energy of the spatial light modulator, and repeats a process of updating the phase pattern such that the value of the loss function decreases.

Description

Hologram optimization method and device thereof

[0001] Embodiments of the present invention relate to a hologram optimization method and device thereof, and more particularly, to a computer-generated hologram optimization method and device thereof.

[0002] In order to reproduce a desired image on a holographic display, a corresponding computer-generated hologram (CGH) must be calculated. Since phase-only spatial light modulators (SLMs) are widely used in typical holographic displays, most hologram optimization algorithms optimize the phase pattern so that the reproduced image is identical to the target image. A representative method is optimization using gradient descent, which calculates the image reproduced by the SLM using the angular spectrum method (ASM), calculates a loss function between the calculated reproduced image and the target image, and then repeats the operation of backpropagating the reproduced image to optimize the phase pattern to minimize the loss function.

[0003] At this time, setting the region of interest in the reproduced image can further improve the image quality. For example, if the resolution of the SLM is 1920*1080, setting the region of interest to a resolution smaller than the SLM's resolution of 1600*900 and calculating the loss function only within the region of interest will improve the quality of the convergence result as the area where optimization is performed becomes smaller. However, holograms optimized in this way can form noise outside the region of interest, and since only a portion of the light incident on the SLM is transmitted inside the region of interest, the energy efficiency can be low. This is a fatal drawback when producing a holographic display in a portable head-mounted form factor. In a head-mounted form factor system, it is difficult to mask the noise outside the region of interest with a separate mask, and when used as a portable device, the low energy efficiency can shorten the usage time.

[0004] The technical problem to be achieved by the embodiment of the present invention is to provide a hologram optimization method and device capable of generating a hologram having high energy efficiency and an optimized phase pattern.

[0005] In order to achieve the above technical task, an example of a hologram optimization method according to an embodiment of the present invention includes the steps of: generating a reproduction image by inputting a phase pattern to a spatial light modulator; obtaining a region of interest from the reproduction image; obtaining a value of a loss function defined as the sum of a first value representing a difference between the reproduction image and a target image and a second value representing a ratio between energy of the region of interest and incident energy of the spatial light modulator; and updating the phase pattern so that the value of the loss function decreases; and repeatedly performing the steps from the step of generating the reproduction image to the step of updating the phase pattern until the value of the loss function becomes less than or equal to a predefined reference value.

[0006] In order to achieve the above technical task, an example of a hologram optimization device according to an embodiment of the present invention includes an image generation unit that inputs a phase pattern to a spatial light modulator to generate a reproducible image and extracts a region of interest from the reproducible image; a loss calculation unit that obtains a value of a loss function defined as the sum of a first value representing a difference between the reproducible image and a target image and a second value representing a ratio between energy of the region of interest and incident energy of the spatial light modulator; an update unit that updates the phase pattern so that the value of the loss function decreases; and a repeating unit that repeatedly performs the process of generating the reproducible image and updating the phase pattern until the value of the loss function becomes lower than a predefined reference value.

[0007] According to an embodiment of the present invention, a hologram having an optimized phase pattern can be generated with high energy efficiency compared to existing hologram generation methods.

[0008] FIG. 1 is a drawing illustrating an example of a hologram optimization method according to an embodiment of the present invention;

[0009] FIG. 2 is a flowchart illustrating an example of a hologram optimization method according to an embodiment of the present invention;

[0010] FIG. 3 is a diagram illustrating a simulation result of hologram optimization according to an embodiment of the present invention;

[0011] FIG. 4 is a graph showing the results of hologram optimization according to an embodiment of the present invention.

[0012] FIG. 5 is a drawing showing an actual reproduction result of an optimized hologram according to an embodiment of the present invention, and

[0013] FIG. 6 is a diagram illustrating an example configuration of a hologram optimization device according to an embodiment of the present invention.

[0014] Hereinafter, a hologram optimization method and device according to an embodiment of the present invention will be described in detail with reference to the attached drawings.

[0015] FIG. 1 is a drawing illustrating an example of a hologram optimization method according to an embodiment of the present invention.

[0016] Referring to Fig. 1, a spatial light modulator (100) forms a complex wave front for a phase pattern. The complex wave front of the spatial light modulator propagates over a certain distance to generate a reproduction image. To generate a reproduction image through simulation, an angular spectrum method (ASM) (110) may be used to propagate the complex wave front.

[0017] A region of interest (180) is extracted from a reproduction image (120), and a loss function is calculated by comparing the image of the region of interest (180) with the target image (140). In one embodiment, the region of interest (180) can be extracted using a binary mask consisting of 0 and 1.

[0018] In this embodiment, the loss function is defined by the error and energy efficiency between the reproduced image (120 or 180) and the target image (140). In one embodiment, the error refers to the difference (e.g., intensity difference, etc.) between the pixels of the two images. The energy efficiency is a value representing the ratio of the energy transmitted into the region of interest (180) of the reproduced image (120) and the energy incident on the spatial light modulator (100). In one embodiment, the energy efficiency (γ) can be expressed as follows.

[0019]

[0020] Here, I recon is the amplitude of the reproduced image, M ROI is a binary mask, φ slm refers to the phase pattern input to the spatial light modulator. Referring to mathematical expression 1, the closer the energy efficiency is to 0, the lower the efficiency is, and the closer it is to 1, the higher the efficiency is.

[0021] The loss function is defined as the sum of the error (e.g., mean square error) between the reproduced image (120 or 180) and the target image (140) and the energy efficiency. If the energy efficiency is as shown in Equation 1 above, the energy efficiency always has a value between 0 and 1. Therefore, if the loss function is minimized, the loss function can be defined as follows using weights so that the energy efficiency approaches 1.

[0022]

[0023] Here, γ represents energy efficiency, and w represents a weight. Depending on the embodiment, the weight (w) can be preset to various values. For example, the extent to which energy efficiency is included in the optimization can be controlled by adjusting the size of the weight (w). The process of optimizing the phase pattern so that the value of the loss function converges is reviewed again in Figure 2 and below.

[0024] FIG. 2 is a flowchart illustrating an example of a hologram optimization method according to an embodiment of the present invention.

[0025] Referring to FIG. 2, the hologram optimization device (hereinafter referred to as the "device") initializes a phase pattern (S200). For example, the device may initialize the phase pattern to have a uniform random distribution. Additionally, the initial phase pattern may be set to various values.

[0026] The device obtains a complex wavefront of a spatial light modulator for a phase pattern and propagates the complex wavefront using a method such as ASM to generate a reproduction image (S210). The device extracts a certain area in the center of the reproduction image as a region of interest (S220). For example, the center of the reproduction image can be extracted as a region of interest by multiplying a binary mask having a resolution (1600*900) lower than the resolution of the reproduction image (e.g., 1920*1080) by the reproduction image. In addition, various methods for extracting the center of the reproduction image, excluding the edges, as a region of interest can be applied to the present embodiment.

[0027] The device calculates a loss function between the reproduced image and the target image (S220). The target image is predefined. In the present embodiment, the loss function is defined as the sum of a first value representing the difference in intensity of each pixel between the reproduced image and the target image, and a second value representing the ratio of the energy of the region of interest to the incident energy of the spatial light modulator. An example of the second value is defined in Equation 1, and an example of the loss function is defined in Equation 2.

[0028] If the value of the loss function is greater than a predefined reference value (ε), the device updates the phase pattern (S250), generates a reproduced image based on the updated phase pattern, and performs the process of obtaining the loss function (S210, S220, S2320) again. The device repeats the above process of updating the phase pattern and obtaining the loss function (S250, S210, S220, S230) until the value of the loss function is less than the reference value. In one embodiment, the device can update the phase pattern through backpropagation using gradient descent so that the loss function is minimized. Since gradient descent itself is already a widely known method, a detailed description thereof is omitted.

[0029] FIG. 3 is a diagram illustrating a simulation result of hologram optimization according to an embodiment of the present invention.

[0030] Referring to Figure 3, the left figure shows the result of a simulation of a hologram generated using a conventional method, and the right figure shows the result of a simulation of a hologram generated using an optimized method according to an embodiment of the present invention. The PSNR (Peak Signal-to-Noise Ratio) was measured only within the region of interest.

[0031] The image reproduced by the existing method has a high PSNR, but the energy efficiency is 0.58 and noise exists outside the region of interest. In the image reproduced in this embodiment, when the weight (w) of the loss function is set to 0.1, the PSNR decreases but the energy efficiency becomes 0.99, confirming that most of the energy incident on the spatial light modulator is used to reproduce the image within the region of interest, and as a result, there is almost no noise outside the region of interest.

[0032] FIG. 4 is a graph illustrating the results of hologram optimization according to an embodiment of the present invention.

[0033] Referring to FIG. 4, the weight (w) of the loss function according to an embodiment of the present invention is changed to four cases of 0, 0.001, 0.01, and 0.1, and the holograms for 100 target images are optimized, and the average PSNR and energy efficiency of the images reproduced with the optimized holograms are shown in the graph. It can be seen that as the weight (w) increases, the PSNR decreases, but the energy efficiency increases. It can be seen that the PSNR and energy efficiency can be adjusted by adjusting the size of the weight.

[0034] FIG. 5 is a drawing showing an actual reproduction result of an optimized hologram according to an embodiment of the present invention.

[0035] Referring to Figure 5, both images were reproduced with the same intensity of light source and captured with the same camera conditions. Similar to the simulation results in Figure 3, as the weight (w) of the loss function increases, the region of interest (the area within the white border) becomes brighter. Furthermore, while the simulation matches the brightness of the reproduced image to that of the target image, the experiment in Figure 5 allows for post-processing, resulting in the brightness of the reproduced image becoming brighter and the area outside the region of interest becoming darker.

[0036] FIG. 6 is a diagram illustrating an example configuration of a hologram optimization device according to an embodiment of the present invention.

[0037] Referring to FIG. 6, the hologram optimization device (600) includes an image generation unit (610), a loss calculation unit (620), an update unit (630), and an iterative execution unit (640). In one embodiment, the hologram optimization device (600) may be implemented as a computing device including a memory, a processor, and an input / output device. In this case, each component of the present embodiment may be implemented as software (program), loaded into the memory, and then executed by the processor.

[0038] The image generation unit (610) inputs a phase pattern into a spatial light modulator to generate a reproduction image. For example, the image generation unit (610) can obtain a complex wavefront of the spatial light modulator for the phase pattern and propagate the complex wavefront a certain distance using a method such as ASM to generate a reproduction image. The image generation unit (610) can calculate the intensity of each pixel of the reproduction image for the phase pattern through a simulation method that mathematically models the spatial light modulator.

[0039] The image generation unit (610) also extracts a region of interest from the reproduced image. The region of interest is a certain area in the center of the reproduced image. In one embodiment, the image generation unit (610) can extract the region of interest by multiplying the reproduced image by a binary mask consisting of 0s and 1s. In this case, the size of the area consisting of 1s in the binary mask becomes the size of the region of interest.

[0040] The loss calculation unit (620) obtains a value of a loss function defined as the sum of a first value representing the difference between the reproduced image and the target image and a second value representing the ratio of the energy of the region of interest and the incident energy of the spatial light modulator. In one embodiment, the first value may be the mean square error between the reproduced image and the target image. For example, the first value may be the accumulated value of the difference in intensity of each pixel of the two images. In addition, various values ​​representing the error between the two images may be used as the first value. An example of the second value representing energy efficiency is defined in Mathematical Expression 1.

[0041] The update unit (630) updates the phase pattern so that the value of the loss function decreases. In one embodiment, the update unit (630) may update the phase pattern so that the value of the loss function converges through gradient descent. In addition, various conventional artificial intelligence learning methods may be applied to this embodiment.

[0042] The repeating unit (640) repeatedly performs the process of generating a reproduction image and updating a phase pattern until the value of the loss function becomes lower than a predefined reference value.

[0043] The present invention can also be implemented as computer-readable program code on a computer-readable recording medium. Computer-readable recording media include all types of recording devices that store data that can be read by a computer system. Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disks, and optical data storage devices. Furthermore, computer-readable recording media can be distributed across network-connected computer systems, allowing computer-readable code to be stored and executed in a distributed manner.

[0044] The present invention has been described above, focusing on preferred embodiments thereof. Those skilled in the art will appreciate that the present invention can be implemented in modified forms without departing from its essential characteristics. Therefore, the disclosed embodiments should be considered illustrative rather than limiting. The scope of the present invention is set forth in the claims, not the foregoing description, and all differences within the scope equivalent thereto should be construed as being encompassed by the present invention.

Claims

1. A step of generating a reproduction image by inputting a phase pattern into a spatial light modulator; A step of obtaining a region of interest from the above playback image; A step of obtaining a value of a loss function defined as the sum of a first value representing the difference between the reproduced image and the target image and a second value representing the ratio between the energy of the region of interest and the incident energy of the spatial light modulator; and A step of updating the phase pattern so that the value of the loss function decreases; and A hologram optimization method, characterized in that it comprises a step of repeatedly performing the steps from the step of generating the above-mentioned reproduction image to the step of updating the above-mentioned phase pattern until the value of the loss function becomes lower than a pre-defined reference value.

2. In paragraph 1, A hologram optimization method characterized in that the above loss function is defined as “the first value + weight * (1 - the second value)”, and the reflection ratio of the second value is adjusted by adjusting the size of the weight.

3. In paragraph 1, the step of obtaining the area of ​​interest comprises: A hologram optimization method comprising: a step of generating the region of interest by multiplying the reproduction image by a binary mask having a resolution smaller than that of the reproduction image; 4. In the first paragraph, the step of updating the phase pattern comprises: A hologram optimization method, characterized by including a step of updating the phase pattern through backpropagation using gradient descent.

5. In paragraph 1, A hologram optimization method, characterized in that the first value is a value representing the mean square error between the region of interest of the reproduced image and the region of the corresponding target image.

6. An image generation unit that inputs a phase pattern into a spatial light modulator to generate a reproduced image and extracts an area of ​​interest from the reproduced image; A loss calculation unit that obtains a value of a loss function defined as the sum of a first value representing the difference between the reproduced image and the target image and a second value representing the ratio between the energy of the region of interest and the incident energy of the spatial light modulator; An update unit that updates the phase pattern so that the value of the loss function decreases; and A hologram optimization device characterized by including an iterative performing unit that repeatedly performs the process of generating the above-mentioned reproduction image and updating the above-mentioned phase pattern until the value of the loss function becomes lower than a pre-defined reference value.

7. In paragraph 6, the image generation unit, A hologram optimization device characterized in that it generates the region of interest by multiplying the reproduction image by a binary mask having a resolution smaller than that of the reproduction image.

8. In paragraph 6, the renewal unit, A hologram optimization device characterized by updating the phase pattern through backpropagation using gradient descent.

9. In paragraph 6, A hologram optimization device, characterized in that the first value is a value representing the mean square error between the region of interest of the reproduction image and the region of the corresponding target image.

10. A computer-readable recording medium having recorded thereon a computer program for performing the method described in paragraph 1.

Images