Method for designing holographic optical element by using deep learning and manufacturing apparatus
The deep learning-based method for holographic optical element design and manufacturing addresses the challenges of varying illumination conditions by generating optimized wavenumber vector maps, enabling efficient production and use in applications like HMDs with improved performance and economic efficiency.
Patent Information
- Application Number
- PCT/KR2023/008187
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-31
- Filing Date
- 2023-06-14
- Publication Date
- 2024-12-05
AI Technical Summary
Conventional methods for designing and manufacturing holographic optical elements face challenges in achieving target performance under varying illumination conditions, particularly in applications like HMDs where limited light sources are available, and require extensive computation time and resources for wavelength compensation.
A deep learning-based method that generates wavenumber vector maps for holographic optical elements, allowing for the use of lasers with different wavelengths and radiation angles from the reproduction conditions, and enables quick design and recording of holographic patterns with high freedom, using a deep learning model to optimize wavefront calculations and stitching for optical recording.
This approach enables efficient production of holographic optical elements that can operate under arbitrary target wavefront conditions, reducing computation time and resource requirements, and allows for the use of inexpensive, small projection modules in HMDs, improving economic efficiency and performance.
Smart Images

Figure KR2023008187_05122024_PF_FP_ABST
Abstract
Description
Holographic optical element design method and manufacturing device using deep learning
[0001] The present invention relates to a technology related to the design and manufacturing of a holographic optical element that can replace a conventional optical element, and more specifically, to a method and device capable of calculating and exposing an optimized holographic pattern according to a given design goal.
[0002] Due to the wavelength- and angle-selective response characteristics of holographic optical devices, it is difficult to properly provide the targeted performance under illumination conditions that differ from the optical characteristics of the laser used during fabrication.
[0003] Therefore, in the past, a laser with the same characteristics as the reproduction conditions was arranged and used for optical recording of a holographic optical element, or a grid pattern of an optimized holographic optical element was designed and utilized through a geometric / diffraction optical simulation that took a long time for wavelength compensation.
[0004] However, the former has the problem of low effectiveness in applications that must satisfy small size / low power characteristics, such as HMD (Head-Mounted Display), due to limited available light sources, and the latter has the limitation of large computational time and computational resources required for diffraction and ray tracing simulation under each simulation condition.
[0005] The present invention has been devised to solve the above problems, and the purpose of the present invention is to provide a method capable of designing a grid pattern of a desired holographic optical element in a short time and recording a holographic optical element pattern with a high degree of freedom by using a laser having a wavelength and an irradiance angle that are different from the reproduction conditions, taking into account both the characteristics of the reproduction light and the characteristics of the holographic recording medium.
[0006] In order to achieve the above object, a method for manufacturing an HOE according to one embodiment of the present invention comprises the steps of: obtaining a wave vector map for a projected wave and a diffracted wave on a surface of an HOE to be manufactured; generating a wave vector map for a reference wave and a signal wave on the surface of an HOE to be manufactured from the obtained wave vector map for the projected wave and the diffracted wave; and manufacturing an HOE based on the generated wave vector map.
[0007] The wave vector map may be a map representing the wave vector for each local region of the HOE surface.
[0008] The acquisition step acquires wave vector maps for the projected wave and the diffracted wave from the target reproduction light conditions on the surface of the HOE to be manufactured, and the reproduction light conditions may include at least one of the wavelength characteristics of the reproduction light, the projection characteristics, the reproduction wavefront, and the medium conditions of the HOE.
[0009] A method for manufacturing an HOE according to one embodiment of the present invention further includes a step of converting wave vector maps for acquired projection waves and diffraction waves into a two-dimensional image; and the generation step inputs the two-dimensional image converted in the conversion step into a deep learning model trained to generate wave vector maps for a reference wave and a signal wave on a surface of an HOE to be manufactured from the two-dimensional image, thereby generating wave vector maps for the reference wave and the signal wave.
[0010] The transformation step can convert the three-dimensional vector components of the frequency vector map into three color values of a two-dimensional image. The transformation step can normalize the three-dimensional vector components of the frequency vector map and then convert them into three color values.
[0011] The deep learning model can generate wave vector maps for the reference wave and signal wave by synthesizing the latent vector encoded through the neural network with the adjustment variables for wavelength conditions and medium properties and extracting the features from the two-dimensional image.
[0012] The fabrication step may include a step of calculating an actual applied wavefront for fabricating the HOE from the generated wave vector map; a step of optically recording and curing the HOE with the calculated actual applied wavefront.
[0013] The calculation step can split the actual applied wavefront and stitch approximate wavefronts in individual local regions if the actual applied wavefront calculated from the generated wave vector map cannot be implemented with a single optical element.
[0014] A HOE manufacturing system according to another embodiment of the present invention includes a computing device that generates wave vector maps for a reference wave and a signal wave on a surface of an HOE to be manufactured from wave vector maps for a projected wave and a diffracted wave on a surface of an HOE to be manufactured; and an optical recording device that manufactures an HOE based on the generated wave vector map.
[0015] A method for designing an HOE according to another embodiment of the present invention includes the steps of: obtaining wave vector maps for a projected wave and a diffracted wave on a surface of an HOE to be manufactured; and generating wave vector maps for a reference wave and a signal wave on the surface of an HOE to be manufactured from the obtained wave vector maps for the projected wave and the diffracted wave.
[0016] According to another embodiment of the present invention, a computer-readable recording medium has recorded thereon a program capable of performing a method for manufacturing an HOE, the method comprising: obtaining a wave vector map for a projected wave and a diffracted wave on a surface of an HOE to be manufactured; and generating a wave vector map for a reference wave and a signal wave on the surface of an HOE to be manufactured from the obtained wave vector map for the projected wave and the diffracted wave.
[0017] As described above, according to embodiments of the present invention, by considering both the characteristics of the reproducing light and the characteristics of the holographic recording medium, it is possible to design a grid pattern of a desired holographic optical element in a short time and record a holographic optical element pattern with a high degree of freedom even when using a laser whose wavelength and radiation angle are different from the reproducing conditions.
[0018] In particular, according to embodiments of the present invention, it is possible to optimize and produce a wave vector map and recording wavefront required for actual optical recording production in a short time even when the target reproduction light conditions change by using deep learning.
[0019] In addition, according to embodiments of the present invention, by relaxing the requirements for matching recording light and playback light during the manufacturing process, it is possible to utilize inexpensive and compact projection modules in HMDs, HUDs, etc. based on holographic optical elements, thereby increasing cost-effectiveness and enabling the manufacturing of highly efficient holographic optical elements capable of operating under Bragg conditions under any target wavefront condition.
[0020] Figure 1 is a HOE design method according to one embodiment of the present invention;
[0021] Figure 2 shows the process of creating a dataset for training and inference of a deep learning model for HOE generation.
[0022] Figure 3 is an example of a two-dimensional image representation of a wave vector map.
[0023] Figure 4 is an example of a deep learning model that trains and infers using a wave vector map expressed as a two-dimensional image.
[0024] Figure 5 is a flowchart showing a series of steps for generating wave vector maps R, S required for manufacturing optical devices from a deep learning model and then optically manufacturing them.
[0025] Figure 6 is a conceptual diagram of manufacturing a wavefront by dividing the wavefront calculated from the wave vector maps R, S into local regions and stitching and recording the wavefront.
[0026] Hereinafter, the present invention will be described in more detail with reference to the drawings.
[0027] A holographic optical element (HOE) can be manufactured by patterning the interference pattern of two light waves (reference wave and signal wave) incident from different directions onto a recording medium such as a photopolymer.
[0028] The interference pattern recorded at this time forms a volume grating that is periodically repeated within the holographic recording medium, and if the probe wave that enters the playback stage later satisfies the Bragg condition of the volume grating, a diffracted wave is output through the diffraction phenomenon. If the incoming light deviates from the Bragg condition of the recorded volume grating, problems such as a decrease in the intensity of the diffracted wave and distortion of the light output direction occur.
[0029] In an embodiment of the present invention, in the HOE design and manufacturing process, a deep learning-based HOE design capable of reproducing a light wave intended by a user with high efficiency without distortion in the reproduction stage by considering the difference (wavelength, light incident direction, etc.) between recording light and reproduction light and the characteristics of the medium is proposed, and a method for manufacturing an HOE pattern designed through the same is proposed.
[0030] Figure 1 is a schematic diagram of an HOE design method according to one embodiment of the present invention. First, a desirable wavevector map (120) on the HOE surface is extracted from the target reproducing light conditions (110, wavelength characteristics and projection characteristics, reproducing wavefront, medium conditions, etc.).
[0031] The wave vector map should be composed of two wavefronts for each wavelength under recording and playback conditions. For example, in the case of recording light, it can be composed of a wave vector map for the reference wave (R) and a wave vector map for the signal wave (S), and in the case of playback light, it can be composed of a wave vector map for the projected wave (P) and a wave vector map for the diffracted wave (D).
[0032] At this time, when utilizing a monochrome recording light, two wave vector maps must be derived, as in the upper part of the extraction map (120), and when utilizing a full-color recording, wave vector maps corresponding to twice the number of wavelength bands to be utilized must be derived. At the lower part of the extraction map (120), wave vector maps of the projected wave and the diffracted wave are simulated for each of the three red, green, and blue (RGB) channels in order to utilize a three-color recording.
[0033] The extracted wave vector map is used as input to a deep learning model (130), and as output, a wave vector map (140) required for HOE manufacturing through optical recording is generated. The entire deep learning inference process can be configured in a processing and analysis system such as a server or an individual local PC.
[0034] Figure 2 schematically illustrates the process of creating a dataset for training and inference of a deep learning model for HOE generation. To ensure that the deep learning model utilizing CNNs can effectively form a proxy model, the dataset should be expressed in the form of a two-dimensional image.
[0035] Therefore, as shown in Fig. 2, a step is required to process the wave vector of the light ray emitted from the source, which is the projection wave, and the wave vector of the light ray arriving at the destination, which is the diffracted wave, into a two-dimensional image form that is easy to recognize in a deep learning model.
[0036] If the locations of the Source and Destination are (x1, y1, z1) and (x2, y2, z2), respectively, the wave vectors P and D at each HOE surface (x, y, z) can be expressed as follows (λ is the wavelength).
[0037]
[0038]
[0039] At this time, since P and D have vector values expressed in a three-dimensional form, it is possible to diagram each component (x, y, z-axis components) as in the example provided in Fig. 2. At this time, in order to express each range, the P and D vectors were normalized to have a maximum size of 1. If the surface of the HOE is N, respectively x and N y If we compute P and D by sampling with the number of , the x, y, and z-axis components of each vector, P x , P y , P z and D x , D y , D z are each N x ×N y It can be expressed in the form of a 1-channel matrix with a resolution of . At this time, all elements constituting the matrix are scalar values limited to the interval [-1, 1].
[0040] For more efficient data handling, it is desirable to represent the wave vector map as a single two-dimensional image (in RGB format) with three channels rather than handling the matrix values of the x, y, and z-axis components individually. For example, P x , P y , P z If each component of is normalized to the interval [0, 1] as shown below and then assigned to the red, green, and blue color channels, P and D can each be expressed as a single two-dimensional image information.
[0041] p i = (P i+1)×0.5
[0042] d i = (D i +1)×0.5
[0043] At this time p i and d i are respectively P x , P y , P z and D x , D y , D z N normalized to the interval [0, 1] x ×N y It is a matrix. An example of red, green, and blue (RGB) type expression using this is shown in Fig. 3. In Fig. 3, for the expression of the red, green, and blue image type, P is used respectively. x , D x Red channel, P y , D y Green channel, P z , D z was assigned to the blue channel (however, in Figure 3, red, green, and blue are expressed as gray levels).
[0044] Since conventional deep learning models, especially those utilizing CNN, are often specialized in image input and output, when an image-based frequency vector map, such as in Fig. 2 or Fig. 3, is used, the application of the deep learning algorithm to input and output becomes easier.
[0045] When implementing a deep learning model for high-speed HOE design, a neural network-based model is suitable, considering that the input / output data is in image format. Specifically, it is desirable to include a Skip Connection structure to preserve high-frequency components of the image even during pooling or downsampling processes. For example, as shown in Figure 4, a conventional U-Net neural network can be applied to train and infer a frequency vector map expressed as a two-dimensional image.
[0046] At this time, unlike the conventional U-Net, a separate neural network block may be required for inserting additional adjustment variables. For example, between the 410 (λ1 condition playback light data set (P, D)) and the 420 (λ2 condition recording light data set (R, S)) datasets, adjustment variables for wavelength conditions and medium properties can be vectorized, encoded through a neural network layer, and then applied in the form of a latent vector (430).
[0047] To train a deep learning model, a pair of pre-calculated wave vector map datasets is required based on the adjustment variables (λ1, λ2, medium shrinkage, etc.) shown in Fig. 430. This can be obtained by applying an optimization algorithm after modifying the input / output patterns of a given HOE using a conventional optical simulation tool to conditions corresponding to each adjustment variable.
[0048] Figure 5 illustrates a series of steps for generating a wave vector map (R, S) required for manufacturing an optical device from a deep learning model for HOE generation based on a target wave vector map (P, D), and then optically manufacturing it.
[0049] First, to generate the wave vector maps of P and D, the area of the entire HOE is N x ×N y After dividing into a set of local areas (S510), P and D that must be incident on each local area are calculated (S520), and the resolution N x ×N y Each wave vector map expressed as a two-dimensional image is generated (S530).
[0050] Even though the grid within the entire HOE is complex and non-periodic, if it is divided into a sufficiently large number of local regions, each local region can be approximated by a periodic and simple grid, so N x and N y As the number of samples increases, the wavefront can be accurately approximated.
[0051] Afterwards, a wave vector map (R, S) of the recording light required for optical fabrication is generated through a deep learning model (S540), and then the actual applied wavefront is calculated from the wave vector map for fabrication of HOE through optical patterning (S550).
[0052] If the wavefront calculated from the wave vector map can be implemented with a conventional single optical element, it can be utilized; otherwise, it can be implemented by locally dividing the wavefront and stitching and recording the approximate wavefront in each local area (S560).
[0053] Afterwards, through the HOE optical recording and curing step (S570), an HOE satisfying the final R and S wave vector maps can be manufactured.
[0054] Figure 6 illustrates a manufacturing concept based on holographic printing technology, which divides the wavefront calculated from the wave vector maps R, S into local regions (unit cells or unit hogels) and stitches and records the wavefront when it is impossible to manufacture it with a general optical element.
[0055] First, an optical recording device is required that can finely adjust the direction and focus position of the incident light on both the reference wave and the signal wave.
[0056] In Fig. 6, the manufacturing device of HOE is designed to cause interference at the photopolymer medium location after the laser light from the red, green, and blue laser is divided into a signal wave and a reference wave through an optical element module (610) that expands and divides the laser light.
[0057] In each optical path, a wavefront adjustment unit (620) is required to adjust the incident light direction and wavefront. Since the wavefront adjustment unit (620) must support adjusting the incident light direction in all directions, it may include a two-axis motorized scanning device using, for example, a Galvo Mirror, a MEMS Mirror, etc. In addition, it may include a composite system composed of a phase spatial light modulator and a filtering optical system to adjust the shape of a wider range of wavefronts.
[0058] The substrate and mount (630) support the photopolymer medium while supporting lateral micro-movement to record individual hogels and then move the medium to adjust the recording area.
[0059] At this time, the light waves that must be incident on the final photopolymer through the wavefront adjustment unit (620) are calculated and output from the deep learning model, and the wave vector maps of R and S are each M. x ×M y It is calculated sequentially by dividing into R and S. For example, the wavefront to be recorded in the Kth hogel is R of the Kth cell in the wave vector map of R and S. x , R y (And S x , S y ) After generating the information, it is integrated in two dimensions to calculate the wavefront shape.
[0060] An example of calculating a wavefront from a wave vector map (640, 650) expressed as a two-dimensional image at 660, 670 is shown. The wavefront is implemented by a wavefront adjustment unit (620) and optically recorded, and then the photopolymer is laterally moved by the size of one hogel using a substrate and mount (630), and then the wavefront of the K+1th hogel is calculated and sequentially optically recorded, thereby implementing a pattern that satisfies the target reproduction conditions of the entire HOE.
[0061] So far, a method for designing a holographic optical element using deep learning and a manufacturing device have been described in detail with preferred embodiments.
[0062] In an embodiment of the present invention, a form optimized for use as input / output of a deep learning algorithm by expressing the light conditions during recording and reproduction as a wave vector map in the form of a two-dimensional image, a design method capable of analyzing and compensating for the difference in wavelength between reproduction light and recording light, medium shrinkage, and light input conditions using a neural network-based deep learning model, and a HOE manufacturing system capable of stitching and recording a wavefront derived from a wave vector map are presented.
[0063] By taking into account both the characteristics of the regenerated light and the characteristics of the holographic recording medium, it is possible to design the grid pattern of the desired holographic optical element in a short time and record the holographic optical element pattern with a high degree of freedom even when using a laser whose wavelength and radiation angle are different from the regenerated conditions.
[0064] The technical idea of the present invention can be implemented as a replacement for conventional optical elements in various applications, such as head-mounted displays, head-up displays, and solar cell concentrators.
[0065] Meanwhile, it goes without saying that the technical idea of the present invention can also be applied to a computer-readable recording medium containing a computer program that performs the functions of the device and method according to the present embodiment. In addition, the technical idea according to various embodiments of the present invention can be implemented in the form of computer-readable code recorded on a computer-readable recording medium. The computer-readable recording medium can be any data storage device that can be read by a computer and store data. For example, the computer-readable recording medium can be a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical disk, a hard disk drive, etc. In addition, the computer-readable code or program stored on the computer-readable recording medium can be transmitted through a network connected between computers.
[0066] In addition, although the preferred embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific embodiments described above, and various modifications can be made by a person having ordinary skill in the art to which the present invention pertains without departing from the gist of the present invention as claimed in the claims, and such modifications should not be understood individually from the technical idea or prospect of the present invention.
Claims
1. A step of obtaining wave vector maps for the transmitted and diffracted waves on the surface of the HOE to be manufactured; A step of generating wave vector maps for reference waves and signal waves on the surface of the HOE to be manufactured from wave vector maps for the acquired projection waves and diffracted waves; and A method for producing an HOE, characterized by comprising a step of producing an HOE based on a generated wave vector map.
2. In claim 1, The wave vector map is, A method for producing an HOE, characterized in that the HOE is a map representing a wave vector for each local area of the HOE surface.
3. In claim 2, The acquisition stage is, Obtain wave vector maps for the incident and diffracted waves from the target reproducible light conditions on the HOE surface to be manufactured, The playback conditions are: A method for manufacturing an HOE, characterized in that it includes at least one of wavelength characteristics of regenerated light, projection characteristics, regenerated wavefront, and medium conditions of the HOE.
4. In claim 2, A step of converting the wave vector map for the acquired projection wave and diffraction wave into a two-dimensional image is further included; The creation phase is, A method for producing an HOE, characterized in that a two-dimensional image transformed in a transformation step is input into a deep learning model trained to generate wave vector maps for a reference wave and a signal wave on a HOE surface to be produced from a two-dimensional image, thereby generating wave vector maps for a reference wave and a signal wave.
5. In claim 4, The conversion step is, A method for producing an HOE characterized by converting three-dimensional vector components of a wave vector map into three color values of a two-dimensional image.
6. In claim 5, The conversion step is, A method for producing an HOE characterized by normalizing the three-dimensional vector components of a wave vector map and then converting them into three color values.
7. In claim 4, The deep learning model is, A method for producing an HOE characterized in that a latent vector encoded through a neural network by vectorizing adjustment variables for wavelength conditions and medium properties is synthesized into a feature extracted from a two-dimensional image, and then a wave vector map for a reference wave and a signal wave is generated.
8. In claim 1, The production stage is, A step of calculating the actual applied wavefront for manufacturing HOE from the generated wave vector map; A method for manufacturing an HOE, comprising the steps of optically recording and hardening the HOE with a calculated actual wavefront.
9. In claim 8, The calculation steps are: A method for fabricating an HOE, characterized in that when the actual applied wavefront calculated from the generated wave vector map cannot be implemented with a single optical element, the actual applied wavefront is divided and an approximate wavefront is stitched in each local area.
10. A computing device that generates wave vector maps for reference waves and signal waves on the surface of the HOE to be manufactured from wave vector maps for the transmitted and diffracted waves on the surface of the HOE to be manufactured; and An HOE manufacturing system comprising: an optical recording device for manufacturing an HOE based on a generated wave vector map.
11. A step of obtaining wave vector maps for the transmitted and diffracted waves on the surface of the HOE to be manufactured; and A method for designing an HOE, comprising: a step of generating wave vector maps for a reference wave and a signal wave on a surface of an HOE to be manufactured from wave vector maps for acquired projected waves and diffracted waves; 12. A step of obtaining wave vector maps for the transmitted and diffracted waves on the surface of the HOE to be manufactured; and A computer-readable recording medium having recorded thereon a program capable of performing a method for manufacturing an HOE, characterized in that it comprises the step of generating wave vector maps for reference waves and signal waves on the surface of an HOE to be manufactured from wave vector maps for acquired projection waves and diffracted waves.
Citation Information
Patent Citations
Hologram screen and method for manufacturing the same
JP2016212232A
Diffraction element and diffraction element design method
JP2019086703A
Method for producing Syrian yogurt and yogurt prepared thereby
KR1020240029637A
Apparatus and method for speckle removal in hologram using light field conversion and deep learning
KR102190773B1
Composition for Preventing or Treating Melanoma Comprising AC_774
KR102459443B1
