Large-angle oblique incidence near-to-eye display system and hologram generation method

Abstract

The invention relates to a wide-angle oblique incidence-oriented near-to-eye display system and a hologram generation method, and the method comprises the steps: obtaining the initial phase distribution of a reflection-type silicon-based liquid crystal spatial light modulator plane, and enabling the initial phase distribution to have a physical resolution; a virtual oversampling grid is constructed, the initial phase distribution is mapped to an expanded high-resolution grid, and the sampling interval of the expanded high-resolution grid is smaller than the physical sampling interval; constructing a modulation light field on the expanded high-resolution grid, and superposing a linear phase term corresponding to oblique incidence; carrying out propagation calculation on the modulated light field by adopting a Shifted angular spectrum method to obtain high-resolution light field distribution of a target plane; the high-resolution light field distribution of the target plane is mapped back to the physical resolution through average downsampling, a reconstructed image is obtained, and the method has the advantages that on the premise that high quality is kept, the calculation efficiency and the generation acceleration capacity are greatly improved, and the technical effect that hologram generation is rapidly completed is achieved.

Description

Technical Field

[0001] This invention relates to the field of visual technology applications, and in particular to a near-eye display system for large-angle oblique incidence and a hologram generation method. Background Technology

[0002] With the continuous evolution of human-computer interaction, new display devices, represented by Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR), are gradually moving from laboratory environments into daily life and industrial settings. In applications such as education and training, remote collaboration, medical-assisted surgery, and entertainment displays, users not only want to observe virtual content but also desire that this virtual content can be directly modulated with light wavefronts to reproduce realistic three-dimensional visual perception, thanks to advancements in hardware miniaturization, computing resources, and the programmability of optical devices. This has led to a growing interest in display technologies that can reproduce realistic three-dimensional visual perception.

[0003] As a display device worn directly on the head, the weight distribution and optical structure volume of a near-eye display system directly determine the user's comfort and wearability during prolonged use. Traditional optical or video-vision near-eye systems typically rely on multiple lens groups, heavy beam splitters (BS), or complex geometric optical components to achieve imaging, optical path folding, and virtual image magnification. Although these components can ensure basic optical performance, their physical thickness based on the principle of refraction significantly increases the weight and volume of the front end of the optical module.

[0004] Against the backdrop of the rapid development of waveguide and freeform surface technologies, although the trend toward flat-panel construction is obvious, existing research has pointed out that as the demand for performance indicators such as field of view, imaging uniformity and light efficiency continues to increase, in order to ensure high-quality beam coupling, the input coupling structure in traditional architectures often becomes more complex, and may even require prisms or stacked structures, which poses a challenge to truly realizing lightweight wearable forms.

[0005] In practical engineering design, there is a close coupling relationship between the light source coupling, the output array, and the compensation structure. In order to maintain the reversibility of the optical path or the normal incidence condition, it is often necessary to introduce additional bending elements at the front end of the optical path. This inevitably brings about volume redundancy and structural complexity to the optical system while achieving high-efficiency propagation.

[0006] Therefore, while traditional near-eye optical architectures can provide high imaging quality, their reliance on beam splitters and the physical stacking of multiple components inevitably bring weight, thickness, and engineering challenges, limiting the development of systems towards extreme thinness and lightness.

[0007] In near-eye display applications, the performance of the SLM (Spatial Lens) is a core determining factor for near-eye holographic systems. However, the limited resolution and pixel structure of the SLM also bring several problems. Pixelation features can lead to high-order artifacts, affecting image clarity; limited spatial bandwidth restricts the field of view or viewing range; and the use of coherent light sources may introduce speckle noise, thereby reducing visual quality. In addition, assembly errors of optical components and wavefront distortion in the system can also affect the final reconstruction effect.

[0008] Currently, no effective solution has been proposed to address the problem of insufficient physical sampling bandwidth caused by the limited pixel spacing of spatial light modulators under large-angle oblique incidence conditions, which leads to spectral aliasing, bandwidth truncation, and significant degradation of high-frequency details and contrast in reconstructed images. Summary of the Invention

[0009] The purpose of this invention is to address the shortcomings of existing technologies by providing a near-eye display system and hologram generation method for large-angle oblique incidence, thereby solving the technical problems of insufficient physical sampling bandwidth caused by the limited pixel spacing of spatial light modulators under large-angle oblique incidence conditions, which leads to spectral aliasing, bandwidth truncation, and significant degradation of high-frequency details and contrast in reconstructed images.

[0010] To achieve the above objectives, the technical solution adopted by the present invention is as follows: This invention provides a method for generating holograms using off-axis calculations for large-angle oblique incidence, comprising: obtaining an initial phase distribution of a reflective silicon-based liquid crystal spatial light modulator plane, the initial phase distribution having physical resolution; constructing a virtual oversampling grid, mapping the initial phase distribution to an extended high-resolution grid, the sampling interval of the extended high-resolution grid being smaller than the physical sampling interval; constructing a modulated light field on the extended high-resolution grid, and superimposing a linear phase term corresponding to oblique incidence; performing propagation calculations on the modulated light field using the Shifted angular spectrum method to obtain a high-resolution light field distribution of the target plane; and mapping the high-resolution light field distribution of the target plane back to physical resolution through average downsampling to obtain a reconstructed image.

[0011] Optionally, the virtual oversampling mesh is constructed using nearest-neighbor upsampling, copying the phase value of each physical pixel to fill the corresponding s×s sub-pixel region, where s is the oversampling factor, and the expanded resolution is... ,in This refers to the physical resolution.

[0012] Optionally, average downsampling groups the high-resolution light field distribution into s×s sub-pixel blocks and averages the pixel values ​​within each sub-pixel block to obtain an output pixel.

[0013] Optionally, the method further includes: calculating the error between the reconstructed image and the target image, and updating the initial phase distribution of the spatial light modulator plane through gradient backpropagation until the convergence condition is met.

[0014] Optionally, the method further includes: constructing a phase prediction neural network, wherein the phase prediction neural network uses U-Net as its backbone architecture and includes an encoder and a decoder. The encoder extracts multi-scale features through multi-layer convolution and downsampling operations, and the decoder restores spatial resolution and fuses features from the encoding stage through upsampling and skip connections; inputting the amplitude distribution of the target image into the phase prediction neural network and outputting an initial phase distribution after a single forward propagation; using the mean square error between the amplitude distributions of the reconstructed image and the target image as a loss function, updating the parameters of the phase prediction neural network using an error backpropagation algorithm to complete network training; and inputting the target image to be generated into the trained phase prediction neural network and generating the corresponding holographic phase distribution after a single forward propagation.

[0015] Further, optionally, U-Net includes 8 downsampling and 8 upsampling operations, with the encoder and decoder being symmetrical; downsampling is implemented using 4×4 convolutions with a stride of 2, and upsampling is implemented using 4×4 transposed convolutions with a stride of 2; the activation function of the downsampling path is LeakyReLU with a slope of 0.2 on the negative half-axis, and the activation functions of the upsampling path and the output path are ReLU; the output phase is constrained to the interval [-π, π] by hard constraints; the network uses batch normalization BatchNorm2d.

[0016] Optionally, parameter updates use the Adam optimizer, which sets an initial learning rate and employs an exponential decay strategy to bring the phase prediction neural network to convergence.

[0017] This invention provides a near-eye display system for large-angle oblique incidence, comprising: a reflective silicon-based liquid crystal spatial light modulator, a light source module, a rotating stage, an industrial camera, and a processing unit. The reflective silicon-based liquid crystal spatial light modulator is used to load a phase hologram; the light source module is used to provide coherent illumination; the rotating stage is used to mount the reflective silicon-based liquid crystal spatial light modulator and adjust its tilt angle relative to the incident light axis; the industrial camera is used to acquire and reconstruct images; and the processing unit, connected to the reflective silicon-based liquid crystal spatial light modulator, the rotating stage, and the industrial camera, is used to perform modulation curve calibration of the reflective silicon-based liquid crystal spatial light modulator, camera geometric calibration, and system debugging.

[0018] Optionally, the modulation curve calibration of the reflective silicon-based liquid crystal spatial light modulator includes: adjusting the minimum and maximum voltage parameters using specific software to ensure that the display effects corresponding to grayscale 0 and 255 are consistent; camera geometry calibration includes: placing a dot array target in the optical path, capturing target images at different positions and orientations, extracting the dot center using OpenCV and solving the mapping matrix, and performing distortion correction on the subsequently acquired reconstructed images; system debugging includes: adjusting the reflective silicon-based liquid crystal spatial light modulator to a vertical position, checking that the reflected light coincides with the optical axis, rotating the reflective silicon-based liquid crystal spatial light modulator to the target oblique incidence angle using a rotary table, fine-tuning the position of the industrial camera, and determining the imaging plane by observing the clarity and focal position of the diffraction pattern.

[0019] This invention employs the above technical solution, obtaining the initial phase distribution of the plane of a reflective silicon-based liquid crystal spatial light modulator, which has physical resolution; constructing a virtual oversampling grid to map the initial phase distribution to an extended high-resolution grid, the sampling interval of which is smaller than the physical sampling interval; constructing a modulation light field on the extended high-resolution grid and superimposing the linear phase term corresponding to oblique incidence; using the Shifted angular spectrum method to calculate the propagation of the modulation light field to obtain the high-resolution light field distribution of the target plane; and mapping the high-resolution light field distribution of the target plane back to the physical resolution through average downsampling to obtain the reconstructed image. Compared with existing technologies, this invention has the following technical effects: significantly improving computational efficiency and accelerating generation capabilities while maintaining high quality, as well as rapidly completing hologram generation. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of a near-eye display system oriented towards large-angle oblique incidence according to an embodiment of the present invention; Figure 2 This is a side view schematic diagram of an oblique incidence experimental device in a near-eye display system facing a large angle oblique incidence according to an embodiment of the present invention; Figure 3 This is a schematic diagram of geometric calibration in a near-eye display system oriented towards large-angle oblique incidence according to an embodiment of the present invention; Figure 4 This is a flowchart illustrating the CITL process in a near-eye display system oriented towards large-angle oblique incidence according to an embodiment of the present invention. Figure 5 This is a flowchart illustrating a method for generating a hologram using large-angle oblique incidence off-axis calculation according to an embodiment of the present invention; Figure 6 This is a schematic diagram of a training framework for rapid generation of large-angle off-axis holograms based on U-Net in a method for generating holograms by large-angle oblique incidence off-axis calculation according to an embodiment of the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this application clearer, the application is described and illustrated below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application.

[0022] Obviously, the accompanying drawings described below are merely some examples or embodiments of this application. Those skilled in the art can apply this application to other similar scenarios based on these drawings without any inventive effort. Furthermore, it is understood that although the efforts made in this development process may be complex and lengthy, for those skilled in the art related to the content disclosed in this application, any changes to design, manufacturing, or production based on the technical content disclosed in this application are merely conventional technical means and should not be construed as insufficient disclosure of the content of this application.

[0023] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment that is mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0024] Unless otherwise defined, the technical or scientific terms used in this application shall have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms “a,” “an,” “an,” “the,” and similar words used in this application do not indicate quantity limitation and may indicate singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that includes a series of steps or units (elements) is not limited to the listed steps or units, but may also include steps or units not listed, or may include other steps or units inherent to these processes, methods, products, or apparatus. The terms “connected,” “linked,” “coupled,” and similar words used in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The terms “multiple” / “several” used in this application refer to two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can indicate: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following objects are in an "or" relationship. The terms "first," "second," and "third" used in this application are merely to distinguish similar objects and do not represent a specific ordering of the objects.

[0025] An illustrative embodiment of the present invention, such as Figure 1 As shown, Figure 1 This is a schematic diagram of a near-eye display system oriented towards large-angle oblique incidence according to an embodiment of the present invention. The near-eye display system oriented towards large-angle oblique incidence provided in this application embodiment includes: The system comprises a reflective liquid crystal on silicon (LCD) spatial light modulator 10, a light source module 12, a rotary stage 14, an industrial camera 16, and a processing unit 18. The reflective LCD spatial light modulator 10 is used to load a phase hologram; the light source module 12 is used to provide coherent illumination; the rotary stage 14 is used to mount the reflective LCD spatial light modulator 10 and adjust its tilt angle relative to the incident light axis; the industrial camera 16 is used to acquire and reconstruct images; and the processing unit 18, connected to the reflective LCD spatial light modulator 10, the rotary stage 14, and the industrial camera 16, is used to perform modulation curve calibration of the reflective LCD spatial light modulator 10, camera geometric calibration, and system debugging.

[0026] Optionally, the modulation curve calibration of the reflective silicon-based liquid crystal spatial light modulator 10 includes: adjusting the minimum and maximum voltage parameters using specific software to ensure that the display effects corresponding to grayscale 0 and 255 are consistent; camera geometry calibration includes: placing a dot array target in the optical path, capturing target images at different positions and orientations, extracting the dot center using OpenCV and solving the mapping matrix, and performing distortion correction on the subsequently acquired reconstructed images; system debugging includes: adjusting the reflective silicon-based liquid crystal spatial light modulator 10 to a vertical state, checking that the reflected light coincides with the optical axis, rotating the reflective silicon-based liquid crystal spatial light modulator 10 to the target oblique incidence angle using a rotary table 14, fine-tuning the position of the industrial camera 16, and determining the imaging plane by observing the clarity and focal position of the diffraction pattern.

[0027] Specifically, Figure 2 This is a side view schematic diagram of an oblique incidence experimental device in a near-eye display system with large-angle oblique incidence according to an embodiment of the present invention; as shown. Figure 2 As shown, the linear polarizer is a linear polarizer, the beam expander is a beam expander, and the SLM (Spatial Light Modulator) is a spatial light modulator (i.e., the reflective silicon-based liquid crystal spatial light modulator 10 in the embodiments of this application). In order to verify the large-angle oblique incidence model and obtain data suitable for deep learning training under real-world conditions, the near-eye display system provided in this application embodiment has designed and built a holographic experimental platform with high stability, high repeatability and suitability for multi-angle adjustment. The light source module 12 uses RGB three-color laser as the light source, and the reflective liquid crystal on silicon (LCOS) spatial light modulator 10 uses reflective liquid crystal on silicon (LCOS) as the modulation core. The adjustable tilt structure realizes continuous switching from normal incidence to 30° oblique incidence.

[0028] In this embodiment, polarization matching involves setting a linear polarizer in the optical path to control the polarization direction of the laser beam illuminating the SLM plane, achieving optimal polarization. Wavefront control is achieved by performing gamma (γ) correction on the SLM to precisely control the wavefront. Mechanical stability is achieved using high-precision optical equipment (rotating stage and stabilized optical platform). Imaging geometric consistency is achieved through OpenCV-based camera geometric calibration.

[0029] The construction process fully considers engineering factors such as beam collimation, polarization matching, wavefront distortion control, mechanical stability and imaging geometric consistency, so that the numerical model can be mapped to the physical system with minimal error.

[0030] In terms of beam collimation factors, to achieve a large-angle oblique incidence of 30°, this embodiment constructs an off-axis optical path by rotating the SLM. Compared to a directly tilted laser, a rotating SLM can more stably maintain the incident polarization direction and the overall optical path structure, thus making it easier to control optical path errors in engineering implementation. The construction process starts with beam collimation, and the optical path is continuously and bidirectionally checked through a combination of beam splitter, target, and lens to ensure that the incident beam and the rotation center of the SLM maintain a strictly orthogonal relationship. Specifically, the expanded laser first passes through a beam splitter, with targets placed on both sides of the beam splitter. By observing whether the light spot falls uniformly at the target center, it is determined whether the beam splitter is tilted relative to the optical path. If a light spot offset is found, the verticality is restored by fine-tuning the beam splitter's attitude. After completing the BS direction correction, the SLM is mounted on a precision rotating stage with an angle scale. First, the SLM is adjusted to a vertical state, and in this state, it is checked whether the reflected light coincides with the optical axis. If the angle deviates, adjust the rotation center and height to achieve perfect reflection at 0°. Then rotate the SLM to 30°. At this point, the optical path geometry becomes more complex because the reflection direction is no longer along the optical axis, requiring manual adjustment of the camera position to recapture the imaging area. By observing the sharpness and focal position of the diffraction pattern, it can be determined whether the camera has reached the imaging plane consistent with the numerical propagation model. To ensure seamless integration of the deep learning network with the simulation model, it is essential to ensure that the camera accurately captures the off-axis light field generated by the SLM across the entire angular range. Therefore, fine-tuning of the imaging device needs to be performed repeatedly at different oblique angles.

[0031] To enhance the repeatability and verifiability of the system, this application embodiment has uniformly planned the main hardware configuration and strictly recorded the parameters. The reflective LCOS is the core modulation device, and its 3.74μm pixel pitch and 4K resolution enable it to perform well in experiments involving large-angle oblique incidence and high-frequency bandwidth compensation. Since the accuracy and linearity of phase modulation directly affect the reliability of deep learning training data, this application embodiment preprocesses the γ calibration curve of the SLM to ensure that its output phase can be accurately mapped to the phase range in the simulation model.

[0032] Regarding the CMOS camera, this embodiment of the application selects an industrial camera 16 with small pixel size and high dynamic range to meet the need for capturing high-contrast reconstructed images.

[0033] To ensure the stability of the optical path at different experimental stages, this embodiment employs a high-precision rotary stage, enabling the relative position of the industrial camera 16 and the SLM to be adjusted within millimeters. This is extremely important for the reproducibility of experimental data, especially since deep learning model training requires continuous acquisition of a large number of samples. Therefore, the stability of the optical platform directly affects whether the model can learn a stable feature distribution from real data.

[0034] Because this embodiment employs a large-angle oblique incidence structure, the system is far more sensitive to geometric errors and optical corrections than traditional normal incidence configurations. Without addressing camera imaging geometry, optical aberrations, and the non-ideal characteristics of SLM modulation, the reconstructed images obtained from experiments often exhibit multiple distortion effects, making it difficult to accurately reflect the performance differences of the propagation model itself. Therefore, before conducting subsequent angle scanning experiments and algorithm comparison experiments, this embodiment performed geometric calibration and aberration correction on the system.

[0035] like Figure 3 As shown, Figure 3 This is a schematic diagram of geometric calibration in a near-eye display system with large-angle oblique incidence according to an embodiment of the present invention. In terms of geometric calibration, this embodiment uses a dot matrix target in conjunction with the OpenCV toolchain to calibrate the camera imaging model. Specifically, a regularly distributed dot matrix target is placed in the optical path, positioned near the imaging plane corresponding to the SLM reconstructed image. Multiple dot matrix images are captured using an experimental system at different positions and orientations. Then, OpenCV is used to extract sub-pixel feature points of the dot matrix in each image. Based on the multi-view observation results, the camera's intrinsic parameter matrix and distortion coefficients are estimated, and the imaging results are processed to correct distortion. This process effectively eliminates barrel or pincushion distortion caused by the lens, enabling the reconstructed image to establish a more accurate correspondence with the target image coordinate system at the pixel level. This step is particularly important under large-angle oblique incidence conditions because the relationship between the camera and the SLM is no longer a simple direct alignment; any slight tilt will be amplified into significant geometric distortion, which, if not corrected, will interfere with subsequent objective evaluation of image quality.

[0036] Regarding aberration correction, to make the control experiments more representative and convincing, this application's embodiments use the Camera-in-the-Loop (CITL) method for optical correction of the system. For example... Figure 4 As shown, Figure 4This is a flowchart illustrating CITL in a near-eye display system with large-angle oblique incidence according to an embodiment of the present invention. CITL introduces the real optical system into an optimization closed loop: First, the phase distribution to be optimized is loaded onto the SLM, propagated through the actual optical path, and the reconstructed image is acquired by the camera; then, the error between the measured reconstruction result and the desired target image is used as the optimization objective to construct a loss function, and the SLM phase is continuously updated iteratively, so that the reconstruction result observed at the camera gradually approaches the target image. Specifically, the optimization process of CITL can be understood as a "system-level correction" strategy: its update direction does not rely on the complete accuracy of the idealized propagation model, but is directly driven by the output feedback of the real system to adjust the parameters, thereby automatically compensating for system errors introduced by factors such as optical component errors, assembly deviations, non-ideal SLM modulation, camera imaging distortion, and environmental disturbances during the optimization process. Compared with traditional pure numerical simulation schemes, CITL incorporates the camera, SLM, and the behavior of the entire optical path into the same optimization loop, thus often exhibiting stronger robustness and transferability under complex experimental conditions.

[0037] also, Figure 4 The document also provides a comparison between different implementation paths: for example... Figure 4 In (a), iterative optimization based solely on simulation relies on the propagation model for updates, while... Figure 4 (b) illustrates the process of directly acquiring experimental results in a real optical system; furthermore, Figure 4 (c) and (d) in the diagram illustrate a hybrid framework that combines neural networks with the physical propagation process. This framework allows for both end-to-end learning to obtain a better initial phase and the introduction of CITL iterative updates based on the network output, thereby further improving reconstruction quality and system adaptability. In summary, from the perspective of aberration correction, CITL can be considered one of the more mature system-level correction schemes currently available, providing effective assurance for optical path error compensation and result reliability in subsequent oblique incidence experiments.

[0038] It should be noted that the purpose of using CITL in this application embodiment is not to propose a new aberration correction algorithm, but rather to use it as a control group to represent the optimal implementation based on the aberration correction approach. Since the specific aberration correction methods used in the comparative work are not entirely clear, to ensure the fairness of the comparison, this application embodiment chooses to use CITL as an upper limit reference for this type of method, achieving the most comprehensive system correction possible. Thus, when the experimental reconstruction results corrected by CITL are presented side-by-side with the reconstruction results generated based on the EBSF framework, it can be considered that both are performing better under the same hardware conditions, which helps to highlight the advantages of EBSF in large-angle oblique incidence scenes.

[0039] Through the above geometric calibration and aberration correction processing, the embodiments of this application minimize additional interference from the system itself at the experimental level, so that subsequent imaging degradation experiments at different tilt angles and comparative experiments based on EBSF can more realistically reflect the influence of the propagation model and sampling strategy itself, providing a reliable data foundation for the following sections.

[0040] In a preferred example, the SLM (spatial light modulator) GEAE2 is a reflective LCOS phase modulator with a resolution of 4K and a pixel pitch of 3.74 μm. Its purpose is to load a phase hologram to generate an obliquely incident off-axis reconstructed light field. The industrial camera (imaging acquisition) is a FLIR industrial camera (used to acquire reconstructed images for calibration / evaluation). The rotation / angle adjustment mechanism (i.e., the rotary stage in this embodiment) is a Thorlabs RP01( / M) rotary stage. Its purpose is to stabilize the angle of the SLM (or key optical components), ensure that the oblique incident angle setting is repeatable, and reduce errors caused by angle drift.

[0041] Since off-axis / oblique incidence structures are more sensitive to geometric errors, the embodiments of this application adopt a method of alignment followed by rotation for adjustment: Step 1: 0° Alignment (Reference State); First, adjust the SLM to the vertical state (0°); check whether the reflected light coincides with the system optical axis. If not, adjust the position and height of the rotation center to align the reflected light with the optical axis at 0°. This step is used to establish a stable "reference optical path" to ensure that the changes introduced by subsequent rotation mainly come from the angle itself, rather than the initial error.

[0042] Step 2: Rotate to the target oblique incidence angle (e.g., 30°); Rotate the SLM to the target angle (e.g., 30°) using a rotary table; Under oblique incidence conditions, the reflection direction is no longer along the optical axis, so the camera position needs to be manually fine-tuned to recapture the complete imaging area; By observing the sharpness and focal position of the diffraction / reconstruction pattern, determine whether the camera is in an imaging plane consistent with the numerical propagation model; To ensure stable acquisition capability under all angle conditions, fine-tuning needs to be repeated at different oblique angle positions so that the system can stably acquire reconstructed images at each angle.

[0043] In the SLM modulation curve calibration method, namely modulation curve / γ calibration, the gray-scale-phase (or voltage-phase) mapping of the SLM directly affects the actual modulation effect after the hologram is loaded. Therefore, calibration is required to ensure that the "loaded phase" can be correctly presented in the actual system.

[0044] This application embodiment uses software tools provided by the manufacturer (i.e., the specific software in this application embodiment) to perform modulation curve calibration. The core operations are as follows: Step 1: Adjust the minimum and maximum voltage parameters using the manufacturer's software.

[0045] Step 2, the adjustment goal is: when the grayscale is 0 and 255, the display effect captured by the camera is "same color / consistent state" (i.e., consistent endpoints, avoiding endpoint phase misalignment or non-linear expansion).

[0046] Step 3: Completing this step ensures that the phase mapping range is stable, which facilitates the correct mapping of the phase output by the algorithm to the phase range loaded by the hardware, thereby improving the consistency and reproducibility of the experiment.

[0047] Based on the above, the phase endpoints corresponding to 0 and 255 are aligned using the manufacturer's software, making the phase modulation range controllable and repeatable.

[0048] In the camera calibration process, namely the dot target and OpenCV calibration, in order to ensure that the acquired reconstructed image corresponds to the target image in spatial geometry (to reduce distortion and coordinate error), the camera is geometrically calibrated and distortion-reduced.

[0049] This application embodiment uses a dot array target in conjunction with an OpenCV calibration process, as detailed below: Step 1: Place a dot array target in the optical path so that the target is located near the imaging plane corresponding to the reconstructed image.

[0050] Step 2: Use a FLIR industrial camera to capture target images at different positions / attitudes, collecting a total of 5 calibration images.

[0051] Step 3: Use OpenCV to extract the center of the dots in each image.

[0052] Step 4: Solve the mapping matrix based on multiple shooting results.

[0053] Step 5: Use the calibrated mapping matrix to perform distortion correction on subsequent experimental images, thereby improving the accuracy of alignment between the reconstructed image and the target image and ensuring the reliability of evaluation (such as PSNR / SSIM) and comparison experiments.

[0054] This invention employs the above technical solution, utilizing a reflective silicon-based liquid crystal spatial light modulator for loading phase holograms; a light source module for providing coherent illumination; a rotating stage for mounting the reflective silicon-based liquid crystal spatial light modulator and adjusting its tilt angle relative to the incident light axis; an industrial camera for acquiring and reconstructing images; and a processing unit connected to the reflective silicon-based liquid crystal spatial light modulator, the rotating stage, and the industrial camera for performing modulation curve calibration of the reflective silicon-based liquid crystal spatial light modulator, camera geometric calibration, and system debugging. Compared with existing technologies, this invention achieves the following technical effects: significantly improving computational efficiency and accelerating generation capabilities while maintaining high quality, as well as rapidly completing hologram generation.

[0055] An illustrative embodiment of the present invention, such as Figure 5 As shown, Figure 5 This is a flowchart illustrating a method for generating holograms using large-angle oblique incidence off-axis calculation according to an embodiment of the present invention, applied to... Figure 1 The present application provides a method for generating holograms using off-axis calculations for large-angle oblique incidence near-eye display systems, as illustrated in this embodiment: Step S502: Obtain the initial phase distribution of the plane of the reflective silicon-based liquid crystal spatial light modulator, wherein the initial phase distribution has physical resolution; Step S504: Construct a virtual oversampling grid and map the initial phase distribution to an extended high-resolution grid. The sampling interval of the extended high-resolution grid is smaller than the physical sampling interval. Optionally, the virtual oversampling mesh is constructed using nearest-neighbor upsampling, copying the phase value of each physical pixel to fill the corresponding s×s sub-pixel region, where s is the oversampling factor, and the expanded resolution is... ,in This refers to the physical resolution.

[0056] Optionally, average downsampling groups the high-resolution light field distribution into s×s sub-pixel blocks and averages the pixel values ​​within each sub-pixel block to obtain an output pixel.

[0057] Step S506: Construct a modulated light field on the extended high-resolution grid and superimpose the linear phase term corresponding to the oblique incidence. Step S508: The Shifted angular spectrum method is used to calculate the propagation of the modulated light field to obtain the high-resolution light field distribution of the target plane; Step S510: The high-resolution light field distribution of the target plane is mapped back to the physical resolution by average downsampling to obtain the reconstructed image.

[0058] Optionally, the method for generating holograms by large-angle oblique incidence off-axis calculation provided in this application embodiment further includes: calculating the error between the reconstructed image and the target image, updating the initial phase distribution of the spatial light modulator plane through gradient backpropagation, until the convergence condition is met.

[0059] Optionally, the method for generating holograms using large-angle oblique incidence off-axis calculation provided in this application embodiment further includes: constructing a phase prediction neural network, wherein the phase prediction neural network uses U-Net as its backbone architecture, and the phase prediction neural network includes: an encoder and a decoder, the encoder extracts multi-scale features through multi-layer convolution and downsampling operations, and the decoder restores spatial resolution and fuses features from the encoding stage through upsampling and skip connections; inputting the amplitude distribution of the target image into the phase prediction neural network, and outputting an initial phase distribution after a single forward propagation; using the mean square error between the amplitude distributions of the reconstructed image and the target image as a loss function, updating the parameters of the phase prediction neural network using an error backpropagation algorithm, and completing network training; inputting the target image to be generated into the trained phase prediction neural network, and generating the corresponding hologram phase distribution after a single forward propagation.

[0060] Further, optionally, U-Net includes 8 downsampling and 8 upsampling operations, with the encoder and decoder being symmetrical; downsampling is implemented using 4×4 convolutions with a stride of 2, and upsampling is implemented using 4×4 transposed convolutions with a stride of 2; the activation function of the downsampling path is LeakyReLU with a slope of 0.2 on the negative half-axis, and the activation functions of the upsampling path and the output path are ReLU; the output phase is constrained to the interval [-π, π] by hard constraints; the network uses batch normalization BatchNorm2d.

[0061] Specifically, the fast generation network in this application embodiment adopts a U-Net-like network: U-Net layer count (depth): It adopts a multi-scale structure with 8 downsampling and 8 upsampling (i.e., U-Net depth is 8 layers), the encoder and decoder are symmetrical, and there are skip connections to preserve detailed information.

[0062] Kernel size: 3×3 convolutions are mainly used for main feature extraction; 4×4 convolutions with a stride of 2 are used for downsampling to halve the size; 4×4 transposed convolutions with a stride of 2 are used for upsampling to restore resolution (bilinear / nearest neighbor upsampling + convolution is also supported). Activation functions: The downsampling path mainly uses LeakyReLU (slope of 0.2 on the negative half axis) to ensure more stable training; the upsampling and output paths mainly use ReLU; the output phase is hard-limited to [-π, π] to ensure numerical stability and avoid phase divergence.

[0063] Optionally, parameter updates use the Adam optimizer, which sets an initial learning rate and employs an exponential decay strategy to bring the phase prediction neural network to convergence.

[0064] In summary, combining steps S502 to S510, the method for generating holograms using off-axis calculations for large-angle oblique incidence provided in this application embodiment requires the holograms to be continuously updated at a speed close to the video frame rate in actual near-eye display applications to support dynamic content display and interactive experiences. Therefore, it is necessary to achieve rapid or even fast generation of holograms while ensuring the physical consistency of oblique incidence propagation. From a system composition perspective, the HoloNet framework can be divided into three tightly coupled modules: a target image encoding module, a phase prediction module, and a physical propagation verification module. First, the target image is typically input into the network in the form of amplitude or light intensity and is mapped to a multi-scale feature space during the encoding stage. Subsequently, the phase prediction module generates a phase distribution consistent with the SLM resolution based on these features. Finally, the predicted phase must undergo forward computation by the physical propagation model, and the error between the propagation result and the target image constitutes the supervision signal during the training stage.

[0065] This application embodiment reorganizes and reimplements the overall generation process to address the physical characteristics of near-eye display systems with large-angle oblique incidence. In terms of network structure and training process design, this application embodiment adopts a basic paradigm similar to existing rapid generation methods, and, considering the propagation characteristics under oblique incidence conditions, makes corresponding adjustments to the generation process at the physical modeling level.

[0066] Figure 6 This is a schematic diagram of a training framework for fast generation of large-angle off-axis holograms based on U-Net in a method for generating holograms by large-angle oblique incidence off-axis projection according to an embodiment of the present invention. Figure 6As shown, the overall process is as follows: First, the amplitude distribution of the target image is used as the network input and introduced into a phase prediction network based on a U-Net structure. This network learns the nonlinear mapping relationship from the target imaging plane to the SLM plane, outputting the phase distribution on the SLM plane. This phase only encodes the optical field phase information, used to achieve the desired far-field reconstruction effect under a given amplitude constraint. Subsequently, to accurately describe the optical propagation characteristics under large-angle oblique incidence conditions, the predicted SLM phase is mapped to an extended high-resolution computational grid to meet the sampling requirements of the spectral bandwidth during oblique incidence propagation. Based on this, a linear phase term corresponding to the incident angle is further superimposed to characterize the phase tilt effect of the obliquely incident wavefront on the SLM plane, thereby achieving a spectral center shift in the frequency domain. Finally, Shifted-ASM is used to calculate the oblique incidence propagation of the modulated complex amplitude optical field, obtaining the reconstructed optical field distribution of the target imaging plane. By measuring the error between the reconstructed amplitude and the target image, an end-to-end training loss function is constructed, thereby achieving inverse optimization of the network parameters.

[0067] This design allows the network to explicitly perceive spectral shifts and bandwidth limitations under oblique incidence conditions during the training phase, thereby automatically compensating for related degradation effects during the prediction phase.

[0068] In terms of network structure and parameter settings, U-Net is used as the backbone structure, which includes two parts: encoder and decoder. The encoder gradually extracts multi-scale features through multi-layer convolution and downsampling operations. The decoder restores spatial resolution through upsampling and skip connections, and fuses the feature information from the encoding stage.

[0069] Table 1 shows the parameters for rapid network generation. The network input is the normalized amplitude distribution of the target image, and the output is a single-channel phase map consistent with the SLM resolution. The phase output is constrained to the interval [0, 2π) through modulo operation to meet the modulation range requirements of the actual SLM.

[0070] During training, the phase hologram output by the network is not directly used for loss calculation, but needs to be verified through a complete physical propagation process. The phase predicted by the network is first mapped to an extended frequency domain sampling grid to meet the spectral bandwidth requirements under oblique incidence conditions. Subsequently, a corresponding linear phase term is loaded according to the system's incident angle parameters to achieve a global translation of the angular spectrum in the frequency domain. Based on this, Shifted-ASM is used to complete numerical propagation to obtain the complex amplitude distribution of the observation plane. The propagation result is then downsampled and mapped back to the physical resolution, and error calculation is performed with the target image to construct the supervision signal for the training phase.

[0071] By embedding the aforementioned propagation link as a differentiable module into the network training process, the error signal can propagate backward along the complete physical path to the network parameters, thereby achieving end-to-end optimization. This design enables the network to explicitly consider spectral shift and sampling constraints under oblique incidence conditions while learning phase mapping relationships, effectively improving the stability of the training process and the physical consistency of the generated results.

[0072] During training, MSE (Mean Separation of Optical Spectrum) was used as the loss function to measure the difference between the reconstructed light intensity distribution after propagation and the target image. This loss function directly reflects the pixel-level deviation of the reconstruction result and exhibits good stability in numerical computation, making it suitable for the hologram generation task in this study. Network parameter updates were implemented using the Adam optimizer, with an initial learning rate set to 1×10⁻⁶. 4 In the later stages of training, an exponential decay strategy is employed to promote model convergence. No additional regularization terms or perceptual loss functions are introduced during training to avoid introducing prior constraints unrelated to the physical propagation model, thereby ensuring that the network primarily optimizes for reconstruction errors under oblique incidence propagation conditions.

[0073] Among them, the exponential decay strategy causes the learning rate to decay exponentially as the number of steps increases. ; This is the learning rate for the current step; The initial learning rate (i.e., 1 × 10) 4 ); This is the attenuation coefficient (usually set between 0.9 and 0.99). This is the stage in which training is carried out.

[0074] The purpose of the initial learning rate in the embodiments of this application is to accelerate training and stabilize results.

[0075] This invention employs the above technical solution, obtaining the initial phase distribution of the plane of a reflective silicon-based liquid crystal spatial light modulator, which has physical resolution; constructing a virtual oversampling grid to map the initial phase distribution to an extended high-resolution grid, the sampling interval of which is smaller than the physical sampling interval; constructing a modulation light field on the extended high-resolution grid and superimposing the linear phase term corresponding to oblique incidence; using the Shifted angular spectrum method to calculate the propagation of the modulation light field to obtain the high-resolution light field distribution of the target plane; and mapping the high-resolution light field distribution of the target plane back to the physical resolution through average downsampling to obtain the reconstructed image. Compared with existing technologies, this invention has the following technical effects: significantly improving computational efficiency and accelerating generation capabilities while maintaining high quality, as well as rapidly completing hologram generation.

[0076] The above description is merely a preferred embodiment of the present invention and does not limit the implementation and protection scope of the present invention. Those skilled in the art should realize that any equivalent substitutions and obvious changes made based on the description and illustrations of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for generating holograms using large-angle oblique incidence off-axis calculation, characterized in that, include: The initial phase distribution of the plane of a reflective silicon-based liquid crystal spatial light modulator is obtained, wherein the initial phase distribution has physical resolution; A virtual oversampling grid is constructed to map the initial phase distribution to an extended high-resolution grid, wherein the sampling interval of the extended high-resolution grid is smaller than the physical sampling interval; A modulated optical field is constructed on the extended high-resolution grid, and a linear phase term corresponding to oblique incidence is superimposed. The propagation of the modulated light field is calculated using the Shifted angular spectrum method to obtain the high-resolution light field distribution on the target plane; The high-resolution light field distribution of the target plane is mapped back to the physical resolution through average downsampling to obtain the reconstructed image.

2. The method for generating holograms using large-angle oblique incidence off-axis calculation according to claim 1, characterized in that, The virtual oversampling mesh is constructed using a nearest-neighbor upsampling method, copying and filling the phase value of each physical pixel into the corresponding s×s sub-pixel region, where s is the oversampling factor, and the expanded resolution is... ,in This refers to the physical resolution.

3. The method for generating holograms using large-angle oblique incidence off-axis calculation according to claim 1, characterized in that, The average downsampling grouped the high-resolution light field distribution into s×s sub-pixel blocks and averaged the pixel values ​​within each sub-pixel block to obtain an output pixel.

4. The method for generating holograms by large-angle oblique incidence off-axis calculation according to claim 1, characterized in that, The method further includes: The error between the reconstructed image and the target image is calculated, and the initial phase distribution of the spatial light modulator plane is updated through gradient backpropagation until the convergence condition is met.

5. The method for generating holograms by large-angle oblique incidence off-axis calculation according to any one of claims 1 to 4, characterized in that, The method further includes: A phase prediction neural network is constructed, wherein the phase prediction neural network uses U-Net as the backbone architecture and includes an encoder and a decoder. The encoder extracts multi-scale features through multi-layer convolution and downsampling operations, and the decoder restores spatial resolution and fuses features from the encoding stage through upsampling and skip connections. The amplitude distribution of the target image is input into the phase prediction neural network, and the initial phase distribution is output after a single forward propagation. The mean square error between the amplitude distribution of the reconstructed image and the target image is used as the loss function, and the parameters of the phase prediction neural network are updated using the error backpropagation algorithm to complete the network training. The target image to be generated is input into the trained phase prediction neural network, and the corresponding holographic phase distribution is generated through a single forward propagation.

6. The method for generating holograms using large-angle oblique incidence off-axis calculation according to claim 5, characterized in that, The U-Net consists of 8 downsampling passes and 8 upsampling passes, with a symmetrical encoder and decoder. Downsampling is implemented using 4×4 convolutions with a stride of 2, and upsampling is implemented using 4×4 transposed convolutions with a stride of 2. The activation function for the downsampling path is LeakyReLU with a slope of 0.2 on the negative half-axis, while the activation functions for the upsampling path and the output path are ReLU. The output phase is hard-constrained within the interval [-π, π]. The network uses batch normalization (BatchNorm2d).

7. The method for generating holograms using large-angle oblique incidence off-axis calculation according to claim 5, characterized in that, The parameter update uses the Adam optimizer, which sets an initial learning rate and employs an exponential decay strategy to bring the phase prediction neural network to converge.

8. A near-eye display system oriented towards large-angle oblique incidence, characterized in that, include: The components include a reflective silicon-based liquid crystal spatial light modulator, a light source module, a rotating stage, an industrial camera, and a processing unit. The reflective silicon-based liquid crystal spatial light modulator is used to load a phase hologram; The light source module is used to provide coherent illumination; The rotating stage is used to mount the reflective silicon-based liquid crystal spatial light modulator and adjust the tilt angle of the reflective silicon-based liquid crystal spatial light modulator relative to the incident optical axis. The industrial camera is used to acquire and reconstruct images; The processing unit is connected to the reflective silicon-based liquid crystal spatial light modulator, the rotary table, and the industrial camera, and is used to perform modulation curve calibration of the reflective silicon-based liquid crystal spatial light modulator, camera geometric calibration, and system debugging.

9. The near-eye display system oriented towards large-angle oblique incidence as described in claim 8, characterized in that, The modulation curve calibration of the reflective silicon-based liquid crystal spatial light modulator includes: adjusting the minimum and maximum voltage parameters through specific software so that the display effects corresponding to grayscale 0 and 255 are in the same color state. The camera geometric calibration includes: placing a dot array target in the optical path, capturing target images at different positions and orientations, extracting the dot centers using OpenCV and solving the mapping matrix, and performing distortion correction processing on the subsequently acquired reconstructed images. The system debugging includes: adjusting the reflective silicon-based liquid crystal spatial light modulator to a vertical state, checking that the reflected light coincides with the optical axis, rotating the reflective silicon-based liquid crystal spatial light modulator to the target oblique incidence angle via the rotary table, fine-tuning the position of the industrial camera, and determining the imaging plane by observing the clarity and focal position of the diffraction pattern.

Images