Intelligent computer-generated hologram generation system and method thereof
By explicitly embedding the light propagation characteristics into the frequency domain enhancement and optical imaging module, the problems of insufficient frequency domain information and distortion in existing hologram generation methods are solved, and high-quality hologram generation is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- TIANJIN UNIV
- Filing Date
- 2026-02-04
- Publication Date
- 2026-04-28
AI Technical Summary
Existing deep learning-based computational hologram generation methods lack the ability to extract information in the frequency domain and fail to fully utilize the effective physical constraints of optical imaging models, resulting in distortion and blurring in the generated holograms.
An intelligent computational hologram generation system is adopted. Through a frequency-domain enhanced phase retrieval module and an optical imaging module, the spatial features are enhanced by frequency domain features, and the light propagation characteristics are explicitly embedded to construct a hologram mapping model that conforms to the laws of optical imaging.
It improves the quality of hologram generation, enhances the ability to extract frequency domain information, and makes the generated holograms more consistent with the actual imaging process, thereby improving the accuracy and clarity of 3D image reconstruction.
Smart Images
Figure CN121934338A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of 3D display and image processing, and specifically relates to an intelligent computational hologram generation system and method. Background Technology
[0002] With the development of multimedia technology, the demand for high-quality 3D displays is becoming increasingly urgent. Traditional 3D display technology has inherent limitations such as visual convergence conflict, which can easily cause eye fatigue for viewers. Holographic 3D displays utilize the principles of light interference and diffraction to create continuous viewing parallax, effectively reducing eye fatigue and thus achieving a realistic 3D display effect.
[0003] Computational holography directly encodes the amplitude and phase information of objects through numerical computation, enabling high-precision reconstruction of 3D scenes with greater flexibility and practicality. In recent years, thanks to the significant advantages of deep learning in fitting complex nonlinear mappings, deep learning-driven computational hologram generation methods have received widespread attention. For example, Shi et al. proposed a physics-guided tensor holographic network that models light propagation as a multi-distance learnable convolutional kernel group, achieving real-time generation of 2K holograms while ensuring the interpretability of the physical process. Wu et al. proposed an autoencoder-based computational hologram generation network that embeds the physical diffraction propagation mechanism into the autoencoder, learning the latent encoding of phase holograms in a self-supervised manner, achieving real-time generation of 4K holograms. Zheng et al. employed a hybrid frequency and spatial domain loss constraint network to improve image detail texture and color contrast.
[0004] However, existing deep learning-based computational hologram generation methods still lack sufficient ability to extract frequency domain information, making it difficult for the network to accurately capture global contextual information. Furthermore, existing methods fail to fully exploit the effective physical constraints in optical imaging models, resulting in distortion and blurring in the generated holograms. Therefore, effectively utilizing frequency domain enhancement and physical constraints to improve the model's ability to learn global nonlinear mapping relationships and construct a hologram mapping model that both conforms to the laws of optical imaging and possesses strong expressive power is of significant research importance. Summary of the Invention
[0005] To improve the extraction capability of frequency domain information and explore the effective physical constraints of optical imaging models in the computational hologram generation process, this invention proposes an intelligent computational hologram generation system and method. In the frequency-domain enhanced phase retrieval module, the global context information contained in the frequency domain features is used to enhance the spatial domain features; in the optical imaging module, multiple sets of position-related convolution kernels are dynamically generated based on the point spread function, explicitly embedding the light propagation characteristics, making the generated phase hologram more consistent with the actual imaging process and effectively improving the quality of 3D image reconstruction.
[0006] To overcome the shortcomings of the prior art, the present invention adopts the following technical solution: A smart computational hologram generation system, comprising: an image feature extraction module, a frequency domain enhanced phase retrieval module, a first optical imaging module, and a second optical imaging module, wherein: The image feature extraction module obtains frequency domain features and spatial domain features of different frequency bands by performing spectrum division calculation on the input image; The frequency-domain enhanced phase recovery module obtains the complex amplitude components of the image by enhancing the spatial feature representation through the frequency domain structure information of the image amplitude components; The first optical imaging module encodes the complex amplitude hologram by calculating the point spread function corresponding to the system; The second optical imaging module performs variable convolution processing on the complex amplitude hologram to obtain a phase hologram.
[0007] Furthermore, the image feature extraction module obtains frequency domain features and spatial domain features of different frequency bands by performing spectral division calculations on the input image, including: The amplitude distribution of the input image is subjected to Fourier transform to obtain a spectrum, and then the spectrum is filtered using bandpass filters with different cutoff frequencies to extract the amplitude distribution. k Zhang has spectrum diagrams with different frequency domain characteristics; By using inverse Fast Fourier Transform (IFFT) to restore the spatial distribution of spectrograms from different frequency bands, the image amplitude components are obtained by mapping and aligning frequency domain information to spatial modes at multiple scales. .
[0008] Furthermore, the frequency-domain enhanced phase retrieval module obtains the image complex amplitude components by enhancing the spatial domain feature representation through the frequency domain structure information of the image amplitude components, including: Based on the amplitude distribution of the input image, multi-scale features are projected onto a high-level semantic space using a multilayer perceptron. Then, average pooling is used to calculate weighted masks for different frequency bands to perform weighted modulation of the spectrogram, obtaining the multi-scale features of the spectrogram. ; Feature enhancement is achieved through spatial attention and channel attention mechanisms. The attention weights corresponding to each channel of the spectrogram are calculated, and adaptive adjustments are made based on the importance of features in different channels. This yields a collaborative representation of frequency domain features and spatial domain features in a shared semantic space, from which enhanced spatial domain features are extracted. ; spatial features The data is fed into a complex convolutional network to gradually extract deep features that are highly correlated with the phase distribution and obtain more accurate phase estimation results. The complex amplitude component of the image is constructed by combining the phase distribution with known amplitude information, and the complex amplitude component is decomposed into plane wave components with different propagation directions to apply the corresponding propagation phase factor. Finally, the complex amplitude hologram is synthesized by inverse Fourier transform.
[0009] Furthermore, the first optical imaging module encodes the complex amplitude hologram using the point spread function corresponding to the target optical system; The point spread function is calculated based on the physical parameters of the target optical system and according to the diffraction theory of monochromatic scalar light waves in free space. The physical parameters of the target optical system include the light source wavelength, propagation distance, and spatial sampling pixel size.
[0010] Furthermore, the process by which the second optical imaging module performs variable convolution processing on the complex amplitude hologram to obtain a phase hologram includes: Multi-scale convolution kernels are generated from complex amplitude holograms using lightweight subnetworks; Spatial adaptive convolution kernels are generated by simulating spatial variations in optics using multi-scale convolution kernels in physically constrained layers. Phase holograms are generated by adaptively weighting and fusing the spatial adaptive convolution results using a gated activation function.
[0011] This invention can also be implemented using the following technical solutions: A method for generating intelligent computational holograms includes the following steps: The frequency domain features and spatial domain features of the input image are calculated to obtain the image amplitude components of different frequency bands according to the following formula; in, Representing bandpass filtered images of different frequency bands. and These represent the Fast Fourier Transform and the Inverse Fast Fourier Transform, respectively. The complex amplitude components of an image are obtained by enhancing the spatial domain feature representation with frequency domain structure information of the image amplitude components, including: Based on the amplitude distribution of the input image, multi-scale features are projected onto a high-level semantic space using a multilayer perceptron. Then, average pooling is used to calculate weighted masks for different frequency bands to perform weighted modulation of the spectrogram, obtaining the multi-scale features of the spectrogram. ; in, Weight masks representing different frequency bands, This represents a multilayer perceptron, used to adaptively project an input image into a higher-level semantic space. Indicates average pooling; Feature enhancement is achieved through spatial attention and channel attention mechanisms. The attention weights corresponding to each channel of the spectrogram are calculated, and adaptive adjustments are made based on the importance of features in different channels. This yields a collaborative representation of frequency domain features and spatial domain features in a shared semantic space, from which enhanced spatial domain features are extracted. ; in, This indicates the enhanced spatial characteristics. This represents weighted multi-scale features. This represents the sigmoid activation function. Indicates channel attention. Indicates spatial attention. This represents the element-wise dot product operation. spatial features The data is fed into a complex convolutional network to gradually extract deep features that are highly correlated with the phase distribution and obtain more accurate phase estimation results. The phase distribution is combined with known amplitude information to construct complex amplitude components of the image. The complex amplitude components are decomposed into plane wave components with different propagation directions to apply corresponding propagation phase factors, and complex amplitude holograms are synthesized through inverse Fourier transform.
[0012] The first optical imaging module encodes the complex amplitude hologram using the point spread function corresponding to the target optical system; it collects parameters such as the wavelength of the light source, the pixel size of the spatial light modulator, and the target display distance; it calculates the point spread function corresponding to the target optical system based on the diffraction theory of monochromatic scalar light waves in free space; and it explicitly embeds the point spread function into the hologram encoding process to perform weighted modulation on the original complex amplitude hologram. The second optical imaging module performs variable convolution processing on the complex amplitude hologram to obtain a phase hologram; in, This represents the convolution kernel predictor. This represents a pre-trained downsampling network. This represents the calculated point spread function. Represents the hyperbolic tangent function. and These represent the corresponding convolution kernel and gating activation function, respectively. Represents convolution of a linear system. This represents the generated hologram; the mechanism can dynamically adjust the weights of features at each scale based on local content, effectively improving the model's ability to express complex diffraction scenes; The training module extracts phase hologram data and fuses frequency and spatial domain features. It then uses spatial adaptive convolution to perform differentiated reconstruction based on the optical propagation characteristics of different regions, while optimizing the training system to output high-quality holograms.
[0013] Beneficial effects 1. This invention proposes an intelligent computational hologram generation system and method. The method enhances the expressive power of key frequency bands and explicitly models the propagation characteristics of light in the optical imaging model during the hologram encoding process, thereby effectively improving the generation quality of holograms.
[0014] 2. This invention constructs a frequency domain enhanced phase recovery module. This module extracts key frequency domain features by dividing frequency bands and enhances spatial domain features by utilizing the global context information contained in the frequency domain features, so that the estimated phase distribution is closer to the spectral characteristics of the real light field.
[0015] 3. This invention dynamically generates multiple sets of position-related convolution kernels based on the point spread function, explicitly embeds the light propagation characteristics, and guides the spatial adaptive generation of convolution kernel parameters, making the feature extraction process more consistent with the real light propagation law. Attached Figure Description
[0016] Figure 1 This is a flowchart of a method for generating intelligent computational holograms according to the present invention. Detailed Implementation
[0017] The following examples illustrate the specific implementation of the intelligent computational hologram generation method of this invention.
[0018] 1. Construct an image feature extraction module The amplitude distribution of the input image is represented as ,in This represents the image resolution. To capture global structural information, a Fast Fourier Transform is performed on the amplitude distribution of the input image to obtain a spectrogram. Then, bandpass filters with different cutoff frequencies are used to filter the spectrogram, thereby extracting the relevant information. k Zhang generates spectrograms with different frequency domain characteristics. Given the significant modal differences between frequency domain and spatial domain characteristics, an inverse fast Fourier transform is used to restore the spatial distribution of the spectrograms from different frequency bands, thereby achieving the mapping and alignment of frequency domain information to spatial modes at multiple scales. The calculation formula for the above process can be expressed as: in, Representing bandpass filtered images of different frequency bands. and These represent the Fast Fourier Transform and the Inverse Fast Fourier Transform, respectively. The formula for a bandpass filter with different cutoff frequencies can be expressed as: in, and Belongs to a predefined set of frequency thresholds These thresholds are used to segment the frequency domain features.
[0019] 2. Construct a frequency-domain enhanced phase recovery module First, spectrograms from different frequency bands are concatenated along the channel dimension to obtain multi-scale feature representations. Based on the amplitude distribution of the input image, a multilayer perceptron is used to project the multi-scale features onto a higher-level semantic space. Average pooling is then used to calculate weight masks for different frequency bands, which are subsequently weighted and modulated to achieve adaptive balancing of frequency band components, thereby highlighting frequency domain information that is more effective for phase retrieval. The calculation formula for the above process can be expressed as: in, Weight masks representing different frequency bands, This represents a multilayer perceptron, used to adaptively project an input image into a higher-level semantic space. This indicates average pooling.
[0020] Based on weighted multi-scale feature representation, feature enhancement is achieved through spatial attention and channel attention. Attention weights for each channel are calculated, and adaptive fusion is performed according to the importance of features from different channels. This enhances the model's ability to perceive key features, effectively aggregates contextual information, and promotes the collaborative expression of frequency domain features and spatial domain features in a shared semantic space. The calculation formula for the above process can be expressed as: in, This indicates the enhanced spatial characteristics. This represents weighted multi-scale features. This represents the sigmoid activation function. Indicates channel attention. Indicates spatial attention. This indicates an element-wise dot product operation.
[0021] Based on this, the enhanced spatial characteristics The signal is fed into a complex convolutional network to progressively extract deep features highly correlated with the phase distribution, resulting in more accurate phase estimation. Subsequently, this phase distribution is combined with known amplitude information to construct a complex amplitude representation. ,in For amplitude components, The phase components are predicted. Finally, the complex amplitude distribution of the input plane is decomposed into plane wave components with different propagation directions, and a corresponding propagation phase factor is applied to each component. The complex amplitude hologram is then synthesized by inverse Fourier transform.
[0022] The frequency-domain enhanced phase recovery module explicitly enhances spatial features through frequency domain features, enabling the model to simultaneously capture spatial dynamic changes and frequency domain structural information, achieving accurate reconstruction of phase distribution, effectively preserving all information of the light field, and ensuring the accuracy and physical rationality of the generated hologram.
[0023] 3. Construct the first optical imaging module Considering that complex amplitude holograms are difficult to directly load into practical optical systems, they need to be encoded as phase holograms to adapt to optical devices. In order to fully preserve the key structure of the original light field and adapt to the physical characteristics of real imaging systems during the encoding process, a first optical imaging module is constructed.
[0024] The first optical imaging module uses the point spread function as the mathematical representation of the optical imaging model and explicitly embeds the hologram encoding process, enabling it to actively capture the spatial response differences of the optical system.
[0025] Since the angular spectral propagation method rigorously solves the Rayleigh-Sommerfeld diffraction integral, the point spread function corresponding to the light propagation process can be accurately calculated based on actual light parameters (such as wavelength, propagation distance, and pixel size), and used as a mathematical representation of the optical imaging model. In this invention, Rayleigh-Sommerfeld is represented as the Rayleigh-Sommerfeld diffraction integral.
[0026] 4. Construct the second optical imaging module To effectively compensate for the spatial response non-uniformity of the optical system, key structural information of the original light field is preserved during the encoding stage, and a second optical imaging module is constructed.
[0027] The second optical imaging module inputs a point spread function into a lightweight sub-network to generate multi-scale, spatially adaptive convolutional kernels. These kernel systems effectively simulate the non-uniform modulation effect in light propagation. Finally, the multi-scale convolution results are adaptively fused using a gated activation function to generate a phase hologram. The calculation formulas for the above process are as follows: in, This represents the convolution kernel predictor. This represents a pre-trained downsampling network. This represents the calculated point spread function. Represents the hyperbolic tangent function. and These represent the corresponding convolution kernel and gating activation function, respectively. Represents convolution of a linear system. This represents the generated hologram. This mechanism can dynamically adjust the weights of features at each scale based on local content, effectively improving the model's ability to represent complex diffraction scenes.
[0028] 5. Training an intelligent computational hologram generation network The intelligent computational hologram generation network proposed in this invention includes an image feature extraction module, a frequency-domain enhanced phase retrieval module, a first optical imaging module, and a second optical imaging module. During training, the network effectively utilizes the spatial response characteristics of the optical system to perform differentiated reconstruction of light distribution in different regions, while simultaneously... Loss optimization training process.
Claims
1. An intelligent computational hologram generation system, characterized in that, The hologram generation system includes: an image feature extraction module, a frequency-domain enhanced phase retrieval module, a first optical imaging module, and a second optical imaging module, wherein: The image feature extraction module obtains frequency domain features and spatial domain features of different frequency bands by performing spectrum division calculation on the input image; The frequency-domain enhanced phase recovery module obtains the complex amplitude components of the image by enhancing the spatial feature representation through the frequency domain structure information of the image amplitude components; The first optical imaging module encodes the complex amplitude hologram by calculating the point spread function corresponding to the system; The second optical imaging module performs position-variable convolution processing on the complex amplitude hologram to obtain a phase hologram.
2. The intelligent computational hologram generation system according to claim 1, characterized in that, The image feature extraction module obtains frequency domain features and spatial domain features of different frequency bands by performing spectral division calculations on the input image, including: The amplitude distribution of the input image is subjected to Fourier transform to obtain a spectrum, and then the spectrum is filtered using bandpass filters with different cutoff frequencies to extract the amplitude distribution. k Zhang has spectrum diagrams with different frequency domain characteristics; By using inverse fast Fourier transform, the spectrograms of different frequency bands are restored to their spatial distribution, thereby achieving the mapping and alignment of frequency domain information to spatial modes at multiple scales and obtaining the image amplitude components. .
3. The intelligent computational hologram generation system according to claim 1, characterized in that, The frequency-domain enhanced phase retrieval module obtains the complex amplitude components of the image by enhancing the spatial feature representation through the frequency domain structure information of the image amplitude components. This process includes: Based on the amplitude distribution of the input image, multi-scale features are projected onto a high-level semantic space using a multilayer perceptron. Then, average pooling is used to calculate weighted masks for different frequency bands to perform weighted modulation of the spectrogram, thereby obtaining the multi-scale features of the spectrogram. ; Feature enhancement is achieved through spatial attention and channel attention mechanisms. The attention weights corresponding to each channel of the spectrogram are calculated, and adaptive fusion is performed based on the importance of features from different channels to obtain the collaborative representation of frequency domain features and spatial domain features in a shared semantic space. Enhanced spatial domain features are then extracted from this representation. ; Enhanced spatial characteristics The data is fed into a complex convolutional network to gradually extract deep features that are highly correlated with the phase distribution and obtain more accurate phase estimation results. The complex amplitude component of the image is constructed by combining the phase distribution with the known amplitude information. The complex amplitude component is then decomposed into plane wave components with different propagation directions to apply the corresponding propagation phase factor. Finally, the complex amplitude hologram is synthesized by inverse Fourier transform.
4. The intelligent computational hologram generation system according to claim 1, characterized in that, The first optical imaging module encodes the complex amplitude hologram using the point spread function corresponding to the target optical system; The point spread function is calculated based on the physical parameters of the target optical system and according to the diffraction theory of monochromatic scalar light waves in free space. The physical parameters of the target optical system include the light source wavelength, propagation distance, and spatial sampling pixel size.
5. The intelligent computational hologram generation system according to claim 1, characterized in that, The process by which the second optical imaging module performs position-variable convolution processing on the complex amplitude hologram to obtain a phase hologram includes: Multi-scale convolution kernels are generated from complex amplitude holograms using lightweight subnetworks; Spatial adaptive convolution kernels are generated by simulating spatial variations in optics using multi-scale convolution kernels in physically constrained layers. Phase holograms are generated by adaptively weighting and fusing the spatial adaptive convolution results using a gated activation function.
6. A method for generating intelligent computational holograms, characterized in that, The method is based on the system described in claims 1-5 and includes the following steps: The frequency domain features and spatial domain features of the input image are calculated to obtain the image amplitude components of different frequency bands according to the following formula; in, This represents the image amplitude components at different frequency bands. and These represent the Fast Fourier Transform and the Inverse Fast Fourier Transform, respectively. The complex amplitude components of an image are obtained by enhancing the spatial domain feature representation with frequency domain structure information of the image amplitude components, including: Based on the amplitude distribution of the input image, multi-scale features are projected onto a high-level semantic space using a multilayer perceptron. Then, average pooling is used to calculate weighted masks for different frequency bands to perform weighted modulation of the spectrogram, obtaining the multi-scale features of the spectrogram. ; in, Weight masks representing different frequency bands, This represents a multilayer perceptron, used to adaptively project an input image into a higher-level semantic space. Indicates average pooling; Feature enhancement is achieved through spatial attention and channel attention mechanisms. The attention weights corresponding to each channel of the spectrogram are calculated, and adaptive fusion is performed based on the importance of features from different channels to obtain the collaborative representation of frequency domain features and spatial domain features in a shared semantic space. Enhanced spatial domain features are then extracted from this representation. ; in, This indicates the enhanced spatial characteristics. This represents weighted multi-scale features. This represents the sigmoid activation function. Indicates channel attention. Indicates spatial attention. This represents the element-wise dot product operation. Enhanced spatial features The data is fed into a complex convolutional network to progressively extract deep features that are highly correlated with the phase distribution and obtain more accurate phase estimation results. The phase distribution is combined with known amplitude information to construct complex amplitude components of the image. The complex amplitude components are decomposed into plane wave components with different propagation directions to apply corresponding propagation phase factors, and complex amplitude holograms are synthesized through inverse Fourier transform. The first optical imaging module encodes the complex amplitude hologram using the point spread function corresponding to the target optical system; calculates the point spread function corresponding to the target optical system based on the diffraction theory of monochromatic scalar light waves in free space; and explicitly embeds the point spread function into the hologram encoding process to perform weighted modulation on the original complex amplitude hologram. The second optical imaging module performs variable convolution processing on the complex amplitude hologram to obtain a phase hologram; in, This represents the convolution kernel predictor. This represents a pre-trained downsampling network. This represents the calculated point spread function. Represents the hyperbolic tangent function. and These represent the corresponding convolution kernel and gating activation function, respectively. Represents convolution of a linear system. This represents the generated hologram.