Diffraction element optimization and lookup table pre-storage-based hologram rapid generation system and method

Abstract

The invention discloses a system and a method for rapidly generating a hologram based on diffraction element optimization and lookup table pre-storage, and relates to the technical field of computer-generated holography, and the method comprises the steps: constructing a depth-check lookup table through preprocessing, decomposing a point spread function into a basic diffraction kernel strongly dependent on depth and a modulation function weakly dependent on a transverse position, and pre-storing the basic diffraction kernel and the modulation function; after receiving three-dimensional scene data, adaptively matching a key depth layer and a kernel function, and segmenting the key depth layer and the kernel function into corresponding binary mask layers; generating a contribution hologram of each depth layer through GPU parallel convolution, and performing coherent superposition after modulation and fine tuning to obtain an initial hologram; the system comprises a preprocessing module, a data receiving module and the like to achieve the process, the generation speed and quality are considered, and the system adapts to AR / VR near-to-eye display, biological microscopic imaging and other scenes.

Description

Technical Field

[0001] This invention relates to the field of computational holography, and more specifically, to a system and method for rapid hologram generation based on diffraction element optimization and lookup table pre-storage. Background Technology

[0002] Holographic dynamic reconstruction of small field-of-view, high depth-of-field scenes is a core technological barrier in fields such as AR / VR near-eye interaction and biological microscopy. The coupled demands for generation rate and depth fidelity have led traditional solutions into an irreconcilable contradiction between the rigor of physical modeling and the efficiency of engineering calculations. The classic angular spectrum method requires performing a complete diffraction integral operation on each object point in the 3D scene. In high depth-of-field scenes, the number of discrete object points increases geometrically, resulting in a single-frame hologram generation time of over 180ms, which is completely unsuitable for the real-time requirements of dynamic interactive scenes. Although some studies have simplified the diffraction calculation process by dimensionality reduction, this leads to an increase in depth layering errors, directly resulting in the loss of the cell-level depth resolution required for biological microscopy.

[0003] While existing deep learning-driven generation schemes have improved the single-frame generation rate to the millisecond level, they suffer from significant limitations in scene generalization. Model training relies on large-scale labeled holographic datasets, which are not well-suited for niche scenarios such as biological tissue imaging. Furthermore, the phase physical constraints of wave optics are ignored during training, and the generated holograms are prone to causing visual fatigue and dizziness when displayed near the eye. As the core diffraction metamodel connecting wave optics theory and holographic computation, the traditional design of the point spread function has not fully explored the dominant role of the depth dimension and has not performed separable approximate decomposition, resulting in a kernel function storage size exceeding 12GB and a low cache hit rate during the call process, further exacerbating the computational efficiency bottleneck.

[0004] The speckle noise problem caused by high coherence illumination has put existing solutions in a quality-rate dilemma: iterative speckle suppression can reduce speckle contrast to 0.13, but it increases the processing time per frame by 5 times; multi-frame averaging does not affect the generation rate, but it leads to a 35% decrease in dynamic resolution. In addition, the lack of an adaptive depth layer selection mechanism in existing technologies means that in scenes with dynamically changing depth ranges, either the depth restoration accuracy will decrease due to kernel function matching deviation, or the efficiency will be wasted due to redundant calculations. This results in a dilemma of both reduced accuracy and efficiency, which seriously restricts the engineering application of holographic technology in niche professional scenarios.

[0005] Therefore, existing technologies suffer from problems such as redundant kernel function storage, insufficient depth matching accuracy, speckle suppression, and rate imbalance. Summary of the Invention

[0006] To overcome the problems of kernel function storage redundancy, insufficient depth matching accuracy, speckle suppression and rate imbalance in existing technologies, this invention discloses a hologram rapid generation system and method based on diffraction element optimization and lookup table pre-storage, which can effectively solve the above-mentioned technical problems.

[0007] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows: A fast hologram generation method based on diffraction element optimization and lookup table pre-stored data includes: Preprocessing constructs a depth-adaptive point spread function kernel library, establishes a parameterized point spread function model based on wave optics theory, decomposes it into basic diffraction kernels and modulation functions through a depth-dominant separable approximation, pre-calculates and stores the kernels and functions corresponding to discrete depth layers, forming a depth-kernel lookup table; Receive target 3D scene data, the data being in the form of a depth image or multiple layers, containing scene color information and depth information; Adaptively select key depth layers and corresponding kernel functions, match key depth layers from a lookup table based on scene depth range, and extract the corresponding basic diffraction kernels and modulation functions; The scene is segmented by depth slicing, and the scene color image is segmented into binary mask layers corresponding to key depth layers based on the depth information; Perform parallel convolution operations, convolving each mask layer with the corresponding basic diffraction kernel in two dimensions to generate contribution holograms for each depth layer; Modulation and coherent fusion are performed. The phase or amplitude of each contributing hologram is finely adjusted by the modulation function. All the finely adjusted contributing holograms are coherently superimposed to obtain the initial hologram. After performing speckle suppression post-processing, the initial hologram is optimized using a lightweight convolutional neural network to output a high-quality hologram.

[0008] Preferably, the preprocessing to construct the depth-adaptive point spread function kernel library includes: Based on wave optics theories such as the angular spectrum method, a mapping model between the point spread function and the three-dimensional coordinates of the point source and the wavelength of light is established. The point spread function is decomposed into a basic diffraction nucleus and a modulation function using a separable approximation. The basic diffraction nucleus is strongly dependent on depth, while the modulation function is weakly dependent on lateral position and changes slowly. Discrete sampling is performed on the depth dimension of the three-dimensional space, and the basic diffraction kernel and modulation function corresponding to each discrete depth layer are pre-calculated. The correspondence between discrete depth layers, basic diffraction kernels, and modulation functions is stored to form a depth-kernel lookup table.

[0009] Preferably, the adaptive selection of the key depth layer and corresponding kernel function includes: Analyze the depth range of the target 3D scene to determine the minimum and maximum depth values ​​of the scene; Filter the key depth layers that cover this depth range from the lookup table. The number of layers to filter is determined based on the complexity of the scene's depth hierarchy. The fundamental diffraction nuclei and modulation functions corresponding to each key depth layer are extracted and used as the core parameters for subsequent calculations.

[0010] Preferably, the depth slicing of the scene includes: Extract the depth value of each pixel from the scene depth image; The depth value of each pixel is matched with the key depth layer to determine the target depth layer to which the pixel belongs; Based on the relationship, a binary mask layer is generated corresponding to each key depth layer. Only the pixel information of the corresponding depth layer is retained in the mask layer.

[0011] Preferably, performing parallel convolution operations includes: By utilizing the GPU parallel computing architecture, two-dimensional convolution operations are performed synchronously on each binary mask layer; Each mask layer is convolved only with its corresponding basic diffraction kernel to generate a contribution hologram specific to that depth layer; During convolution, the spatial resolution of the image is maintained to ensure accurate mapping of depth information.

[0012] Preferably, the implementation of modulation and coherent fusion includes: Each contributing hologram is multiplied pixel by pixel with its corresponding modulation function to correct off-axis aberrations; Complex-domain coherent superposition is performed on all modulated contribution holograms to obtain the initial hologram; During the superposition process, the phase information of each contributing hologram is kept intact to ensure accurate restoration of the depth level.

[0013] Preferably, the optional speckle suppression post-processing includes: A lightweight UNet convolutional neural network with no more than 5 layers is constructed. The network takes the initial hologram generated by convolution as input and the high-quality hologram after iterative optimization as output label. The network is trained to learn the mapping relationship from the initial hologram to the high-quality hologram, and the mapping focuses on speckle noise suppression; The initial hologram is input into the trained network, and the optimized hologram after speckle suppression is output.

[0014] Preferably, the method further includes a lookup table update optimization step: Collect hologram generation feedback data from different scenarios and analyze the correlation between kernel function matching accuracy and generation quality; Supplement sampling is performed on sparse regions of depth sampling, and new kernel functions and modulation functions corresponding to the depth layers are added; The existing kernel function and modulation function are iteratively optimized to improve adaptability in different scenarios.

[0015] Preferably, the method is applicable to microscopic or near-eye display scenarios with small field of view and high depth of field, including AR / VR virtual dashboards and biomedical microscopic imaging scenarios.

[0016] Preferably, the hologram rapid generation system based on diffraction element optimization and lookup table pre-stored includes: The preprocessing module is used to build a depth-adaptive point spread function kernel library. Through parametric modeling, separable approximation, pre-computation and storage, it generates a depth-kernel lookup table. The data receiving module is used to receive target 3D scene data, which is in the form of depth images or multi-layer layers, and to parse and extract color and depth information. The kernel function matching module is used to adaptively select key depth layers from the lookup table based on the scene depth range and extract the corresponding basic diffraction kernels and modulation functions. The depth slicing module is used to segment the scene color image into binary mask layers corresponding to key depth layers based on depth information. The convolution calculation module is used to perform parallel two-dimensional convolution on each mask layer and the corresponding basic diffraction kernel to generate contribution holograms for each depth layer. The modulation and fusion module is used to fine-tune the contribution holograms through a modulation function and coherently superimpose all the contribution holograms to obtain the initial hologram. The post-processing module contains a lightweight convolutional neural network for speckle suppression optimization of the initial hologram, outputting a high-quality hologram. The system optimizes the calculation of diffraction elements through separable approximation, improves the generation speed by using lookup table pre-stored data and parallel convolution, and is suitable for microscopic or near-eye display scenarios with small field of view and high depth of field.

[0017] Compared with existing technologies, the beneficial effects of this invention are as follows: This method solves the kernel function storage redundancy problem through depth-dominated separable approximation and lookup table optimization design. It first decomposes the point spread function into a fundamental diffraction kernel that strongly depends on depth and a modulation function that weakly depends on lateral position, based on wave optics theory. It pre-computes and stores the corresponding kernel and function to form a lookup table only for discrete depth layers, avoiding the repeated storage of the point spread function across the entire space as in traditional methods. Because the modulation function changes slowly and does not require high-frequency updates, the storage quantity is precisely controlled according to the number of discrete depth samples, reducing redundancy and thus reducing memory overhead during data retrieval, providing data for parallel computing. Regarding the problem of insufficient depth matching accuracy, a breakthrough is achieved through scene depth analysis and adaptive key layer selection. First, the minimum and maximum depth values ​​of the scene are determined, and then the key depth layers covering the coverage area are matched from the lookup table based on the depth hierarchy complexity. Combined with pixel-level depth matching and binary mask segmentation, each... Pixels are precisely assigned to the target depth layer, ensuring accurate matching between the basic diffraction kernel and the corresponding depth region. This solves the kernel function mismatch problem caused by coarse depth layering in traditional methods, providing accuracy assurance for holographic depth information reconstruction. To address speckle suppression and rate imbalance, a collaborative strategy of lightweight networks and parallel computing is adopted. On one hand, GPU architecture is used to simultaneously perform convolution operations on multiple mask layers, enabling parallel generation of holograms from each depth layer and improving computational speed. On the other hand, a lightweight UNet network with fewer than 5 layers is constructed, focusing on dedicated mapping learning for speckle noise suppression, avoiding the drag on speed caused by complex networks. Simultaneously, phase integrity is preserved through modulation function fine-tuning and coherent superposition in the complex domain. After optimization by the lightweight network, the initial hologram achieves effective speckle noise suppression while maintaining the generation rate. Ultimately, this achieves the dual goals of high-quality holograms and rapid generation, particularly suitable for AR / VR and biological microscopy imaging scenarios with small field of view and high depth of field. Attached Figure Description

[0018] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely exemplary. For those skilled in the art, other embodiments can be derived from the provided drawings without creative effort.

[0019] Figure 1 This is a diagram illustrating the steps of the method of the present invention; Figure 2 This is a system structure diagram of the present invention. Detailed Implementation

[0020] The accompanying drawings are for illustrative purposes only and should not be construed as limiting the scope of this patent. To better illustrate this embodiment, some parts in the accompanying drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions; It will be understood by those skilled in the art that certain well-known structures and their descriptions may be omitted in the accompanying drawings.

[0021] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments. Example 1

[0022] This method is based on wave optics theory, a branch of optics that studies the propagation laws of light, mainly involving light interference, diffraction, and polarization. In hologram generation, the point spread function (PSF) model is crucial, as it describes the response of an optical system to a point source and reflects the system's resolution and imaging quality. Based on wave optics theories such as the angular spectrum method, a mapping model between the PSF and the three-dimensional coordinates of the point source and the wavelength of light is established. This model uses mathematical formulas to accurately describe the response of a point source in the optical system under different three-dimensional coordinates and the influence of light wavelength, providing a foundation for hologram calculation.

[0023] The separable approximation is one of the key techniques of this method. It decomposes the point spread function into a basic diffraction nucleus and a modulation function. The basic diffraction nucleus is strongly dependent on depth, which means that its shape and characteristics change significantly with the depth of the object. Different depths will cause significant changes in the shape, size and other parameters of the basic diffraction nucleus. The modulation function is weakly dependent on the lateral position and changes slowly, which means that the change of the modulation function in the lateral position has little impact on it, and this change is relatively gentle and will not fluctuate drastically. Through this decomposition method, the complex point spread function is simplified into two relatively simple parts, which facilitates subsequent calculation and processing.

[0024] The construction of a depth-kernel lookup table is an important means to achieve rapid hologram generation. The depth dimension of three-dimensional space is discretely sampled, and the continuous depth range is divided into a series of discrete depth values. Then, for each discrete depth layer, the corresponding basic diffraction kernel and modulation function are pre-calculated, and their correspondence is stored to form a depth-kernel lookup table. In this way, when actually generating a hologram, it is not necessary to repeatedly calculate these kernel functions. The corresponding kernel function can be quickly obtained from the lookup table based on the depth information of the scene, which improves the computational efficiency.

[0025] Please see Figure 1 A fast hologram generation method based on diffraction element optimization and lookup table pre-stored data includes: Preprocessing constructs a depth-adaptive point spread function kernel library, establishes a parameterized point spread function model based on wave optics theory, decomposes it into basic diffraction kernels and modulation functions through a depth-dominant separable approximation, pre-calculates and stores the kernels and functions corresponding to discrete depth layers, forming a depth-kernel lookup table; Receive target 3D scene data, the data being in the form of a depth image or multiple layers, containing scene color information and depth information; Adaptively select key depth layers and corresponding kernel functions, match key depth layers from a lookup table based on scene depth range, and extract the corresponding basic diffraction kernels and modulation functions; The scene is segmented by depth slicing, and the scene color image is segmented into binary mask layers corresponding to key depth layers based on the depth information; Perform parallel convolution operations, convolving each mask layer with the corresponding basic diffraction kernel in two dimensions to generate contribution holograms for each depth layer; Modulation and coherent fusion are performed. The phase or amplitude of each contributing hologram is finely adjusted by the modulation function. All the finely adjusted contributing holograms are coherently superimposed to obtain the initial hologram. After performing speckle suppression post-processing, the initial hologram is optimized using a lightweight convolutional neural network to output a high-quality hologram.

[0026] The preprocessing constructs a depth-adaptive point spread function kernel library, which includes: Based on wave optics theories such as the angular spectrum method, a mapping model between the point spread function and the three-dimensional coordinates of the point source and the wavelength of light is established. The point spread function is decomposed into a basic diffraction nucleus and a modulation function using a separable approximation. The basic diffraction nucleus is strongly dependent on depth, while the modulation function is weakly dependent on lateral position and changes slowly. Discrete sampling is performed on the depth dimension of the three-dimensional space, and the basic diffraction kernel and modulation function corresponding to each discrete depth layer are pre-calculated. The correspondence between discrete depth layers, basic diffraction kernels, and modulation functions is stored to form a depth-kernel lookup table.

[0027] The adaptive selection of key depth layers and corresponding kernel functions includes: Analyze the depth range of the target 3D scene to determine the minimum and maximum depth values ​​of the scene; Filter the key depth layers that cover this depth range from the lookup table. The number of layers to filter is determined based on the complexity of the scene's depth hierarchy. The fundamental diffraction nuclei and modulation functions corresponding to each key depth layer are extracted and used as the core parameters for subsequent calculations.

[0028] The process of performing depth slicing and segmentation of the scene includes: Extract the depth value of each pixel from the scene depth image; The depth value of each pixel is matched with the key depth layer to determine the target depth layer to which the pixel belongs; Based on the relationship, a binary mask layer is generated corresponding to each key depth layer. Only the pixel information of the corresponding depth layer is retained in the mask layer.

[0029] The parallel convolution operation includes: By utilizing the GPU parallel computing architecture, two-dimensional convolution operations are performed synchronously on each binary mask layer; Each mask layer is convolved only with its corresponding basic diffraction kernel to generate a contribution hologram specific to that depth layer; During convolution, the spatial resolution of the image is maintained to ensure accurate mapping of depth information.

[0030] The implementation of modulation and coherent fusion includes: Each contributing hologram is multiplied pixel by pixel with its corresponding modulation function to correct off-axis aberrations; Complex-domain coherent superposition is performed on all modulated contribution holograms to obtain the initial hologram; During the superposition process, the phase information of each contributing hologram is kept intact to ensure accurate restoration of the depth level.

[0031] The optional speckle suppression post-processing includes: A lightweight UNet convolutional neural network with no more than 5 layers is constructed. The network takes the initial hologram generated by convolution as input and the high-quality hologram after iterative optimization as output label. The network is trained to learn the mapping relationship from the initial hologram to the high-quality hologram, and the mapping focuses on speckle noise suppression; The initial hologram is input into the trained network, and the optimized hologram after speckle suppression is output.

[0032] The method also includes a lookup table update optimization step: Collect hologram generation feedback data from different scenarios and analyze the correlation between kernel function matching accuracy and generation quality; Supplement sampling is performed on sparse regions of depth sampling, and new kernel functions and modulation functions corresponding to the depth layers are added; The existing kernel function and modulation function are iteratively optimized to improve adaptability in different scenarios.

[0033] The method is applicable to microscopic or near-eye display scenarios with small field of view and high depth of field, including AR / VR virtual dashboards and biomedical microscopic imaging scenarios.

[0034] In practical implementation, based on wave optics theory, the angular spectral method is used to establish a mapping model between the point spread function and the three-dimensional coordinates and wavelength of the point source. For example, using relevant wave equations and diffraction formulas, the propagation and diffusion characteristics of the point source under different positions and wavelengths are accurately described. Through separable approximation, the point spread function is decomposed into a basic diffraction nucleus and a modulation function. The depth dimension of the three-dimensional space is discretely sampled. Assuming the depth range is from 0 to 100 mm, discrete sampling is performed at 1 mm intervals to obtain 100 discrete depth layers. For each discrete depth layer, the corresponding basic diffraction nucleus and modulation function are pre-calculated, and the correspondence between the discrete depth layer, the basic diffraction nucleus, and the modulation function is stored to form a depth-nucleus lookup table. For example, for a discrete depth layer with a depth of 10 mm, the corresponding basic diffraction nucleus is calculated as [specific nucleus value], and the modulation function is calculated as [specific function expression], and this information is stored in the lookup table.

[0035] Data acquisition devices, such as depth cameras or other 3D data acquisition sensors, are used to receive target 3D scene data. This data can be in the form of depth images or multi-layer layers. Assuming that the received data is a depth image containing scene color information and depth information, the color information of each pixel, such as RGB values, and the depth information, i.e., the object depth value corresponding to each pixel, are extracted from the depth image through image parsing algorithms. This information will serve as the basic data for subsequent processing.

[0036] The depth range of the target 3D scene is analyzed. By analyzing the depth values ​​of all pixels in the depth image, the minimum and maximum depth values ​​of the scene are determined. Assuming the minimum depth is 10mm and the maximum depth is 50mm, key depth layers covering this depth range are selected from a lookup table. The number of key depth layers to select is determined based on the complexity of the scene's depth hierarchy. If the scene's depth hierarchy is relatively simple, 3-5 key depth layers may be sufficient; if the complexity is high, 5-10 key depth layers may need to be selected. For example, if 3 key depth layers are selected, namely 10mm, 30mm, and 50mm, the basic diffraction kernel and modulation function corresponding to each key depth layer are extracted. The basic diffraction kernel and modulation function corresponding to depths of 10mm, 30mm, and 50mm are obtained from the lookup table as the core parameters for calculation.

[0037] Extract the depth value of each pixel in the scene depth image. Iterate through each pixel in the depth image to obtain its corresponding depth value. Match the depth value of each pixel with the key depth layers to determine the target depth layer to which the pixel belongs. For example, if the depth value of a pixel is 25mm, by comparison, it is determined that its target depth layer is 30mm. Generate binary mask layers corresponding to each key depth layer based on the belonging relationship. For each key depth layer, create a binary mask layer with the same size as the depth image. In the mask layer, only the pixel information of the corresponding depth layer is retained, and the pixel values ​​of other depth layers are set to 0. For example, for the mask layer of the 30mm depth layer, pixels with depth values ​​in the range of 25-35mm are set to 1, and other pixels are set to 0.

[0038] By leveraging the parallel computing architecture of GPUs, two-dimensional convolution operations are performed synchronously on each binary mask layer. Each mask layer is convolved only with its corresponding basic diffraction kernel. For example, the mask layer with a depth of 30mm is convolved with the basic diffraction kernel corresponding to a depth of 30mm to generate a contribution hologram specific to that depth layer. The image spatial resolution is maintained during the convolution process to ensure accurate mapping of depth information. By setting the convolution parameters appropriately, the size and resolution of the convolved image are the same as those of the original mask layer, thereby ensuring that depth information is not lost or deformed during the convolution process.

[0039] Each contributing hologram is multiplied pixel-by-pixel with its corresponding modulation function. Based on the characteristics of the modulation function, the phase or amplitude of the contributing hologram is finely adjusted to correct off-axis aberrations. For example, for a given contributing hologram, the phase or amplitude of each pixel is adjusted according to its corresponding modulation function to improve image quality and accuracy. Complex domain coherent superposition is performed on all modulated contributing holograms. All modulated contributing holograms are superimposed according to complex number operation rules to obtain the initial hologram. During the superposition process, the integrity of the phase information of each contributing hologram is maintained to ensure accurate restoration of depth levels. Through precise complex number operations, the phase information of each contributing hologram is preserved, so that the superimposed initial hologram can accurately reflect the information of different depth levels and present a clear three-dimensional effect.

[0040] A lightweight UNet convolutional neural network with no more than 5 layers is constructed. The network structure is designed to include an input layer, convolutional layers, pooling layers, upsampling layers, and an output layer. The initial hologram generated by convolution is used as input, and the high-quality hologram after iterative optimization is used as the output label. Through a large amount of training data, the network is trained to learn the mapping relationship from the initial hologram to the high-quality hologram. During the training process, the network parameters are adjusted so that the network can accurately identify and remove speckle noise in the initial hologram. The initial hologram is then input into the trained network, and the network processes the initial hologram according to the learned mapping relationship, outputting an optimized hologram with speckle suppression, thereby improving the quality and clarity of the hologram.

[0041] This study collects feedback data on hologram generation in different scenarios. By monitoring and recording the hologram generation process and results in various scenarios, it analyzes the correlation between kernel function matching accuracy and generation quality. For example, it statistically analyzes the matching accuracy of specific kernel functions and the quality indicators of generated holograms, such as sharpness and contrast, in certain scenarios. It supplements sampling in sparse depth-sampling regions, identifies areas with insufficient depth sampling based on the analysis results, adds sampling points of discrete depth layers in these regions, adds corresponding depth-layer kernel functions and modulation functions, and iteratively optimizes existing kernel and modulation functions. Through algorithm optimization, it adjusts the parameters of kernel and modulation functions to improve adaptability in different scenarios, enabling the lookup table to better adapt to the needs of various complex scenarios and improve the generation quality and efficiency of holograms. Example 2

[0042] The hologram rapid generation system based on diffraction element optimization and lookup table pre-storage mainly consists of a preprocessing module, a data receiving module, a kernel function matching module, a depth slicing module, a convolution calculation module, a modulation fusion module, and a post-processing module. These modules cooperate to complete the entire process from receiving 3D scene data to generating a high-quality hologram. The overall system operation flow is as follows: First, the preprocessing module constructs a depth-adaptive point spread function kernel library and generates a depth-kernel lookup table; then, the data receiving module receives the target 3D scene data and extracts its color and depth information; next, the kernel function matching module selects key depth layers and corresponding kernel functions from the lookup table according to the scene depth range; the depth slicing module segments the scene color image into binary mask layers based on the depth information; the convolution calculation module performs parallel two-dimensional convolution on the mask layers and the corresponding basic diffraction kernels to generate contribution holograms for each depth layer; the modulation fusion module fine-tunes the contribution holograms using modulation functions and then coherently superimposes them to obtain the initial hologram; finally, the post-processing module uses a lightweight convolutional neural network to optimize the speckle suppression of the initial hologram and outputs a high-quality hologram.

[0043] Please see Figure 2A hologram rapid generation system based on diffraction element optimization and lookup table pre-stored data includes: The preprocessing module is used to build a depth-adaptive point spread function kernel library. Through parametric modeling, separable approximation, pre-computation and storage, it generates a depth-kernel lookup table. The data receiving module is used to receive target 3D scene data, which is in the form of depth images or multi-layer layers, and to parse and extract color and depth information. The kernel function matching module is used to adaptively select key depth layers from the lookup table based on the scene depth range and extract the corresponding basic diffraction kernels and modulation functions. The depth slicing module is used to segment the scene color image into binary mask layers corresponding to key depth layers based on depth information. The convolution calculation module is used to perform parallel two-dimensional convolution on each mask layer and the corresponding basic diffraction kernel to generate contribution holograms for each depth layer. The modulation and fusion module is used to fine-tune the contribution holograms through a modulation function and coherently superimpose all the contribution holograms to obtain the initial hologram. The post-processing module contains a lightweight convolutional neural network for speckle suppression optimization of the initial hologram, outputting a high-quality hologram. The system optimizes the calculation of diffraction elements through separable approximation, improves the generation speed by using lookup table pre-stored data and parallel convolution, and is suitable for microscopic or near-eye display scenarios with small field of view and high depth of field.

[0044] The preprocessing module constructs a depth-adaptive point spread function (PSF) kernel library. Based on wave optics theories such as the angular spectrum method, this module establishes a mapping model between the PSF and the three-dimensional coordinates and wavelength of the point source. Using a separable approximation, the PSF is decomposed into a basic diffraction kernel and a modulation function. The basic diffraction kernel is strongly dependent on depth, while the modulation function is weakly dependent on the lateral position and changes slowly. Discrete sampling is performed on the depth dimension of the three-dimensional space, assuming a sampling interval of [missing value]. From minimum depth To maximum depth Sampling was performed to obtain a series of discrete depth values. (i = 0, 1, 2, ..., N, where N is the number of sampling points). For each discrete depth layer, the corresponding fundamental diffraction kernel and modulation function are pre-calculated, and the correspondence between discrete depth layers, fundamental diffraction kernels, and modulation functions is stored to form a depth-kernel lookup table. For example, for a depth of... The discrete depth layers were used to calculate the basic diffraction nuclei. and modulation function And store them in the corresponding positions in the lookup table.

[0045] The data receiving module receives target 3D scene data through various sensors or data interfaces. This data can be depth images captured by a depth camera or multi-layered data generated by other 3D modeling software. The data receiving module parses the received data, extracting color and depth information. Taking a depth image as an example, for each pixel (x, y) in the image, its color value is extracted, such as the color value in the RGB color space. and the corresponding depth value This information will provide data for subsequent processing.

[0046] The kernel function matching module parses the depth range of the target 3D scene and determines the minimum depth of the scene through statistical analysis of the depth values ​​of all pixels in the depth image. With maximum depth value The key depth layers covering this depth range are selected from the lookup table. The number of layers to select is determined by the complexity of the scene's depth hierarchy. For example, for a depth range of... In scenarios where the depth complexity is relatively low, three key depth layers can be selected, which are approximately equal to... of intermediate depth and close of Extract the fundamental diffraction nuclei and modulation functions corresponding to each key depth layer, and obtain the depth values ​​from the lookup table. , , The corresponding fundamental diffraction nuclei , , and modulation function , , , which serves as the core parameter for subsequent calculations.

[0047] The depth slicing module extracts the depth value of each pixel in the scene depth image, iterates through all pixels (x, y) in the depth image, and obtains their depth values. The depth value of each pixel is matched with the key depth layer to determine the target depth layer to which the pixel belongs. For example, for a pixel... Its depth value By comparison With key depth layer , , The size relationship determines the target depth layer to which it belongs, assuming Then the target depth layer to which this pixel belongs is Based on the relationship, generate binary mask layers corresponding to each key depth layer. For each key depth layer, create a binary mask layer with the same size as the depth image. In the mask layer, only the pixel information of the corresponding depth layer is retained, and the pixel values ​​of other depth layers are set to 0. For example, for... Depth layer mask layer When the depth value of pixel (x, y) belongs to When the corresponding depth range is, ,otherwise .

[0048] The convolution computation module utilizes a GPU parallel computing architecture, leveraging the multi-core parallel processing capabilities of the GPU to synchronously perform two-dimensional convolution operations on each binary mask layer. Each mask layer is convolved only with its corresponding basic diffraction kernel, for example... Depth layer mask layer With fundamental diffraction nuclei Perform convolution to generate a contribution hologram specific to this depth layer. During the convolution process, the spatial resolution of the image is maintained. By reasonably setting parameters such as the size and stride of the convolution kernel, the size and resolution of the convolutional contribution hologram are ensured to be the same as those of the original mask layer, thereby ensuring accurate mapping of depth information and enabling the contribution hologram to accurately reflect the information of the corresponding depth layer.

[0049] The modulation fusion module multiplies each contributing hologram pixel-by-pixel with its corresponding modulation function. Based on the characteristics of the modulation function, it fine-tunes the phase or amplitude of the contributing holograms to correct off-axis aberrations. For example, for the contributing hologram... Combine it with the modulation function Pixel-by-pixel multiplication yields the modulated contribution hologram. This method adjusts the phase or amplitude of the contribution holograms to improve image quality and accuracy. Complex-domain coherent superposition is performed on all modulated contribution holograms to create the modulated contribution holograms. , , The initial hologram is obtained by superimposing the holograms according to the rules of complex number operations. During the superposition process, the phase information of each contributing hologram is kept intact. Through precise complex number operations, the phase information of each contributing hologram is preserved, so that the initial hologram after superposition can accurately reflect the information of different depth layers and present a clear three-dimensional effect.

[0050] The post-processing module constructs a lightweight UNet convolutional neural network with no more than 5 layers. The network structure is designed to include an input layer, convolutional layers, pooling layers, upsampling layers, and an output layer. It takes the initial hologram generated by convolution as input and the high-quality hologram after iterative optimization as output label. Through a large amount of training data, the network is trained to learn the mapping relationship from the initial hologram to the high-quality hologram. During the training process, the network parameters, such as the weights and biases of the convolutional kernels, are continuously adjusted to enable the network to accurately identify and remove speckle noise in the initial hologram. The initial hologram is then input into the trained network, which processes the initial hologram according to the learned mapping relationship and outputs an optimized hologram with speckle suppression, thereby improving the quality and clarity of the hologram and meeting the needs of practical applications.

[0051] In AR / VR virtual dashboard scenarios, this system can quickly generate high-quality holograms, providing users with an immersive interactive experience. Taking a car AR virtual dashboard as an example, the system first receives vehicle driving data collected by the car's sensors, including information such as vehicle speed, RPM, and fuel level, as well as 3D data of the vehicle's surrounding environment. This data is parsed by the data receiving module to extract color and depth information. The preprocessing module uses a pre-built depth-kernel lookup table to provide a basis for calculation. The kernel function matching module selects the key depth layer and corresponding kernel function according to the scene's depth range. The depth slicing module slices the dashboard and its surroundings. The environmental color image is segmented into binary mask layers according to depth information. The convolutional computation module uses GPU parallel computing to quickly generate contribution holograms for each depth layer. The modulation and fusion module fine-tunes the contribution holograms and then coherently superimposes them to obtain the initial hologram. The post-processing module optimizes the initial hologram with speckle suppression through a lightweight convolutional neural network, and finally outputs a clear and realistic car AR virtual dashboard hologram. After wearing the AR device, users can intuitively see the virtual dashboard floating in the real driving environment, displaying information such as vehicle speed and RPM, and it has a realistic three-dimensional effect, improving the convenience and technological feel of driving.

[0052] In the field of biomedical microscopy, this system helps doctors observe the three-dimensional structure of biological samples more accurately, aiding in disease diagnosis and treatment planning. Taking cell tissue imaging as an example, three-dimensional data of biological samples are collected through a microscope. The data receiving module receives this data and extracts color and depth information. The depth-kernel lookup table constructed by the preprocessing module plays a role in the calculation of different depth layers. The kernel function matching module selects the appropriate key depth layer and kernel function according to the depth range of the sample. The depth slicing module segments the color image of cell tissue into binary mask layers corresponding to the depth layers. The convolution calculation module generates contribution holograms of each depth layer in parallel. The modulation and fusion module processes them to obtain the initial hologram. The post-processing module optimizes the initial hologram with speckle suppression and outputs a high-quality cell tissue hologram. By observing the hologram, doctors can clearly see the morphology, distribution, and internal structure of cells from different angles. Compared with traditional two-dimensional imaging, it can obtain more comprehensive information, which helps to diagnose diseases more accurately and study the physiological and pathological processes of cells.

[0053] The same or similar labels correspond to the same or similar parts; The terms used to describe positional relationships in the accompanying drawings are for illustrative purposes only and should not be construed as limiting this patent. Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the implementation of the present invention. For those skilled in the art, other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all implementation methods here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the claims of the present invention.

Claims

1. A method for rapid hologram generation based on diffraction element optimization and pre-stored lookup tables, characterized in that, include: Preprocessing constructs a depth-adaptive point spread function kernel library, establishes a parameterized point spread function model based on wave optics theory, decomposes it into basic diffraction kernels and modulation functions through a depth-dominant separable approximation, pre-calculates and stores the kernels and functions corresponding to discrete depth layers, forming a depth-kernel lookup table; Receive target 3D scene data, the data being in the form of a depth image or multiple layers, containing scene color information and depth information; Adaptively select key depth layers and corresponding kernel functions, match key depth layers from a lookup table based on scene depth range, and extract the corresponding basic diffraction kernels and modulation functions; The scene is segmented by depth slicing, and the scene color image is segmented into binary mask layers corresponding to key depth layers based on the depth information; Perform parallel convolution operations, convolving each mask layer with the corresponding basic diffraction kernel in two dimensions to generate contribution holograms for each depth layer; Modulation and coherent fusion are performed. The phase or amplitude of each contributing hologram is finely adjusted by the modulation function. All the finely adjusted contributing holograms are coherently superimposed to obtain the initial hologram. After performing speckle suppression post-processing, the initial hologram is optimized using a lightweight convolutional neural network to output a high-quality hologram.

2. The method according to claim 1, characterized in that, The preprocessing constructs a depth-adaptive point spread function kernel library, which includes: Based on wave optics theories such as the angular spectrum method, a mapping model between the point spread function and the three-dimensional coordinates of the point source and the wavelength of light is established. The point spread function is decomposed into a basic diffraction nucleus and a modulation function using a separable approximation. The basic diffraction nucleus is strongly dependent on depth, while the modulation function is weakly dependent on lateral position and changes slowly. Discrete sampling is performed on the depth dimension of the three-dimensional space, and the basic diffraction kernel and modulation function corresponding to each discrete depth layer are pre-calculated. The correspondence between discrete depth layers, basic diffraction kernels, and modulation functions is stored to form a depth-kernel lookup table.

3. The method according to claim 1, characterized in that, The adaptive selection of key depth layers and corresponding kernel functions includes: Analyze the depth range of the target 3D scene to determine the minimum and maximum depth values ​​of the scene; Filter the key depth layers that cover this depth range from the lookup table. The number of layers to filter is determined based on the complexity of the scene's depth hierarchy. The fundamental diffraction nuclei and modulation functions corresponding to each key depth layer are extracted and used as the core parameters for subsequent calculations.

4. The method according to claim 1, characterized in that, The process of performing depth slicing and segmentation of the scene includes: Extract the depth value of each pixel from the scene depth image; The depth value of each pixel is matched with the key depth layer to determine the target depth layer to which the pixel belongs; Based on the relationship, a binary mask layer is generated corresponding to each key depth layer. Only the pixel information of the corresponding depth layer is retained in the mask layer.

5. The method according to claim 1, characterized in that, The parallel convolution operation includes: By utilizing the GPU parallel computing architecture, two-dimensional convolution operations are performed synchronously on each binary mask layer; Each mask layer is convolved only with its corresponding basic diffraction kernel to generate a contribution hologram specific to that depth layer; During convolution, the spatial resolution of the image is maintained to ensure accurate mapping of depth information.

6. The method according to claim 1, characterized in that, The implementation of modulation and coherent fusion includes: Each contributing hologram is multiplied pixel by pixel with its corresponding modulation function to correct off-axis aberrations; Complex-domain coherent superposition is performed on all modulated contribution holograms to obtain the initial hologram; During the superposition process, the phase information of each contributing hologram is kept intact to ensure accurate restoration of the depth level.

7. The method according to claim 1, characterized in that, The optional speckle suppression post-processing includes: A lightweight UNet convolutional neural network with no more than 5 layers is constructed. The network takes the initial hologram generated by convolution as input and the high-quality hologram after iterative optimization as output label. The network is trained to learn the mapping relationship from the initial hologram to the high-quality hologram, and the mapping focuses on speckle noise suppression; The initial hologram is input into the trained network, and the optimized hologram after speckle suppression is output.

8. The method according to claim 1, characterized in that, The method also includes a lookup table update optimization step: Collect hologram generation feedback data from different scenarios and analyze the correlation between kernel function matching accuracy and generation quality; Supplement sampling is performed on sparse regions of depth sampling, and new kernel functions and modulation functions corresponding to the depth layers are added; The existing kernel function and modulation function are iteratively optimized to improve adaptability in different scenarios.

9. The method according to claim 1, characterized in that, The method is applicable to microscopic or near-eye display scenarios with small field of view and high depth of field, including AR / VR virtual dashboards and biomedical microscopic imaging scenarios.

10. A rapid hologram generation system based on diffraction element optimization and lookup table pre-storage, characterized in that, The method applied to any one of claims 1 to 9 includes: The preprocessing module is used to build a depth-adaptive point spread function kernel library. Through parametric modeling, separable approximation, pre-computation and storage, it generates a depth-kernel lookup table. The data receiving module is used to receive target 3D scene data, which is in the form of depth images or multi-layer layers, and to parse and extract color and depth information. The kernel function matching module is used to adaptively select key depth layers from the lookup table based on the scene depth range and extract the corresponding basic diffraction kernels and modulation functions. The depth slicing module is used to segment the scene color image into binary mask layers corresponding to key depth layers based on depth information. The convolution calculation module is used to perform parallel two-dimensional convolution on each mask layer and the corresponding basic diffraction kernel to generate contribution holograms for each depth layer. The modulation and fusion module is used to fine-tune the contribution holograms through a modulation function and coherently superimpose all the contribution holograms to obtain the initial hologram. The post-processing module contains a lightweight convolutional neural network for speckle suppression optimization of the initial hologram, outputting a high-quality hologram. The system optimizes the calculation of diffraction elements through separable approximation, improves the generation speed by using lookup table pre-stored data and parallel convolution, and is suitable for microscopic or near-eye display scenarios with small field of view and high depth of field.

Images