Light field display method and system with holography as representation medium

Abstract

The invention discloses a light field display method and system with holography as a representation medium, and the method comprises the following steps: S1, collecting a multi-view image of a target scene, carrying out the coding of the multi-view image, and generating a light field tensor containing space and angle information; s2, mapping the light field tensor in the step S1 into a two-dimensional space phase diagram; s3, converting the two-dimensional space phase diagram in the step S2 into a modulation diagram used for driving a spatial light modulator; and S4, driving the spatial light modulator to perform light wavefront modulation according to the modulation graph in the step S3, and reproducing the image of the target scene through optical diffraction. According to the invention, integration from dynamic scene information acquisition to high-quality three-dimensional image reproduction is realized, a low-redundancy and high-fidelity processing link is realized, and the efficiency and quality of holographic display are greatly improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of holographic display technology, and in particular to a light field display method and system taking holography as a representation medium. BACKGROUND

[0002] As a key technology for realizing real three-dimensional visual experience, holographic display has been widely studied in recent years in the fields of virtual reality, augmented reality, remote communication and medical imaging. Holographic technology can completely record the wavefront information of an object and realize three-dimensional image reconstruction with strong depth perception without the need for additional glasses or headsets. However, the existing holographic generation methods mostly rely on complex interference experiments or static calculations, lack real-time and flexibility, and are difficult to adapt to dynamic scenes.

[0003] In terms of light field technology, although rich spatial and directional information can be provided through multi-angle acquisition, the common acquisition method relies on a dense camera array, bringing about a large amount of data and high computational consumption. When combined with holographic technology, the problems of data redundancy and conversion efficiency are more prominent. In addition, existing solutions mostly stop at the static mapping of light field to hologram, and fail to form a systematic processing link, resulting in poor universality in different scenarios. At the same time, the existing holographic modulation methods mostly use uniform quantization, without considering the perception differences of the human eye in different regions, resulting in insufficient clarity in key regions and resource waste in non-key regions, and low overall energy efficiency of the system. More importantly, there is a lack of organic connection between the complete links from light field acquisition, phase map generation, modulation loading to final image reconstruction, making it difficult to realize efficient data flow transmission and overall coordination of physical links. Therefore, how to reduce the redundancy of light field acquisition while maintaining the integrity of spatial information, how to ensure coherence and directional consistency in the conversion process from light field to hologram, how to optimize resource allocation in the modulation link according to the human eye perception law, and how to form a closed-loop integrated system in the entire link, have become the core challenges faced by current holographic light field display technology.

[0004] The disclosure of the above background art content is only used to assist in understanding the concept and technical solutions of the present application, and it does not necessarily belong to the prior art of the present patent application. In the absence of explicit evidence that the above content has been disclosed on the filing date of the present patent application, the above background art should not be used to evaluate the novelty and inventiveness of the present application. SUMMARY

[0005] To solve the above technical problems, the present application proposes a light field display method and system taking holography as a representation medium, realizing an integrated, low-redundancy, high-fidelity processing link from dynamic scene information acquisition to high-quality three-dimensional image reproduction, greatly improving the efficiency and quality of holographic display.

[0006] In order to achieve the above object, the present application adopts the following technical solutions:

[0007] In a first aspect, the present application discloses a light field display method using holography as a characterization medium, comprising the following steps:

[0008] S1: Collecting multi-view images of a target scene, and performing encoding processing on the multi-view images to generate a light field tensor containing spatial and angular information;

[0009] S2: Mapping the light field tensor in step S1 into a two-dimensional spatial phase map;

[0010] S3: Converting the two-dimensional spatial phase map in step S2 into a modulation map for driving a spatial light modulator;

[0011] S4: Driving the spatial light modulator to perform light wavefront modulation according to the modulation map in step S3, and reproducing an image of the target scene through optical diffraction.

[0012] Preferably, S1 includes: collecting multi-view images of a target scene using a sparse camera array, and performing normalization preprocessing on the multi-view images, and then performing encoding processing on the multi-view images to generate a light field tensor containing spatial and angular information.

[0013] Preferably, the light field tensor containing spatial and angular information generated in S1 is:

[0014]

[0015] Wherein: F LF (x,y,u,v) is the output light field tensor, representing the feature response at spatial position (x,y) and viewing angle (u,v); f conv (I i,j (x,y)) represents the result of shared two-dimensional convolution processing on each input image I i,j (x,y) in the multi-view images; α i,j (x,y) is a fusion weight map.

[0016] Preferably, S2 includes:

[0017] S21: Expanding the light field tensor in step S1 into a set of spatial feature maps in the angular dimension, wherein each feature map in the spatial feature map is dynamically combined through a directional coupling residual module;

[0018] S22: Based on the spatial feature map, an unconstrained continuous phase map is obtained by regression;

[0019] S23: Calculate the direction gradient difference between the feature maps of each pair of adjacent viewing angles, and integrate it into the phase prediction target as a penalty term to multi-objective structure optimize the regression obtained unconstrained continuous phase map, and output a two-dimensional spatial phase map.

[0020] Preferably, the regression obtained unconstrained continuous phase map in S22 is:

[0021]

[0022] wherein: Φ raw (x,y) is the regression obtained unconstrained continuous phase map; represents the projection value of the feature map after convolution compression in the i-th direction; w i (x,y) is the direction weighted map;

[0023] The two-dimensional spatial phase map output in S23 is:

[0024]

[0025] wherein: Φ(x,y) is the final output phase map; is a two-dimensional Laplace regularization term; D coh (x,y) is a direction coherence bias term, which refers to the direction gradient difference of the feature maps corresponding to each pair of adjacent viewing angles; λ s and λ d are two hyperparameters for controlling the strength of the regularization; mod(·, 2π) represents the phase range mapping required by the modulation link.

[0026] Preferably, S3 includes:

[0027] S31: Generate a modulation resource weight map by combining the structural response and the perception distribution;

[0028] S32: Adjust the quantization step size based on the modulation resource weight map, and convert the two-dimensional spatial phase map in step S2 into a modulation map for driving the spatial light modulator using a non-uniform bit depth allocation mechanism.

[0029] Preferably, the modulation resource weight map generated in S31 is:

[0030]

[0031] wherein R(x,y) is the modulation resource weight map, normalized to [0,1]; is the spatial phase gradient intensity; W(x,y) is the perception weight map; γ is the balance parameter;

[0032] The modulation map for driving the spatial light modulator generated in S32 is:

[0033]

[0034] wherein: Φ SLM (i,j) is the output modulation pattern, whose value ranges from 0 to N-1; N is the modulation level of the spatial light modulator; and a is the local modulation enhancement factor; is the global mean value of R(x,y); denotes the floor function.

[0035] Preferably, S4 comprises:

[0036] S41: obtaining a spatial light modulator response lookup table, converting the modulation pattern in step S3 into an actual phase loading pattern according to the spatial light modulator response lookup table, and driving the spatial light modulator to modulate the light wavefront based on the actual phase loading pattern;

[0037] S42: based on a two-dimensional diffraction model, calculating the light intensity pattern on the observation plane of the light wave modulated by the spatial light modulator, and reproducing the image of the target scene based on the light intensity pattern.

[0038] Preferably, the actual phase loading pattern obtained in S41 is:

[0039]

[0040] wherein: denotes the phase value loaded by each pixel position of the spatial light modulator; and Γ(·) is the spatial light modulator response lookup table, which contains the nonlinear mapping relationship from the gray value to the actual phase value.

[0041] The light intensity pattern on the observation plane of the light wave modulated by the spatial light modulator calculated in S42 is:

[0042]

[0043] wherein: I out (x,y) is the gray value distribution of the final reproduced image; is the complex amplitude of the wavefront generated by the spatial light modulator; and F and F -1 are the two-dimensional Fourier transform and inverse transform, respectively; H(f x ,f y ) is the Fresnel transfer function; and the frequency variables f x ,f y are the frequency domain coordinate grids corresponding to the spatial position (x,y) one by one.

[0044] In a second aspect, the present application discloses a light field display system taking holography as a characterization medium, comprising:

[0045] The light field acquisition and encoding module is configured to acquire multi-view images of a target scene and encode the multi-view images to generate a light field tensor containing spatial and angular information;

[0046] The holographic phase map generation module is connected with the light field acquisition and encoding module and is configured to map the light field tensor generated by the light field acquisition and encoding module into a two-dimensional spatial phase map;

[0047] The spatial light modulator modulation map generation module is connected with the holographic phase map generation module and is configured to convert the two-dimensional spatial phase map generated by the holographic phase map generation module into a modulation map for driving the spatial light modulator;

[0048] The image reproduction module is connected with the spatial light modulator modulation map generation module and is configured to drive the spatial light modulator to modulate a light wave front according to the modulation map generated by the spatial light modulator modulation map generation module, and reproduce an image of the target scene through optical diffraction.

[0049] In a third aspect, the present application discloses a computer readable storage medium, and the computer readable storage medium stores a computer program, wherein the computer program is arranged to be run by a processor to execute the holographic light field display method of the first aspect.

[0050] Compared with the prior art, the holographic light field display method and system have the following beneficial effects: the multi-view images of a target scene are first acquired and encoded to generate a light field tensor containing spatial and angular information, thereby effectively reducing the amount of redundant data; then the light field tensor is mapped into a two-dimensional spatial phase map to generate a phase map through structural constraints, and is further converted into a modulation map for driving a spatial light modulator; finally, the spatial light modulator is driven by the modulation map, and an image of the target scene is reproduced through optical diffraction; through the cooperation of light field encoding, angular coherence constraints and further modulation, the overall performance of the holographic light field display system is optimized, the system data processing burden and hardware resource demand are significantly reduced while ensuring high display quality, and the efficiency and quality of holographic display are greatly improved.

[0051] In further schemes, the present application has the following beneficial effects:

[0052] (1) In S1, a controllable sparse camera array is used in combination with an encoding mechanism to obtain a structure-preserving light field tensor, thereby further effectively reducing the amount of redundant data;

[0053] (2) In S2, angular coupling and coherence constraints are introduced in the conversion link, so that the phase map maintains the consistency of the viewing angle and the continuity of the structure while ensuring the realizability of the physical modulation;

[0054] (3) S3 in the modulation stage, the dynamic quantization allocation strategy guided by the structural features and the human eye perception model, makes the limited modulation resources of the spatial light modulator concentrate on the key area, thereby improving the display clarity without increasing the hardware complexity.

[0055] (4) S4 in the final image reconstruction stage, the actual observation image is generated by combining the nonlinear response of the spatial light modulator and the optical propagation model, ensuring the consistency from digital control to physical imaging.

[0056] Other beneficial effects of the embodiments of the present application will be further described below. BRIEF DESCRIPTION OF DRAWINGS

[0057] Figure 1 is a flow chart of the light field display method disclosed in Embodiment One of the present application, which takes holography as a characterization medium. DETAILED DESCRIPTION

[0058] The embodiments of the present application will be described in detail below. It should be emphasized that the following description is merely exemplary, and is not intended to limit the scope of the present application and its applications.

[0059] It should be noted that when an element is referred to as being "fixed" or "disposed" on another element, it can be directly on the other element or indirectly on the other element. When an element is referred to as being "connected" to another element, it can be directly connected to the other element or indirectly connected to the other element. In addition, the connection can be for fixing or for electrical / signal communication.

[0060] It should be understood that the terms "length", "width", "upper", "lower", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are merely for the convenience of describing the embodiments of the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the present application.

[0061] In addition, the terms "first", "second", are used only for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined with "first", "second" can explicitly or implicitly include one or more of the features. In the description of the embodiments of the present application, the meaning of "a plurality of" is two or more, unless otherwise specifically limited.

[0062] To solve the problems of redundancy at the collection end, inefficiency in the conversion link, insufficient clarity in the display stage, and isolation between links in the prior art, as shown in Figure 1 The preferred embodiment of the present application discloses a light field collection, rendering and display method using holography as a representation medium, comprising the following steps:

[0063] S1: Collecting multi-view images of the target scene, and encoding the multi-view images to generate a light field tensor containing spatial and angular information;

[0064] Further, this step S1 uses a sparse camera array to collect multi-view images of the target scene, and performs normalization preprocessing on the multi-view images, and then encodes the multi-view images to generate a light field tensor containing spatial and angular information.

[0065] Specifically, this step S1 includes dynamic sparse light field collection and encoding. This step aims to complete the initial collection of multi-angle light field information, and construct a four-dimensional light field tensor with spatial and angular expression capability through structure perception. To avoid the problems of high equipment cost and difficult data processing caused by a large number of redundant views in traditional light field collection, this scheme uses a sparse camera array in combination with an encoding network to extract representative spatial-view joint expressions from a real three-dimensional scene, providing an expression basis with directional continuity and structural integrity for the subsequent hologram generation step.

[0066] The input data is a set of multi-view RGB image sequences of a three-dimensional scene, denoted as I u,v (x,y), where (u,v) represents the view number (indexed in horizontal and vertical arrangement logic), and (x,y) represents the image space coordinates. Each image is a three-channel color image with a pixel size of 1280x720, and the image data is represented by 32-bit floating point, normalized to the [0,1] interval.

[0067] These images are obtained by a set of mechanically controllable camera arrays. The camera array consists of 9 industrial-grade CMOS cameras (model such as Basler acA1920-40uc), installed on a fixed platform with an X-Y dual-axis slide control system, with a slide control accuracy of 0.1mm. The slide moves along a preset path during shooting, and each camera sequentially collects the target. To improve shooting efficiency, a time-sharing trigger synchronization control method is used in actual deployment, i.e., 9 cameras cooperate with the electric slide to complete a complete collection cycle in batches and in order at 3x3 preset view points.

[0068] Take a typical indoor desktop scene as an example, the camera moves a total of 40 mm in two directions during the shooting process, and the sampling interval is 20 mm. Images are taken from 9 typical perspectives, such as the top left, middle, and bottom right. After collection, all images are uniformly sized, cropped, and color normalized as input for the subsequent encoding network.

[0069] The 9 images collected are input into an encoding network with structure preservation capability. The network aims to convert the image sequence into a four-dimensional tensor F LF (x,y,u,v), which represents the image response of each spatial position at each view angle.

[0070] The network structure consists of three layers. The first layer is a two-dimensional convolution layer with shared parameters, with a kernel size of 3x3, an input channel of 3, and an output channel of 64. It is used to extract primary edge and texture information independently for each image. All convolution layers use ReLU activation and are connected to a batch normalization module at the output to enhance feature stability.

[0071] The second layer is a multi-view feature fusion layer that combines features extracted from different views in a direction-aware manner. It outputs an intermediate feature tensor with direction-dependent characteristics. The weighting coefficients are calculated based on spatial brightness gradients and edge response intensity. For example, edge regions and depth change regions have larger brightness gradients and greater impact on the final hologram quality, so they are given higher weights α i,j (x,y), and the final fusion method is as follows:

[0072]

[0073] Where:

[0074] F LF (x,y,u,v) is the output light field tensor (also known as light field feature tensor), representing the feature response at spatial position (x,y) and view angle (u,v);

[0075] f conv (I i,j (x,y)) represents the result of shared two-dimensional convolution processing on each input image I i,j (x,y), with two layers of convolution, and channel numbers of 3→64→16, kernel size of 3x3; i,j represent view angle indices;

[0076] α i,j (x,y) is the fusion weight map, generated by normalizing the Sobel gradient map of the input image, which emphasizes edge and structure information.

[0077] The last layer is a compression mapping module, which uses 3D convolution (kernel size 3x3x3, input channel 64, output channel 16) to map and compress the fused perspective dimension into a tensor representation of fixed angle dimension, obtaining the final F LF (x, y, u, v) output tensor, where u, v e {1, 2, 3} represents the sampling results of 9 directions.

[0078] The entire tensor processing process does not change the true physical dimension of the image, because all input images have been normalized in preprocessing, the convolution kernel parameters are shared and have no unit meaning, so the final output tensor preserves the normalized feature expression. Convolution, weighting and projection are dimensionless operations, and the dimension is consistent throughout the process.

[0079] The output of this step is a light field tensor F LF (x, y, u, v), which contains the joint image structure information of space and angle under sparse perspectives, and the tensor format is 320x180x3x3, and the data precision is 32-bit floating point. This tensor will be directly used as the input of step S2 to participate in the subsequent holographic phase map generation process.

[0080] S2: Map the light field tensor in step S1 into a two-dimensional spatial phase map;

[0081] Further, the present step S2 comprises:

[0082] S21: Expand the light field tensor in step S1 in the angle dimension into a group of spatial feature maps, wherein each feature map in the spatial feature map is dynamically combined by a directional coupling residual module;

[0083] S22: Based on the spatial feature map, a non-constrained continuous phase map is obtained by regression;

[0084] S23: Calculate the directional gradient difference between the feature maps of each pair of adjacent perspectives, and integrate it into the phase prediction target as a penalty term to perform multi-objective structure optimization on the non-constrained continuous phase map obtained by regression, and output a two-dimensional spatial phase map; wherein, when outputting the two-dimensional spatial phase map, the optimization loss includes a directional coherence deviation term formed by the average value of the spatial deviation of the phase estimation between each pair of adjacent perspectives, so as to construct a structure penalty quantity from the light wave coherence of the holographic interferogram, aiming to reduce the difference between the predicted phases of different angles, thereby improving the coherence of the interference fringes, and further ensuring the holographic display quality under the condition of sparse angle acquisition.

[0085] Specifically, the present step is a holographic phase map generation under structure constraint; the goal of this step is to map the light field tensor F LF(x, y, u, v) is mapped to a two-dimensional phase map Φ(x, y), which provides physically loadable phase information for the subsequent SLM (Spatial Light Modulator) modulation link. The holographic system requires the final display image to accurately restore the three-dimensional structure of the original scene, so in this step we must ensure that the spatial and angular characteristics in the light field can be correctly converted into a phase map in the form of a coherent wavefront. Unlike conventional image transformation, this conversion is not just information compression, but involves a cross-physical domain mapping from the view-space to the phase modulation domain. This step proposes a phase map generation method that combines structure preservation and control of physical coherence, with an explicit multi-angle coupling mechanism and an adjustable directional coherence penalty term introduced in the loss structure, which is a specially designed innovative improvement to meet the requirements of holographic imaging.

[0086] The input is the light field feature tensor F output by step S1 LF (x, y, u, v), with a tensor size of 320x180x3x3, representing a joint image representation of spatial position (x, y) in 9 sparse angle (u, v) directions. All data has been normalized to the unit interval, and the format is a 32-bit floating-point tensor. This tensor is the only input source for the subsequent phase generation module, which contains spatial edge structure, angular disparity information and depth level features.

[0087] To map the above tensor to a two-dimensional phase map Φ(x, y), this embodiment designs a phase generation network with an angle coupling mechanism, which consists of an angle unfolding encoding module, a directional coupling residual fusion module and a structure constraint output module, respectively responsible for feature extraction, angle consistency modeling and optimization of the physical realizability of the phase map.

[0088] In the first stage, the angle unfolding encoding module will unfold F LF Unfolded into a set of spatial feature maps in the angle dimension Each F i is a 16-channel convolutional feature, extracted using a shared 2D convolutional network (two layers of convolution, convolution kernel 3x3, channel number 16→32, activation function LeakyReLU). Instead of being directly averaged or concatenated, the feature maps are dynamically combined through a directional coupling residual module.

[0089] In the second stage, the directional coupling residual module (a module for dynamically combining feature maps of different directions) constructs a directional consistency preservation mechanism: for each pair of adjacent viewing angles (i, j), the directional gradient difference of the corresponding feature map is calculated, and in the subsequent third stage it is used as a penalty term and incorporated into the phase prediction target. The specific phase map Φ raw (x, y) regression method is as follows:

[0090]

[0091] wherein:

[0092] Φ raw (x,y) is the regression obtained unconstrained continuous phase map;

[0093] represents the projection value of the feature map in the i-th direction after 1x1 convolution compression; specifically, it refers to the single-channel or low-dimensional feature response value obtained by linearly weighting and compressing the feature map in the channel dimension through the 1x1 convolution operation, which is used to represent the structural response in the direction, and is a two-dimensional matrix.

[0094] w i (x,y) is the direction weighting map, which is generated by the inter-view structural difference estimation module, and measures the contribution degree of different views to the phase, and a typical implementation is wherein, F i is the pixel value of the feature map in the i-th direction, is the average value of the pixels of all feature maps.

[0095] The adjacent views involved in the present scheme refer to the camera shooting angles adjacent in position in the light field, and the corresponding feature map is the representation of the adjacent direction. The adjacent view and the adjacent direction are equivalent concepts, which are expressed in the geometric acquisition space (view angle) and expressed in the feature map calculation (direction).

[0096] In the third stage, in order to make the phase map satisfy the spatial continuity and the view consistency, the embodiment introduces two physical constraint terms to constitute a joint optimization loss, and the loss is explicitly written into the final output formula:

[0097]

[0098] wherein:

[0099] Φ(x,y) is the final output phase map, the unit is radian, and the range is [0, 2π];

[0100] is a two-dimensional Laplace regularization term, which is used to limit the dramatic change of phase gradient and avoid high-frequency jitter;

[0101] D coh (x,y) is the direction coherence deviation term specially designed in the embodiment, which is defined as the difference of the direction gradient of the feature map corresponding to each pair of adjacent views, and is used to punish the inconsistent direction interference mode between views;

[0102] λ s and λ d are two hyperparameters for controlling the strength of the regularization, and in the preferred embodiment, a large number of experiments have verified that when λs With λ d When the ratio is approximately 1:3, a good balance can be achieved between smoothness and coherence constraints; the specific value of the hyperparameter can be fine-tuned according to the baseline distance of the actual acquisition system and the depth range of the target scene; in a specific embodiment, the values ​​are 0.1 and 0.3 respectively;

[0103] mod(·,2π) represents the phase range mapping required by the modulation stage.

[0104] The core innovation of this formula lies in the introduction of D. coh (x,y) is not a conventional image regularization term, but a structural penalty quantity constructed based on the optical wave coherence of the holographic interferogram. It aims to reduce the difference between the predicted phases at different angles, thereby improving the coherence of the interference fringes. This is a unique design specifically proposed to ensure the holographic display quality of this system under sparse angle acquisition conditions.

[0105] The pseudo-labels constructed using simulated reference interferograms are used during the network training phase. During the deployment phase, the phase map is calculated as a result of a forward inference, which has low time overhead and a clear structure.

[0106] The output of this step is a two-dimensional phase map Φ(x,y), a 32-bit floating-point matrix of 320×180. Each pixel corresponds to the phase control value of a modulation unit on the SLM, in radians, with a value range of [0, 2π]. This output provides the basis for subsequent SLM modulation map generation and is the direct physical mapping carrier of "holography as the representation medium" in this system.

[0107] S3: Convert the two-dimensional spatial phase diagram in step S2 into a modulation diagram for driving the spatial light modulator;

[0108] Furthermore, step S3 includes:

[0109] S31: A modulation resource weight map is generated by combining structural response and perceptual distribution. Specifically, the modulation focus area is determined by the intensity of structural change and human eye sensitivity, ensuring that the modulation distribution not only responds to the visual focus point but also preserves the interference texture structure.

[0110] S32: Adjust the quantization step size based on the modulation resource weight map, and use a non-uniform bit depth allocation mechanism to transform the two-dimensional spatial phase map in step S2 into a modulation map for driving the spatial light modulator. In this step, a joint non-uniform mapping strategy based on spatial visual sensitivity and local phase complexity is introduced. Through modulation region weight control and constraint penalty mechanism, the phase features of the human eye’s attention area are preferentially preserved when modulation resources are limited, thereby improving the subjective sharpness and interference quality of the final image.

[0111] Specifically, this step includes perception-guided SLM modulation map generation; the task of this step is to further transform the two-dimensional phase map Φ(x,y) generated in step S2 into a modulation map Φ that can be directly loaded by an SLM (spatial light modulator). SLM (i,j) and actively integrates the laws of human visual perception and the modulation capability limitations of display hardware during the conversion process. Unlike the uniform quantization strategy commonly used in image coding, this step introduces a joint non-uniform mapping strategy based on spatial visual sensitivity and local phase complexity in the modulation map generation stage. Through modulation region weight control and constraint penalty mechanism, it prioritizes the preservation of phase features of the regions of human visual interest when modulation resources are limited (such as fixed grayscale bit width and limited refresh rate), thereby improving the subjective sharpness and interference quality of the final image.

[0112] The input is the spatial phase map Φ(x,y) output from step S2, with dimensions of 320×180 radians and a range of [0,2π]. This map represents the phase distribution obtained after angle fusion and structural regularization optimization of the light field data, and already possesses physical significance for hologram reconstruction. In actual SLM driving, this phase map needs to be further converted into 8-bit or 10-bit integer modulation values ​​before being loaded into the SLM controller. Furthermore, a perceptual weight map W(x,y) is generated during system deployment. It is calculated based on a human eye-sensitive region prediction model (such as based on an MTF model combined with image frequency response simulation) and is used to guide the adaptive distribution of the modulation strategy in this step.

[0113] This step employs a two-stage processing mechanism: the first stage performs modulation distribution control, and the second stage generates the modulation map.

[0114] The first stage generates a modulation resource weight map R(x,y) by combining the structural response and perceived distribution. This weight map determines the modulation resource intensity (e.g., grayscale depth or refresh frequency weight) that should be allocated to each location. To simultaneously consider spatial structure and human visual perception, this scheme designs the following fusion strategy:

[0115]

[0116] in:

[0117] R(x,y) is the modulation resource weight map, normalized to [0,1];

[0118] It is the spatial phase gradient intensity, calculated using the Sobel operator;

[0119] W(x,y) is the perceptual weight map, which comes from the visual model trained during the deployment phase;

[0120] γ is a balance parameter, which controls the degree of fusion of structure and perception, and can be adaptively adjusted according to the specific configuration of the human eye perception model; a typical value is 0.6.

[0121] The formula embodies an innovation point of the patent system: the modulation key area is determined by the structural change intensity and the human eye sensitivity, ensuring that the modulation distribution not only responds to the visual focus point, but also retains the interference texture structure.

[0122] The second stage uses a non-uniform bit depth allocation mechanism to generate the final modulation map. In order to adapt to the fixed modulation level of the SLM (such as 8 bits for 256 levels), direct uniform quantization in the traditional method may result in insufficient accuracy in high complexity areas. Therefore, the present scheme proposes a mechanism based on R(x, y) adjusting quantization step:

[0123]

[0124] Wherein:

[0125] Φ SLM (i,j) is the output modulation map, which is an integer, and the value range is [0, N-1];

[0126] N is the SLM modulation level, such as N=256;

[0127] α is the local modulation enhancement coefficient, which controls the weight amplification ratio, and can be adaptively adjusted according to the bit depth of the SLM used; it is usually set to 0.3;

[0128] is the global mean value of R(x, y);

[0129] represents rounding down, which is used for discretization into SLM input format.

[0130] This design ensures that the modulation level does not overflow in the physical range, and the local area modulation accuracy is adaptively changed with R(x, y), so that high complexity and high attention area can be allocated more intensive gray scale, and other areas reduce gray scale coverage to save modulation energy and refresh resources.

[0131] The substantive breakthrough of this modulation method is that it changes the traditional fixed quantization strategy to a dynamic bit depth modulation strategy based on structure-perception weight map, which not only does not increase the total modulation complexity, but also improves the imaging quality of key areas, and is particularly suitable for 3D / VR display scenarios with limited display resources and obvious changes in user visual line concentration.

[0132] The output is the modulation map Φ SLM(i,j), size of 320x180, data type of integer matrix, each value corresponds to the phase modulation gray value that the SLM controller should load, ranging between [0, N-1] (for example [0, 255]). The figure will be directly sent into the SLM modulation module control hardware driver, which is the final control input before the physical reconstruction of the image.

[0133] S4: driving the spatial light modulator according to the modulation figure in step S3 to modulate the light wavefront, and reproducing the image of the target scene through optical diffraction.

[0134] Further, the present step S4 includes:

[0135] S41: obtaining a spatial light modulator response lookup table, converting the modulation figure in step S3 into an actual phase loading figure according to the spatial light modulator response lookup table, and driving the spatial light modulator to modulate the light wavefront based on the actual phase loading figure;

[0136] S42: calculating the light intensity figure on the observation surface based on the two-dimensional diffraction model, and reproducing the image of the target scene based on the light intensity figure.

[0137] Specifically, the present step includes SLM loading and physical image reproduction; wherein the target of the present step is to load the SLM modulation figure Φ SLM (i,j) generated in step S3 into the SLM device or simulation system, drive the laser irradiation and modulate the light wavefront, and generate the reproduced image I out (x,y) through optical diffraction propagation. As the final display link of the whole system, the present step no longer introduces modeling or modulation optimization mechanism, but directly converts the digital modulation figure into physical phase value, and calculates the image light intensity at the set spatial propagation distance. The scheme particularly considers the nonlinear characteristics of SLM response and the actual system configuration, realizes the modulation value to phase through the lookup table method, and carries out image reconstruction through the standard Fresnel diffraction formula, to ensure real deployment and reproduction.

[0138] The input is the modulation figure Φ SLM (i v ,j v ) output in step S3, which is an integer image with a size of 320x180, a gray level range of [0, 255], and a unit of dimensionless integer, representing the modulation level that each SLM pixel should load. At the same time, the present system presets an SLM response lookup table Γ(q), which is obtained through SLM factory calibration, and records the nonlinear mapping relationship from the gray value q to the actual phase value , wherein, (i v ,j v) is the pixel on the SLM plane. The laser is set to wavelength λ = 532 nm, and the propagation distance between the SLM and the observation plane is z = 0.6 m.

[0139] First, the integer modulation map Φ SLM (i v ,j v is converted to the actual phase loading map by table lookup:

[0140]

[0141] wherein:

[0142] represents the phase value loaded by each pixel position of the SLM, with the unit of radian;

[0143] Γ(·) is a 256-item lookup table, which is obtained by the SLM in the calibration experiment, and the phase change relationship (usually nonlinear, specifically the nonlinear mapping relationship from gray value q to actual phase value ) is obtained by inverse calculation of the voltage response curve;

[0144] If the lookup table position is not an integer, the intermediate value is obtained by linear interpolation, and the error is less than 0.01 rad.

[0145] Then, using the two-dimensional diffraction model based on Fresnel approximation, the light intensity map on the observation plane of the light wave propagated by the SLM modulation is calculated:

[0146]

[0147] wherein:

[0148] I out (x,y) is the gray value distribution of the final reproduced image, with the unit of light intensity;

[0149] is the complex amplitude of the wavefront generated by the SLM modulation;

[0150] F and F -1 are two-dimensional Fourier transform and inverse transform, respectively;

[0151] H(f x ,f y ) is the Fresnel transfer function, defined as λ is the wavelength of the laser, and z is the S Distance of LM plane to viewing plane ;

[0152] The frequency variable f x ,f y is the frequency domain coordinate grid corresponding to the spatial position (x, y), with the sampling range corresponding to the SLM pixel size, with the unit of 1 / mm.

[0153] The Fourier transform part can use a standard DFT algorithm (such as FFT) to run on a GPU or a high-performance CPU, and the simulation time is within 100 ms, meeting the real-time display requirements of the system.

[0154] The output is a two-dimensional image I out (x, y), which has the same size as the SLM (320x180), and each pixel represents the light intensity received at the corresponding position, with a unit of dimensionless normalized value (range [0, 1]). This image can be directly used for light field reconstruction display, and is the final image expression of the "holographic image" presented in front of the human eye in the embodiment of the application.

[0155] Embodiment one of the application proposes a light field acquisition, rendering and display method using holography as a characterization medium, which realizes a closed-loop link from a real scene to the final holographic display through four closely connected steps of dynamic sparse acquisition, phase map generation, perception-guided modulation and physical reconstruction. The application obtains a structure-preserved light field tensor by combining a controllable sparse camera array with an encoding mechanism at the acquisition end, effectively reducing the amount of redundant data; angle coupling and coherence constraints are introduced in the conversion link to ensure that the phase map maintains view consistency and structural continuity while ensuring the feasibility of physical modulation; in the modulation stage, a dynamic quantization allocation strategy guided by structural features and a human eye perception model enables limited SLM modulation resources to be concentrated in key areas, thereby improving display clarity without increasing hardware complexity; in the final image reconstruction stage, an actual observation image is generated by combining the nonlinear response of the SLM and the optical propagation model, ensuring consistency from digital control to physical imaging. Through the above closely linked solutions, the application effectively solves the problems of redundancy at the acquisition end, inefficiency in the conversion link, insufficient clarity in the display stage and isolation between links in the prior art, forming an efficient, compact and practical application-oriented holographic light field display system.

[0156] Embodiment two of the application discloses a light field display system using holography as a characterization medium, comprising:

[0157] The light field acquisition and encoding module is configured to acquire multi-view images of a target scene and perform encoding processing on the multi-view images to generate a light field tensor containing spatial and angular information; further, the specific steps and effects performed by the light field acquisition and encoding module are the same as those in step S1 of embodiment one, and will not be repeated here.

[0158] The holographic phase map generation module is connected with the light field acquisition and encoding module and is configured to map the light field tensor generated by the light field acquisition and encoding module into a two-dimensional spatial phase map; further, the specific steps and effects performed by the holographic phase map generation module are the same as those in step S2 of embodiment one, and will not be repeated here.

[0159] The spatial light modulator modulation map generation module is connected with the holographic phase map generation module and is configured to convert the two-dimensional spatial phase map generated by the holographic phase map generation module into a modulation map for driving a spatial light modulator. Further, the specific steps performed by the spatial light modulator modulation map generation module and the effects are the same as the content in step S3 in Embodiment 1, and will not be repeated here.

[0160] The image reproduction module is connected with the spatial light modulator modulation map generation module and is configured to drive the spatial light modulator to perform light wave front modulation according to the modulation map generated by the spatial light modulator modulation map generation module, and reproduce the image of the target scene through optical diffraction. Further, the specific steps performed by the image reproduction module and the effects are the same as the content in step S4 in Embodiment 1, and will not be repeated here.

[0161] Embodiment 3 of the present application discloses a computer readable storage medium, and the computer readable storage medium stores a computer program, wherein the computer program is arranged to be run by a processor to execute the steps of the light field display method taking holography as a representation medium in Embodiment 1.

[0162] Optionally, the computer readable storage medium can include but is not limited to a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store computer programs.

[0163] The background section of the present application can contain background information about the problems or environment of the present application, rather than the prior art described by others. Therefore, the content contained in the background section is not the admission of the prior art by the applicant.

[0164] The above description is further to the present application in conjunction with specific / preferred embodiments, and cannot be deemed as limiting the specific implementation of the present application to these descriptions. For those skilled in the art to which the present application belongs, without departing from the concept of the present application, they can make several substitutions or variations to the described embodiments, and these substitutions or variations shall be deemed as falling within the protection scope of the present application. In the description of the present application, the description of the terms "an embodiment", "some embodiments", "a preferred embodiment", "an example", "a specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are contained in at least one embodiment or example of the present application. In the present application, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. Furthermore, the skilled in the art can combine and combine the different embodiments or examples described in the present application and the features of the different embodiments or examples without contradiction. Although the embodiments of the present application and their advantages have been described in detail, it should be understood that various changes, substitutions and modifications can be made herein without departing from the scope defined by the appended claims.

Claims

1. A light field display method using holography as a representation medium, characterized by, The method comprises the following steps: S1: acquiring multi-view images of a target scene, and performing encoding processing on the multi-view images to generate a light field tensor containing spatial and angular information; S2: mapping the light field tensor in step S1 into a two-dimensional spatial phase map; S3: converting the two-dimensional spatial phase map in step S2 into a modulation map for driving a spatial light modulator; S4: driving the spatial light modulator to perform light wave front modulation according to the modulation map in step S3, and reproducing an image of the target scene through optical diffraction.

2. The light field display method using holography as a representation medium according to claim 1, wherein, S1 comprises: acquiring multi-view images of a target scene by using a sparse camera array, performing normalization preprocessing on the multi-view images, and then performing encoding processing on the multi-view images to generate a light field tensor containing spatial and angular information; preferably, the light field tensor containing spatial and angular information generated in S1 is: wherein: F LF (x,y,u,v) is the output light field tensor, expressing the feature response at spatial position (x,y) and view angle (u,v); f conv (I i,j (x,y)) represents the result of shared two-dimensional convolution processing on each input image I i,j (x,y) in the multi-view image; a i,j (x,y) is the fusion weight map.

3. The light field display method using holography as a representation medium according to claim 1, wherein, S2 comprises: S21: expanding the light field tensor in step S1 into a group of spatial feature maps in the angular dimension, wherein each feature map in the spatial feature map is dynamically combined through a directional coupling residual module; S22: based on the spatial feature map, an unconstrained continuous phase map is obtained by regression; S23: calculating the directional gradient difference between each pair of adjacent view feature maps and integrating it as a penalty term into the phase prediction target to perform multi-objective structure optimization on the unconstrained continuous phase map obtained by regression, and outputting a two-dimensional spatial phase map.

4. The light field display method using holography as a representation medium according to claim 3, wherein, The unconstrained continuous phase map obtained by regression in S22 is: wherein: Φ raw (x, y) is the regression obtained unconstrained continuous phase map; represents the projection value of the feature map in the i-th direction after convolution compression; w i (x, y) is the direction weighted map; The two-dimensional spatial phase map output in S23 is: where: Φ(x, y) is the final output phase map; is a two-dimensional Laplacian regularization term; D coh (x, y) is a directional coherence bias term, which refers to the directional gradient difference of the feature map corresponding to each pair of adjacent viewing angles; λ s and λ d are two hyperparameters that control the regularization strength; mod(·, 2π) represents the phase range mapping required by the modulation link.

5. The holography-based light field display method of claim 1, wherein, S3 comprises: S31: generating a modulation resource weight map through joint structure response and perception distribution; S32: adjusting the quantization step based on the modulation resource weight map, and converting the two-dimensional spatial phase map in step S2 into a modulation map for driving the spatial light modulator by using a non-uniform bit depth allocation mechanism.

6. The light field display method using holography as a representation medium according to claim 5, wherein, The modulation resource weight map generated in S31 is: where R(x, y) is a modulation resource weight map, normalized to [0, 1]; is the spatial phase gradient strength; W(x, y) is a perceptual weight map; and γ is a balance parameter. The modulation map for driving the spatial light modulator generated in S32 is: wherein: Φ SLM (i,j) is the output modulation map, the value range is [0, N-1]; N is the modulation level of the spatial light modulator; a is the local modulation enhancement coefficient; is the global mean value of R(x,y); denotes the floor function.

7. The holography-based light field display method of claim 1, wherein, S4 comprises: S41: obtaining a spatial light modulator response lookup table, converting the modulation map in step S3 into an actual phase loading map according to the spatial light modulator response lookup table, and driving the spatial light modulator to perform light wave front modulation based on the actual phase loading map; S42: based on a two-dimensional diffraction model, calculating the light intensity map on the observation surface of the light wave modulated by the spatial light modulator, and reproducing the image of the target scene based on the light intensity map.

8. The light field display method using holography as a characterization medium according to claim 7, wherein The actual phase loading map obtained in S41 is: wherein: represents the phase value loaded by each pixel position of the spatial light modulator; Γ(·) is the spatial light modulator response lookup table containing the nonlinear mapping relationship of the gray value to the actual phase value; The light intensity map on the observation surface of the light wave modulated by the spatial light modulator calculated in S42 is: where: I out (x,y) is the gray value distribution of the final reproduced image; is the complex amplitude of the wavefront modulated by the spatial light modulator; F and F -1 are the two-dimensional Fourier transform and inverse transform, respectively; H(f x ,f y ) is the Fresnel transfer function; the frequency variable f x ,f y is the frequency domain coordinate grid corresponding to the spatial position (x,y).

9. An optical field display system using holography as a representation medium, characterized by, It comprises: A light field acquisition and encoding module configured to acquire multi-view images of a target scene and perform encoding processing on the multi-view images to generate a light field tensor containing spatial and angular information; A holographic phase map generation module connected with the light field acquisition and encoding module and configured to map the light field tensor generated by the light field acquisition and encoding module into a two-dimensional spatial phase map; The spatial light modulator modulation map generation module is connected with the holographic phase map generation module and is configured to convert the two-dimensional spatial phase map generated by the holographic phase map generation module into a modulation map for driving a spatial light modulator. The image reproduction module is connected with the spatial light modulator modulation map generation module and is configured to drive the spatial light modulator to perform light wave front modulation according to the modulation map generated by the spatial light modulator modulation map generation module, and reproduce an image of the target scene through optical diffraction.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, wherein the computer program is arranged to be run by a processor to execute the light field display method taking holography as a representation medium according to any one of claims 1 to 8.

Images