Multi-depth hologram prediction method, device, equipment, medium and product

Abstract

The invention discloses a multi-depth hologram prediction method and device, equipment, a medium and a product, and relates to the field of holographic coding, and the method comprises the steps: obtaining a wide depth-of-field scene image; performing depth division on the wide-depth-of-field scene image, and determining a wide-depth-of-field hologram plane complex amplitude by using an angular spectrum theory; inputting the complex amplitude of the wide-depth-of-field hologram into a trained CNN model to obtain a wide-depth-of-field multi-depth hologram; the trained CNN model is obtained by training the CNN model by taking a narrow field depth hologram plane complex amplitude as input, taking a narrow field depth multi-depth hologram as output and taking a residual error between a reconstructed light field and the amplitude of an intensity image as a loss function; the intensity image is an intensity image of a narrow depth-of-field scene image. According to the method and the device, the prediction precision of the multi-depth hologram can be improved.

Description

Technical Field

[0001] The present application relates to the field of holographic coding, and in particular to a multi-depth hologram prediction method, device, equipment, medium and product. Background Art

[0002] Convolutional Neural Networks (CNNs) have developed powerful encoding capabilities through training, enabling them to efficiently encode diffraction fields into holograms. Specifically, when fed a diffraction field, CNNs can generate high-quality holograms by learning its intrinsic characteristics. However, in practice, CNNs are rarely applied to holographic encoding. Therefore, there is a need to apply CNNs to holography to improve the accuracy of hologram generation. Summary of the Invention

[0003] The purpose of this application is to provide a multi-depth hologram prediction method, device, equipment, medium and product, which can improve the prediction accuracy of multi-depth holograms.

[0004] To achieve the above objectives, this application provides the following solutions:

[0005] In a first aspect, the present application provides a multi-depth hologram prediction method, comprising:

[0006] Acquire scene images with a wide depth of field;

[0007] Dividing the wide depth of field scene image into depths, and determining the wide depth of field hologram plane complex amplitude using angular spectrum theory;

[0008] The complex amplitude of the wide depth of field hologram is input into a trained CNN model to obtain a multi-depth hologram with a wide depth of field; the trained CNN model is obtained by training the CNN model with the plane complex amplitude of the narrow depth of field hologram as input, the multi-depth hologram with a narrow depth of field as output, and the residual between the amplitude of the reconstructed light field and the intensity image as the loss function; the intensity image is the intensity image of the narrow depth of field scene image.

[0009] Optionally, the wide depth of field scene image is depth-divided, and the wide depth of field hologram plane complex amplitude is determined using angular spectrum theory, specifically including:

[0010] Performing depth division on the wide depth of field scene image to obtain a plurality of depth slices; the depth slices include a depth image and an intensity image;

[0011] Calculate the complex amplitude of the depth slice using angular spectrum theory;

[0012] The complex amplitude of the wide depth of field hologram plane is calculated using angular spectrum theory according to the complex amplitude and inverse diffraction distance of the depth slice; the inverse diffraction distance is the inverse diffraction distance from the depth slice to the hologram plane.

[0013] Optionally, the complex amplitude of the depth slice is expressed as:

[0014]

[0015] Among them, U n (x,y) represents the complex amplitude of the nth depth slice, U n-1 (x,y) represents the complex amplitude of the n-1th depth slice, Λ represents the angular spectrum diffraction process, Δz represents the diffraction distance, represents the amplitude of the intensity image, j is the imaginary unit, j 2 =-1, φ0 represents a constant phase, M n (x,y) corresponds to the nth depth slice S n Boolean mask of ASM Δz Represents the angular spectrum method for diffraction.

[0016] Optionally, the expression of the wide depth of field hologram plane complex amplitude is:

[0017]

[0018] Among them, U h (x,y) represents the wide depth of field hologram plane S h The complex amplitude at U N-1 (x,y) represents the complex amplitude of the N-1th depth slice, N represents the total number of depth slices, -z h -(N-1)Δz represents the slice from depth S N-1 To the hologram plane S h The inverse diffraction distance, Δz represents the diffraction distance, z h Denotes the depth slice S0 to the hologram plane S h distance.

[0019] Optionally, the training process of the CNN model specifically includes:

[0020] The complex amplitude of the narrow depth of field hologram plane is decomposed using Euler's formula to obtain the real channel and the imaginary channel;

[0021] Expanding the real channel and the imaginary channel using pixel decomposition to obtain an input matrix;

[0022] The input matrix is used to obtain a phase matrix using a CNN model;

[0023] The phase matrix and the phase grating are superimposed using a phase recombination operation to obtain a multi-depth hologram with a narrow depth of field;

[0024] For multi-depth holograms with narrow depth of field, the light field is reconstructed at different focal planes to obtain a focused light field;

[0025] Superimposing and reconstructing the light field of the target focal plane according to the focused light field to obtain a reconstructed light field;

[0026] Determining a loss function based on the residual of the intensity image of the reconstructed light field and the narrow depth of field scene image;

[0027] The loss function is used to update the parameters of the CNN model to obtain a trained CNN model; the CNN model is back-propagated through the gradient descent method.

[0028] Optionally, the trained CNN model includes a splicing module, a reorganization module, an initial convolution module, a pooling module, an intermediate convolution module, a transposed convolution module and an output convolution module connected in sequence; the transposed convolution module is also connected to the initial convolution module.

[0029] In a second aspect, the present application provides a multi-depth hologram prediction device, comprising:

[0030] An image acquisition module, used for acquiring scene images with a wide depth of field;

[0031] A depth division and complex amplitude calculation module, configured to perform depth division on the wide depth of field scene image and determine the plane complex amplitude of the wide depth of field hologram using angular spectrum theory;

[0032] The encoding module is used to input the complex amplitude of the wide depth of field hologram into a trained CNN model to obtain a multi-depth hologram with a wide depth of field; the trained CNN model is obtained by training the CNN model with the plane complex amplitude of the narrow depth of field hologram as input and the multi-depth hologram with a narrow depth of field as output, and the residual between the amplitude of the reconstructed light field and the intensity image as the loss function; the intensity image is the intensity image of the narrow depth of field scene image.

[0033] In a third aspect, the present application provides a computer device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement any of the multi-depth hologram prediction methods described above.

[0034] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the multi-depth hologram prediction methods described above.

[0035] In a fifth aspect, the present application provides a computer program product, comprising a computer program, which, when executed by a processor, implements any of the multi-depth hologram prediction methods described above.

[0036] According to the specific embodiments provided in this application, this application has the following technical effects:

[0037] The present application provides a multi-depth hologram prediction method, apparatus, equipment, medium and product. Before performing multi-depth hologram on a wide depth of field scene image, a trained CNN model is first obtained, wherein the CNN model is trained by plane complex amplitude of a narrow depth of field hologram and multi-depth hologram of a narrow depth of field. The CNN model uses narrow depth of field scene images during training and predicts wide depth of field scene images. This method can reduce the difficulty of obtaining training data and reduce training costs. In addition, high-precision prediction of wide depth of field scene images can be achieved by only using narrow depth of field scene images to train the CNN model, further improving the flexibility and adaptability of holographic coding. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0039] Figure 1 Schematic diagram of a flow chart of a multi-depth hologram prediction method in one embodiment of the present application;

[0040] Figure 2 This is a process diagram of a multi-depth hologram prediction method provided in one embodiment of the present application. DETAILED DESCRIPTION

[0041] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0042] The encoding capabilities of CNNs are somewhat similar to those of traditional holographic encoding methods, such as the Gerchberg-Saxton (GS) algorithm and the dual-phase method. This means that the encoding process is less dependent on the specific method used to obtain the diffraction field. Regardless of whether the diffraction field is calculated using point-based or plane-based methods, as long as the CNN is provided with sufficiently diverse learnable diffraction field features, it can be trained to efficiently encode any diffraction field.

[0043] Based on this theory, this application proposes an innovative training-generalization mechanism: if the near-field diffraction field contains sufficiently rich diffraction field features, then a CNN model can be trained under near-field conditions and enabled to make high-precision predictions under far-field conditions. This mechanism not only reduces the difficulty of obtaining training data but also significantly improves the model's generalization performance in practical applications, providing new ideas for the further development of holographic coding technology.

[0044] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0045] like Figure 1 As shown, the present application provides a multi-depth hologram prediction method including:

[0046] Step 101: Acquire a scene image with a wide depth of field.

[0047] Step 102: performing depth division on the wide depth of field scene image, and determining the plane complex amplitude of the wide depth of field hologram using angular spectrum theory.

[0048] Step 103: Input the complex amplitude of the wide depth of field hologram into the trained CNN model to obtain a multi-depth hologram with a wide depth of field; the trained CNN model is obtained by training the CNN model with the plane complex amplitude of the narrow depth of field hologram as input, the multi-depth hologram with a narrow depth of field as output, and the residual between the amplitude of the reconstructed light field and the intensity image as the loss function; the intensity image is the intensity image of the narrow depth of field scene image.

[0049] In step 102-step 103, a trained CNN model is first obtained, wherein the CNN model is trained using the plane complex amplitude of the narrow depth of field hologram and the multi-depth hologram of the narrow depth of field. The CNN model uses the narrow depth of field scene image during training and predicts the wide depth of field scene image. This method can reduce the difficulty of obtaining training data and reduce the training cost. In addition, high-precision prediction of the wide depth of field scene image can be achieved by only using the narrow depth of field scene image to train the CNN model, further improving the flexibility and adaptability of holographic coding.

[0050] In an exemplary embodiment, the wide depth of field scene image is depth-divided, and the complex amplitude of the wide depth of field hologram plane is determined using angular spectrum theory, specifically including: dividing the wide depth of field scene image into depth slices to obtain multiple depth slices; the depth slices include depth images and intensity images; the complex amplitudes of the depth slices are calculated using angular spectrum theory; the complex amplitude of the wide depth of field hologram plane is calculated using angular spectrum theory based on the complex amplitudes of the depth slices and the inverse diffraction distance; the inverse diffraction distance is the inverse diffraction distance from the depth slice to the hologram plane.

[0051] In practical applications, the complex amplitude of the depth slice is expressed as:

[0052]

[0053] Among them, U n (x,y) represents the complex amplitude of the nth depth slice, U n-1 (x,y) represents the complex amplitude of the n-1th depth slice, Λ represents the angular spectrum diffraction process, Δz represents the diffraction distance, represents the amplitude of the intensity image, j is the imaginary unit, j 2 =-1, φ0 represents a constant phase, M n (x,y) corresponds to the nth depth slice S n Boolean mask of ASM Δz Represents the angular spectrum method for diffraction.

[0054] In practical applications, the expression of the plane complex amplitude of the wide depth of field hologram is:

[0055]

[0056] Among them, U h (x,y) represents the wide depth of field hologram plane S h The complex amplitude at U N-1 (x,y) represents the complex amplitude of the N-1th depth slice, N represents the total number of depth slices, -z h -(N-1)Δz represents the slice from depth S N-1 To the hologram plane S h The inverse diffraction distance, Δz represents the diffraction distance, z h Denotes the depth slice S0 to the hologram plane S h distance.

[0057] In practical applications, step 102 is to calculate the multi-depth diffraction field, such as Figure 2 Taking a 3D strawberry scene as an example, diffraction recording and reconstruction are performed at a single wavelength. Assuming the scene depth is Z, it is evenly divided into N depth slices with a depth interval of Δz. The nth depth slice is represented by Sn (n∈[0,N-1]), such as Figure 2 (a) in the equation. Figure 2 (a) in the figure generates intensity and depth images (RGB-D) for a three-dimensional scene. Figure 2 (b) in the figure is a multi-depth diffraction field calculation diagram, such as Figure 2 (b) Based on the Angular Spectrum Method (ASM), the process of calculating the complex amplitude using ASM can be expressed as:

[0058]

[0059] Where Λ represents the angular spectrum diffraction process, and Δz represents the diffraction distance. represents the amplitude of the intensity image, j is the imaginary unit, j 2 =-1, φ0 indicates a constant phase. M n (x,u) corresponds to S n The mask is a black and white image, where the white area represents the object part belonging to the depth slice (value 1), and the black area represents the background (value 0) that does not belong to the layer. N-1 Represents n depth slice planes from front to back (closest to far from the camera / hologram). To calculate the nth depth slice S n The initial amplitude of the upper object part, I is the intensity of the original color image. is the amplitude. n =0 represents the initial phase assigned to the object on the nth depth slice, which is set to 0 in this application. (x,g) The multi-depth superimposed light field reconstructed for the hologram generated by CNN, usually intensity or amplitude, is the output of the model.

[0060] Symbol ASM Δz Represents the ASM used for diffraction, and the calculation process is: ASM{U n (x,y)} ΔZ =F -1 {F{U n (x,y)}·H Δz (f x ,f y )}, Fourier transform is represented by F, and inverse Fourier transform is represented by F -1 Indicates. H Δz (f x ,f y ) represents the transfer function, and the calculation formula is:

[0061]

[0062] in is the wave number, and Δp is the pixel pitch of the SLM, λ is the wavelength, and f x and f y is the spatial frequency component in the frequency domain, corresponding to the frequency of the original signal in the x and y directions respectively. By calculating layer by layer, the hologram plane S is obtained. h The complex amplitude U at h (x,y), the calculation process is expressed as: Where (-z h -(N-1)Δz) is the depth slice S N-1 To the hologram plane S h The inverse diffraction distance of the complex amplitude U h (x,y) can be used to encode multi-depth holograms with the assistance of CNN.

[0063] In an exemplary embodiment, the training process of the CNN model specifically includes: decomposing the complex amplitude of the narrow depth of field hologram plane using Euler's formula to obtain a real channel and an imaginary channel; expanding the real channel and the imaginary channel using pixel decomposition to obtain an input matrix; using the input matrix to obtain a phase matrix using the CNN model; superimposing the phase matrix and the phase grating using a phase recombination operation to obtain a multi-depth hologram with a narrow depth of field; reconstructing the light field on different focal planes for the multi-depth hologram with a narrow depth of field to obtain a focused light field; superimposing and reconstructing the light field of the target focal plane based on the focused light field to obtain a reconstructed light field; determining a loss function based on the residual of the reconstructed light field and the intensity image of the narrow depth of field scene image; using the loss function to update the parameters of the CNN model to obtain a trained CNN model; and backpropagating the CNN model through the gradient descent method.

[0064] In an exemplary embodiment, the trained CNN model includes a splicing module, a reorganization module, an initial convolution module, a pooling module, an intermediate convolution module, a transposed convolution module and an output convolution module connected in sequence; the transposed convolution module is also connected to the initial convolution module.

[0065] At the input layer, the CNN model receives a complex image data consisting of real and imaginary parts, with eight input channels. The concatenation module of the CNN model first concatenates the real and imaginary parts of the input along the channel dimension. The reassembly module then reorganizes the image data using a pixel unshuffle operation, which increases the number of channels but reduces spatial resolution.

[0066] At the feature extraction layer, the CNN model consists of two main stages. The first stage uses an initial convolutional module, which includes a 3x3 convolution, batch normalization, and a LeakyReLU activation function to convert the eight input channels into four feature channels. In the second stage, the CNN model's pooling module uses both max pooling and average pooling for downsampling. The results of these two pooling methods are concatenated to form an eight-channel feature map. These features are then processed by an intermediate convolutional module, reducing the number of channels back to four.

[0067] In the feature fusion layer, the CNN model uses a transposed convolution module to upsample the features from the second stage. This upsampling result is then fused with the features from the first stage to form a richer feature representation. This design embodies the principle of skip connections and helps preserve more image detail.

[0068] At the output layer, the CNN model's output convolution module uses a 1x1 convolution operation to integrate all features, adjusting the number of channels to the required 4. Finally, a pixel shuffle operation is performed to adjust the spatial structure of the feature map to obtain the final output.

[0069] The output layer uses a 1×1 convolution operation to process the previously concatenated feature maps. This convolution layer receives 8-channel input features and then maps them to 4-channel output features. The 1×1 convolution plays the role of a "feature integrator" here. It effectively fuses feature information from different sources by learning linear combinations between channels, while reducing the number of channels to the required number of output channels. Subsequently, the network applies the PixelShuffle operation to reorganize the convolution output. PixelShuffle is a special pixel rearrangement operation that redistributes information from the channel dimension to the spatial dimension. Specifically, it reassigns r 2 Each pixel of the four channels is reorganized into a single-channel pixel of r×r area. In this network, the upsampling factor is 2, which means that the feature map of the four channels is reorganized into a feature map of one channel but with a spatial resolution increased by 4 times (width and height increased by 2 times).

[0070] The unique design of this CNN model lies in its integration of multiple pooling methods and multi-scale feature extraction. By using both max pooling and average pooling, the CNN model is able to capture different types of image features. The skip connection structure helps the CNN model preserve original details while transforming features. This design makes the CNN model particularly suitable for transforming and encoding complex image data.

[0071] In practical applications, the training of CNN models specifically includes the following aspects:

[0072] Data input processing: The calculated hologram plane complex amplitude U h (x, y) serves as the input to the CNN. First, the complex amplitude is decomposed into real and imaginary parts using Euler's formula, forming two channels. Next, these two channels are expanded to eight channels using a "pixel decomposition" operation, ultimately generating a 1920 × 1080 × 8 matrix as the input to the CNN.

[0073] Figure 2 (c) in the figure is the process diagram of CNN encoding pure phase hologram, as shown in Figure 2 As shown in (c) of Figure 1, the network structure and output processing: The CNN model consists of three convolutional modules, each followed by a specific pooling and upsampling operation. Skip connections are used between modules to enhance the transmission effect of feature maps. The output of the CNN is a 1920×1080×4 phase matrix. Through the "phase reorganization" operation, the phase matrix is combined with the following Figure 2 The phase gratings shown in (d) are superimposed to generate multi-depth holograms. CNN uses L1 loss (mean absolute error) as the loss function LOSS, where α represents the normalization coefficient multiplied by the reconstruction result R(x,y) before calculating the L1 loss. The purpose is to align the overall brightness of the reconstruction result with the target image. N =hologram represents the phase information finally output by the CNN model, i.e. the calculated phase hologram. The subscript N indicates that the information of all N depths is encoded. By simulating light to illuminate the phase hologram (φ′ N +grating), and at different reconstruction distances (corresponding to the depth of the original scene) and the calculated reconstructed image plane, ideally, the contents of the corresponding depth in the original scene should be clearly displayed.

[0074] Figure 2 (d) in the figure is a target focused stack image generated by holographic reconstruction of multi-depth diffraction fields, as shown in Figure 2 As shown in (d), holographic reconstruction of multi-depth diffraction fields: reconstructing the focused light field R at different focal planes m (x, y), the calculation process involves the forward diffraction formula and related parameters, such as reconstruction distance, complex amplitude of plane wave source, Boolean mask, etc. Get the focused light field R of each focal plane m (x, y), the reconstructed light field R(x, y) is obtained by superimposing and reconstructing the light field of the target focal plane.

[0075] Loss Function and Training: The L1 loss function (mean absolute error) is used to measure the residual between the reconstructed light field and the actual intensity image amplitude, and this residual is used to update the CNN model parameters. Specifically, the intensity residual of the input RGB image is used as the loss function. The model uses gradient descent to perform backpropagation, gradually optimizing the encoding quality of the phase hologram. The hologram encoding process is the process of generating the hologram using the CNN model.

[0076] Step 103 is a generalized prediction process for wide depth of field in practical application. Specifically, the scene with a depth of field of Y (Y>Z) is divided into N depth slices, and the complex amplitude U of the hologram plane is obtained by calculation. h (x,y) and input it into the trained CNN model to generate multi-depth holograms.

[0077] The method presented in this application uses a convolutional neural network to perform narrow depth of field training and generalized wide depth of field holographic encoding. Its core innovation lies in the mechanism of training in the near diffraction field and generalizing in the far diffraction field. This mechanism leverages training data from a narrow depth of field and the generalization capability from near-field diffraction to far-field diffraction, enabling the CNN model to make high-precision predictions of wide depth of field holograms. This approach not only reduces training costs but also significantly improves the flexibility and applicability of holographic encoding.

[0078] This application uses a dataset containing a 20mm depth of field to train a CNN encoding model. Holograms with depths of field of 20mm, 40mm, and 80mm are optically reconstructed. The model's performance is evaluated by observing the focusing and defocusing effects of the optical reconstructions, as well as the clarity of pixel edges in the focused state.

[0079] Experimental results show that the proposed CNN model can achieve precise focus across a wide depth of field range of 20mm, 40mm, and 80mm, fully demonstrating its superior performance in depth of field extension and generalization. Specifically, the model effectively maintains the clarity and detail of the reconstructed image, particularly in the focused area, where pixel edges are sharp and detail is highly restored. This result not only verifies the model's robustness in depth of field generalization but also provides strong support for the flexibility and adaptability of holographic imaging technology in practical applications.

[0080] Based on the same inventive concept, embodiments of the present application also provide a multi-depth hologram prediction device for implementing the multi-depth hologram prediction method described above. The solution provided by this device is similar to the solution described in the method described above. Therefore, the specific limitations of one or more of the multi-depth hologram prediction device embodiments provided below can be found in the limitations of the multi-depth hologram prediction method described above and will not be repeated here.

[0081] In an exemplary embodiment, a multi-depth hologram prediction apparatus is provided, comprising:

[0082] The image acquisition module is used to acquire scene images with a wide depth of field.

[0083] The depth division and complex amplitude calculation module is used to perform depth division on the wide depth of field scene image and determine the plane complex amplitude of the wide depth of field hologram using angular spectrum theory.

[0084] The encoding module is used to input the complex amplitude of the wide depth of field hologram into a trained CNN model to obtain a multi-depth hologram with a wide depth of field; the trained CNN model is obtained by training the CNN model with the plane complex amplitude of the narrow depth of field hologram as input and the multi-depth hologram with a narrow depth of field as output, and the residual between the amplitude of the reconstructed light field and the intensity image as the loss function; the intensity image is the intensity image of the narrow depth of field scene image.

[0085] In an exemplary embodiment, a computer device is provided, which may be a server or a terminal. The computer device includes a processor, a memory, an input / output interface (I / O) and a communication interface. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface is connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store multi-depth hologram prediction data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a multi-depth hologram prediction method is implemented.

[0086] Those skilled in the art will understand that the structure shown in this application is merely a block diagram of a portion of the structure related to the solution of this application and does not constitute a limitation on the computer device to which the solution of this application is applied. A specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement. In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the above-mentioned method embodiments when executing the computer program.

[0087] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, which implements the above-mentioned method embodiments when executed by a processor.

[0088] In an exemplary embodiment, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the above method embodiments are implemented.

[0089] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0090] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0091] The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processors involved in the various embodiments provided herein may include, but are not limited to, general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic units, data processing logic units based on quantum computing, and the like.

[0092] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0093] This document uses specific examples to illustrate the principles and implementation methods of this application. The description of the above examples is only intended to help understand the method and core concept of this application. At the same time, for those skilled in the art, based on the concept of this application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A multi-depth hologram prediction method, characterized in that: The multi-depth hologram prediction method comprises: Acquire scene images with a wide depth of field; Dividing the wide depth of field scene image into depths, and determining the wide depth of field hologram plane complex amplitude using angular spectrum theory; The complex amplitude of the wide depth of field hologram is input into a trained CNN model to obtain a multi-depth hologram with a wide depth of field; the trained CNN model is obtained by training the CNN model with the plane complex amplitude of the narrow depth of field hologram as input, the multi-depth hologram with a narrow depth of field as output, and the residual between the amplitude of the reconstructed light field and the intensity image as the loss function; the intensity image is the intensity image of the narrow depth of field scene image.

2. The multi-depth hologram prediction method according to claim 1, characterized in that: The wide depth of field scene image is depth-divided, and the wide depth of field hologram plane complex amplitude is determined using angular spectrum theory, specifically including: Performing depth division on the wide depth of field scene image to obtain a plurality of depth slices; the depth slices include a depth image and an intensity image; Calculate the complex amplitude of the depth slice using angular spectrum theory; The complex amplitude of the wide depth of field hologram plane is calculated using angular spectrum theory according to the complex amplitude and inverse diffraction distance of the depth slice; the inverse diffraction distance is the inverse diffraction distance from the depth slice to the hologram plane.

3. The multi-depth hologram prediction method according to claim 2, characterized in that: The expression of the complex amplitude of the depth slice is: Among them, U n (x,y) represents the complex amplitude of the nth depth slice, U n-1 (x,y) represents the complex amplitude of the n-1th depth slice, Λ represents the angular spectrum diffraction process, Δz represents the diffraction distance, represents the amplitude of the intensity image, j is the imaginary unit, j 2 =-1, φ0 represents a constant phase, M n (x,y) corresponds to the nth depth slice S n Boolean mask of ASM Δz Represents the angular spectrum method for diffraction.

4. The multi-depth hologram prediction method according to claim 1, wherein: The expression of the plane complex amplitude of the wide depth of field hologram is: Among them, U h (x,y) represents the wide depth of field hologram plane S h The complex amplitude at U N-1 (x,y) represents the complex amplitude of the N-1th depth slice, N represents the total number of depth slices, -z h -(N-1)Δz represents the slice from depth S N-1 To the hologram plane S h The inverse diffraction distance, Δz represents the diffraction distance, z h Denotes the depth slice S0 to the hologram plane S h distance.

5. The multi-depth hologram prediction method according to claim 1, characterized in that: The training process of the CNN model specifically includes: The complex amplitude of the narrow depth of field hologram plane is decomposed using Euler's formula to obtain the real channel and the imaginary channel; Expanding the real channel and the imaginary channel using pixel decomposition to obtain an input matrix; The input matrix is used to obtain a phase matrix using a CNN model; The phase matrix and the phase grating are superimposed using a phase recombination operation to obtain a multi-depth hologram with a narrow depth of field; For multi-depth holograms with narrow depth of field, the light field is reconstructed at different focal planes to obtain a focused light field; Superimposing and reconstructing the light field of the target focal plane according to the focused light field to obtain a reconstructed light field; Determining a loss function based on the residual of the intensity image of the reconstructed light field and the narrow depth of field scene image; The loss function is used to update the parameters of the CNN model to obtain a trained CNN model; the CNN model is back-propagated through the gradient descent method.

6. The multi-depth hologram prediction method according to claim 1, characterized in that: The trained CNN model includes a splicing module, a reorganization module, an initial convolution module, a pooling module, an intermediate convolution module, a transposed convolution module and an output convolution module connected in sequence; the transposed convolution module is also connected to the initial convolution module.

7. A multi-depth hologram prediction device, characterized in that: The multi-depth hologram prediction device comprises: An image acquisition module, used for acquiring scene images with a wide depth of field; A depth division and complex amplitude calculation module, configured to perform depth division on the wide depth of field scene image and determine the plane complex amplitude of the wide depth of field hologram using angular spectrum theory; The encoding module is used to input the complex amplitude of the wide depth of field hologram into a trained CNN model to obtain a multi-depth hologram with a wide depth of field; the trained CNN model is obtained by training the CNN model with the plane complex amplitude of the narrow depth of field hologram as input and the multi-depth hologram with a narrow depth of field as output, and the residual between the amplitude of the reconstructed light field and the intensity image as the loss function; the intensity image is the intensity image of the narrow depth of field scene image.

8. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the multi-depth hologram prediction method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the multi-depth hologram prediction method according to any one of claims 1 to 6 is implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the multi-depth hologram prediction method according to any one of claims 1 to 6 is implemented.

Images