A digital holographic compression transmission method using a quantum compensation hybrid neural network

Abstract

The application provides a digital holographic compression transmission method using a quantum compensation hybrid neural network, an original image is recorded into a digital hologram through a Fresnel off-axis holographic algorithm, a quantum compensation hybrid neural network is constructed and trained to compress and decompress the digital hologram, quantization and dequantization are performed by using an 8-bit quantization and an adding random noise method, compression data is encoded and decoded through Huffman coding, and finally a recovered hologram is reproduced to obtain a reproduction image. The provided method has faster parallel processing speed and better image recovery quality, is suitable for the calculation and processing of the large data volume of the digital hologram, and can complete the training of the compression network structure in fewer training times, thereby improving the compression transmission speed of the digital hologram.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of optical information processing, and relates to a compression and transmission method of computer-generated hologram (digital hologram) using a quantum compensation hybrid neural network, specifically including methods of digital hologram image production and reproduction, digital hologram compression scheme using quantum compensation hybrid neural network, information quantization encoding and decoding, etc. BACKGROUND

[0002] Holographic display can reconstruct the entire light field of a three-dimensional scene and has the potential to provide all depth cues that can be perceived by the human eye. In three-dimensional display technology, holographic projection technology is considered as the most ideal way to achieve three-dimensional display because it can completely record the amplitude and phase information of a three-dimensional object and can perfectly reproduce a three-dimensional image identical to the original object under certain conditions. Therefore, holographic display can be considered as a promising candidate for future three-dimensional display technology. In recent years, thanks to the progress of optical, electronic and computer technologies and the continuous development of new algorithms, computer holography technology has developed rapidly. Computer-generated hologram (CGH) is a technology that converts a three-dimensional (3D) object scene into a two-dimensional (2D) complex-valued hologram. The computer-generated hologram contains a large amount of original object information, including the amplitude and phase information of the object, and the bandwidth requirement of a 3D holographic display system reaches 100 Gbps-1Tbps. In order to more effectively transmit the huge information data carried by the interference fringes on the computer-generated hologram, compression of the hologram is particularly important.

[0003] Traditional image compression techniques and schemes are already very mature, such as JPEG and other standard-based methods. However, computer-generated holograms are composed of interference fringes and contain more high-frequency information than natural images, so the performance of JPEG and other standard-based methods is severely limited in hologram compression [1] . Therefore, with the continuous development of artificial intelligence, deep learning technology has shown unexpected ability in hologram calculation and generation, and document two [2] uses a data-driven deep learning method to compress holograms and achieves good results. However, hologram compression schemes based on deep learning methods generally have slow model convergence speed, limited memory capacity, and poor recovery effect of holograms at high compression ratios, which to some extent limit the effect of hologram compression transmission and the value of practical application. SUMMARY

[0004] In order to overcome the above-mentioned deficiencies of the prior art, the present application provides an end-to-end computer-generated hologram compression and transmission method, and the present application designs a quantum compensation-based hybrid neural network to compress and transmit the computer-generated hologram, the network can complete convergence in fewer training times, and good recovery effect can be obtained under high compression ratio, greatly improving the compression and transmission efficiency of the computer-generated hologram.

[0005] The principle of the present application is that the quantum compensation-based hybrid neural network technology is used for the compression and transmission of the computer-generated hologram, since the computer-generated hologram is an image composed of interference fringes, each pixel carries part of the information of other pixels in the spatial domain, and the information amount carried by the whole image is large; in quantum computing theory, quantum bits have faster parallel processing speed and stronger data storage capacity through quantum state superposition and entanglement, and the information represented by the quantum bits can be characterized by amplitude and phase in the complex domain, which is consistent with the information of the amplitude and phase of the object contained in the computer-generated hologram, and embedding the information of the amplitude and phase of the object into the quantum bits for calculation can increase the information capacity of the whole network and the high-order characteristics of the complex domain. In addition, according to the basic principle of quantum computing, an n-bit quantum register can simultaneously store 2 n n binary numbers, and the exponentially growing storage capacity of the quantum computing system can process all 2 n n numbers in parallel, and one operation is equivalent to 2 n n operations of classical conventional computing. Therefore, the quantum compensation-based hybrid neural network has better effect than the classical neural network in processing the compression of the computer-generated hologram.

[0006] The technical scheme provided by the present application is:

[0007] A digital holographic compression and transmission method using a quantum compensation hybrid neural network, including the production and reproduction of computer-generated holograms for simulating optics, compression and decompression based on quantum compensation hybrid neural networks, information encoding and transmission based on Huffman lossless encoding, etc., has faster parallel computing speed and stronger information storage capacity, fewer iteration times are required for convergence, high compression ratio, fast transmission speed, good image quality effect of reproduction, and is suitable for the calculation and processing of computer-generated holograms with large information amount; comprising the following steps:

[0008] 1) The image set containing the original 3D object is recorded and generated into a computer-generated hologram data set by the Fresnel off-axis holographic calculation method;

[0009] 2) Construct a hybrid neural network based on quantum compensation, the network is composed of compression block and decompression block, model training stage joint training, model test or service stage will be two modules apart, respectively, in the transmission system of the sending end and the receiving end; using the computer generated hologram data set obtained in step 1) into the hybrid neural network based on quantum compensation for training, get the training of hybrid neural network based on quantum compensation, the network is split into compression block and decompression block, wherein the compression block is placed in the sending end, and the decompression block is placed in the receiving end;

[0010] 3) the holographic floating point matrix obtained by compressing the compression block of the hybrid neural network based on quantum compensation in step 2) is quantized to an integer value between 0 and 255 by 8 bits, and then the quantized matrix is encoded into a binary bit stream by using the Huffman lossless coding algorithm for transmission;

[0011] 4) after receiving the bit stream sent by the sending end in step 3) at the receiving end of the transmission system, the integer matrix is obtained by decoding through the Huffman decoding algorithm, and then the floating point matrix is obtained by adding the quantization noise generated by the uniform distribution to the integer matrix, and the floating point matrix is decompressed by the decompression block of the hybrid neural network based on quantum compensation trained in step 2) to obtain the recovered hologram; to

[0012] 5) the hologram obtained by decompressing in step 4) is reproduced to obtain the original object reproduction image.

[0013] Further, the holographic recording calculation method of step 1) includes Fresnel diffraction and off-axis reference light interference;

[0014] The approximate formula of the Fresnel diffraction is calculated as formula 1:

[0015]

[0016] In formula 1, k is the wave number, j is the imaginary unit, (x0, y0) represents the object wave plane, (x, y) is the corresponding diffraction plane; U0(x0, y0) is the information of the original object, U(x, y) is the complex amplitude function of the object light wave, and z represents the diffraction distance. The Fresnel diffraction integral is calculated by using a computer, which can be quickly Fourier transformed by formula 2:

[0017]

[0018] ​In formula 2, F represents a Fourier transform, z is a diffraction distance, k is a wave number, j is an imaginary unit, L0 is an initial object wave plane sampling range, the number of sampling points of the initial object wave plane is N*N, that is, the number of pixels of the initial image, the interval of the initial object wave plane sampling points is Δx0=Δy0=L0 / N, Δx=Δy is a sampling interval after the initial object wave plane is discretely Fourier transformed, and in the spatial domain, the sampling interval corresponds to that of a diffraction plane.

[0019] The off-axis reference light interference calculation method records a wave front complex amplitude function to form a hologram I(x, y) by using the interference principle of light, as shown in formula 3:

[0020]

[0021] In formula 3, I(x, y) is the light intensity recorded by the hologram, R(x, y) is a wave front function of a reference wave, and U(x, y) is a wave front function of an original image. * (x, y) and U * (x, y) correspond to the conjugate of R(x, y) and U(x, y) respectively.

[0022] Further, the quantum compensation-based hybrid neural network in step 2) is composed of a compression block and a decompression block, wherein the compression block is composed of three classical convolution layers, wherein the first convolution layer and the third convolution layer have the same image matrix dimension of the input and the output, and only have the function of feature extraction; the second convolution layer aggregates and compresses the input image matrix by adjusting the convolution stride, so that the ratio of the input matrix size to the output matrix size of the second convolution layer (i.e., the compression ratio) is the square of the convolution stride. The input of the entire compression block is denoted as X generated , and the output is denoted as B compress ; the floating point matrix B compress output by the compression block is output to the decompression block for calculation, and the decompression block is composed of two links (a main link and a compensation link). The compensation link performs down-sampling on the matrix B compress to obtain the matrix B quantum , and obtains the matrix B compensation through calculation by a quantum compensation module and through an up-sampling module; the main link obtains the matrix B upsampling through up-sampling calculation, introduces a quantum compensation coefficient α, obtains a weighted result (1-α)×B upsampling +α×B compensation after quantum compensation, and obtains the final restored output result X recovered after a plurality of convolution layers and residual errors.

[0023] wherein the quantum compensation module is composed of an embedding layer, an entanglement layer and a measurement layer. The quantum compensation module is inputted with 4 qubits in |0> state as initial state, and set as |ψ> 1，...，d wherein d = 4; |ψ> 1，...，d can be represented as a superposition state of 2 d computational basis states in a d-dimensional Hilbert space. It can be represented as formula 4:

[0024]

[0025] In formula 4, H 2 represents a two-dimensional Hilbert space with basis set {|0>, |1>}. Therefore, |ψ> 1，...，d can be expressed as formula 5:

[0026]

[0027] In formula 5, a i , i∈{1,..., 2 d} represents a complex amplitude assigned to the computational basis state .

[0028] The computation of the entire quantum compensation module includes the following three computation steps:

[0029] 221) input the above-mentioned input qubit state |ψ>1, ...，d into the embedding layer of the quantum compensation module, specifically, the input qubit state |ψ> 1，...，d is first calculated by a Hadamard gate, and then the classical data is embedded in the form of phase into the RY gate to output a new quantum state

[0030] 222) input the quantum state outputted in step 221) into the entanglement layer of the quantum compensation module for calculation, and obtain the final result by quantum entanglement on different quantum circuits through a series of unitary operators U; a d-bit U gate can be represented as U d (Θ), wherein Θ is a set of all free parameters of the U gate; these gate levels are cascaded to form a quantum neural network, which is similar to a classical deep neural network. The sequential cascade of L unitary gates can be represented as formula 6:

[0031]

[0032] In formula 6, represents the unitary gate of the i-th layer, and Θ = {Θ1,..., Θ L} is a set of all free parameters of the unitary gate.

[0033] The input of the entanglement layer Quantum neural network representation through entanglement layers The entanglement layer output is

[0034] 223) Output in step 222) The measurement layer of the quantum compensation module is sent to measure, and the final output result of the entire quantum compensation block is obtained, and the compensation matrix B described in step 22) is obtained by upsampling. compensation .

[0035] Among them, the measurement result of the Pauli operator for the i-th quantum bit is Equation 7:

[0036]

[0037] In formula 7, y i represents the measurement result of the i-th quantum, and Z is the Pauli matrix.

[0038] Furthermore, the training step of the entire hybrid neural network based on quantum compensation in step 2) includes the following steps:

[0039] 231) dividing the computer-generated hologram data set generated in step 1) into several batches and sequentially inputting them into a hybrid neural network based on quantum compensation to obtain the forward propagation calculation results of the network;

[0040] 232) Record the error between the actual output of the entire network and the target output, and adjust the parameters of each layer through the back propagation of the neural network according to the gradient descent method;

[0041] 233) Repeat steps 231) to 232) until the network training rounds reach the set number of times.

[0042] Furthermore, in step 5), the restored hologram is reproduced, specifically according to the Fresnel off-axis holography principle, through two processes of off-axis reference light illumination and Fresnel diffraction, to achieve the reconstruction of the restored hologram into the original three-dimensional object image;

[0043] According to the off-axis holographic reconstruction formula, that is, Formula 8, the hologram I(x, y) is reconstructed using the reconstruction light R(x, y) that is consistent with the reference light:

[0044]

[0045] In Equation 8, I(x, y) is the light intensity recorded by the hologram, R(x, y) is the wavefront function of the reference light and the reproduced light, and U(x, y) is the reproduced wavefront function of the original image. *(x, y) is a conjugate term of the reconstruction image;

[0046] The diffraction process of the computer-generated hologram light wave irradiated by the reconstruction light is calculated by the Fresnel diffraction formula (formula 1~formula 2), and the three-dimensional reconstruction image of the original object is obtained in the diffraction plane.

[0047] Compared with the prior art, the present application has the following advantages:

[0048] The present application provides a kind of digital holographic compression transmission method using quantum compensation hybrid neural network, by proposing the mechanism of quantum compensation, it can assist the convergence of less training rounds in classical neural network, and the recovery effect is better under high compression ratio, greatly reduce the data amount of processing of compression transmission system.For the compression transmission task of computer-generated hologram, the ability of quantum parallel computing is verified.Finally, a 3D information transmission system of end-to-end is realized, which inputs three-dimensional object image in sending end and outputs three-dimensional object image in receiving end.

[0049] The advantages of the present application mainly include the following aspects:

[0050] (I) the hybrid neural network based on quantum compensation proposed in the present application can well compensate the defects of classical neural network in processing large data volume of computer-generated hologram by virtue of the faster parallel processing speed and stronger data storage capacity of quantum computing in quantum compensation module, and is more suitable for compression and transmission of large data volume of computer-generated hologram;

[0051] (II) for the same data set, the hybrid neural network based on quantum compensation can complete convergence in less training rounds compared with traditional neural network, thereby improving the speed of transmitting holographic information;

[0052] (III) the image recovery quality of the hybrid neural network based on quantum compensation of the present application under high compression ratio is obviously better than that of traditional neural network without quantum compensation, and the compression ratio of the network can be adjusted according to the compression level. BRIEF DESCRIPTION OF DRAWINGS

[0053] Figure 1 is the flow framework diagram of the compression and transmission system of computer-generated hologram based on the hybrid neural network of quantum compensation provided by the present application;

[0054] Figure 2 is a schematic diagram of computer-generated hologram recording and reconstruction, wherein (a) is the hologram recording process, wherein I is the light intensity of hologram recording, R is the wavefront function of reference wave, and U is the wavefront function of original image. * and U * correspond to the conjugate terms of U and R respectively; wherein (b) is the hologram reconstruction process;

[0055] Figure 3 is an architecture schematic diagram of the hybrid neural network based on quantum compensation;

[0056] Figure 4 is a schematic diagram of the representation of a quantum bit on a Bloch sphere;

[0057] Figure 5 is a schematic diagram of the quantum gate circuit of the quantum compensation module;

[0058] Figure 6 is a schematic diagram of the embedding of classical data into the quantum compensation module;

[0059] Figure 7 is a schematic diagram of Huffman coding and floating-point matrix quantization and recovery; wherein (a) is a schematic diagram of the basic principle of Huffman coding and decoding, (b) is a schematic diagram of the basic principle of floating-point matrix quantization and recovery;

[0060] Figure 8 is a comparison diagram of the hologram and the reconstructed reproduction of the baboon image in the Set14 dataset under seven different compression ratios without quantum compensation of the traditional neural network and the hybrid neural network based on quantum compensation;

[0061] Figure 9 is an evaluation on the Set14 dataset, wherein (a) is a comparison schematic diagram of the traditional neural network without quantum compensation and the hybrid neural network based on quantum compensation on the PSNR index with the change of bpp; (b) is a comparison schematic diagram of the traditional neural network without quantum compensation and the hybrid neural network based on quantum compensation on the SSIM index with the change of bpp;

[0062] Figure 10 is the relationship between the training loss Loss and the training round Epoch under different compression ratios, wherein (a) is the training of the traditional neural network without quantum compensation on the BSDS200 dataset, (b) is the training of the hybrid neural network based on quantum compensation on the BSDS200 dataset. DETAILED DESCRIPTION

[0063] The present application will be further described by the embodiments with reference to the accompanying drawings, but the scope of the present application is not limited in any way.

[0064] A flow framework diagram of a digital holographic compression transmission method using a quantum compensation hybrid neural network provided by the present application is shown as Figure 1 , which sequentially includes a hologram generation module, a compression module, an encoding module, a transmission channel, a decoding module, a decompression module, and a hologram reproduction module. The present application is suitable for processing Fresnel off-axis holograms, including those generated by computer simulation or recorded by optical systems, and is not limited to holograms. In the embodiments of the present application, the method provided by the present application specifically includes the following steps:

[0065] 1) A set of images comprising original 3D objects is recorded and generated as a computer generated hologram dataset by Fresnel off-axis holographic computation method;

[0066] According to the principle of Fresnel off-axis holographic algorithm, the original image (object) is recorded as a computer generated hologram (digital hologram). The recording of the Fresnel off-axis hologram consists of two parts: Fresnel diffraction and off-axis reference light interference.

[0067] Wherein, the Fresnel diffraction refers to the diffraction of light waves in the near field region. The Fresnel diffraction integral formula can be used to approximate the propagation of light waves in the near field region. The Fresnel diffraction of the object light wave emitted by each point on the surface of the object to the holographic recording plane is calculated and superimposed, that is, the object wave front on the holographic recording plane can be calculated.

[0068] Off-axis reference light interference refers to the simultaneous existence of reproduction image and twin image and zero-order diffraction bright spot during hologram reconstruction, which mutually superimpose and affect the quality of the reconstructed image. When recording the hologram, a reference light with a certain angle with the object wave front is introduced, which can separate the different order diffraction images, so as to obtain a clear reconstructed image. The recording principle of off-axis reference light interference hologram is shown in Figure 2 (a), and the hologram reconstruction is shown in 2(b).

[0069] The diffraction process of the object light wave is calculated by the Fresnel diffraction formula (formula 1):

[0070]

[0071] In formula 1, k is the wave number, j is the imaginary unit, (x0, y0) represents the object wave plane, (x, y) is the corresponding diffraction plane; U0(x0, y0) is the information of the original object, U(x, y) is the complex amplitude function of the object wave front, and z represents the diffraction distance. Using a computer to calculate the Fresnel diffraction integral, the fast Fourier transform can be carried out through formula 2:

[0072]

[0073] In formula 2, F represents the Fourier transform, z is the diffraction distance, k is the wave number, and j is the imaginary unit; L0 is the initial object wave plane sampling range; the sampling point number of the initial object wave plane is N x N, that is, the pixel number of the initial image; the sampling interval of the initial object wave plane is Δx0= Δy0= L0 / N; Δx= Δy is the sampling interval after the initial object wave plane discrete Fourier transform, which corresponds to the sampling interval of the diffraction plane in the spatial domain.

[0074] According to the off-axis holographic formula, a hologram I(x, y) is formed by using the diffraction wave front complex amplitude function of the interference recording light, as shown in equation 3:

[0075]

[0076] In equation 3, I(x, y) is the recorded light intensity of the hologram, R(x, y) is the wave front function of the reference wave, and U(x, y) is the wave front function of the original image. * (x, y) and U * (x, y) correspond to the conjugate of R(x, y) and U(x, y), respectively.

[0077] 2) Construct a hybrid neural network based on quantum compensation, the schematic diagram of the whole network is shown in Figure 3 , the network is composed of compression block and decompression block, model training stage joint training, using the computer generated hologram data set obtained in step 1) into the hybrid neural network based on quantum compensation for training, get the trained hybrid neural network based on quantum compensation, split the network into compression block and decompression block, where the compression block is placed in the sending end, and the decompression block is placed in the receiving end;

[0078] 2.1 The hybrid neural network based on quantum compensation is composed of compression block and decompression block

[0079] The hybrid neural network based on quantum compensation is composed of compression block and decompression block, as shown in Figure 3 , the compression block is composed of three layers of classical convolution layers, among which the first layer and the third layer make the input and output image matrix dimensions consistent, only for feature extraction function; the second layer of convolution will adjust the convolution stride to aggregate and compress the input image matrix, so that the ratio of the input matrix size to the output matrix size of this layer (i.e. compression ratio) is the square of the convolution stride, and this layer is also called down-sampling layer. The input of the whole compression block is denoted as X gene+ated , and the output is denoted as B compress .

[0080] The decompression block is composed of two links (main link and compensation link), among which the quantum compensation module down-samples the matrix B compress to obtain the matrix B quantum , and after calculation through the quantum compensation module, the compensation matrix B compensation is obtained through the up-sampling module; the decompression main module of the main link obtains the matrix B upsampling through up-sampling calculation, introduces the quantum compensation coefficient α, and obtains the weighted result of quantum compensation (1-α)×B upsampling +α×B compensation , and the compensation result is obtained through several layers of convolution and once residual to obtain the final recovered output result X recovared .

[0081] 2.2 Quantum compensation module based on quantum neural network

[0082] Quantum gates are the basis of physically implementing quantum computing, and any quantum gate group network can be composed of quantum gates, which is also called quantum neural network. The application designs three different structures of quantum gate-based network layers, i.e. embedding layer, entanglement layer and measurement layer. In order to facilitate application, the quantum state and general quantum logic gate group are represented in complex form.

[0083] Different from classical bits, a quantum bit state |ψ> is in a coherent superposition state of |0> and |1>, which is represented by formula 4:

[0084] |ψ>=α|0>+β|1> (Formula 4)

[0085] Wherein, α and β are complex numbers representing probability amplitudes, satisfying the normalization requirement |α| 2 +|β| 2 =1. Mapping the quantum bit to the point coordinates on the Bloch sphere As Figure 4 shown, the quantum bit can be represented by formula 5:

[0086]

[0087] Wherein, θ represents the angle between the vector represented by the coordinate point and the positive direction of the z axis, and φ represents the angle between the projection of the vector represented by the coordinate point on the xy plane and the positive direction of the x axis.

[0088] The quantum compensation module in the decompression block of the quantum compensation-based hybrid neural network is as shown in Figure 5 The module is composed of embedding layer, entanglement layer and measurement layer. The quantum compensation module is input with 4 quantum bits in |0> state as initial state, and is set as |ψ> 1，...，d , wherein d=4; |ψ> 1，...，d can be represented as a superposition state of 2 d d-dimensional Hilbert space, and the d-dimensional Hilbert space can be represented by formula 6:

[0089]

[0090] Wherein, H 2 represents a two-dimensional Hilbert space with basis {|0>, |1>}. Therefore, |ψ> 1，...，d can be expressed by formula 7:

[0091]

[0092] where a i , i e {1,..., 2 d} represents the complex amplitude assigned to the computational basis state .

[0093] The computation of the quantum neural network consisting of the entire quantum compensation module includes the following three computation steps:

[0094] (1) The input quantum bit state |ψ> 1，...，d is sent to the embedding layer of the quantum compensation module, and the main role of the embedding layer is to embed the classical data into the superposition state of the quantum bit; specifically, the input quantum bit state |ψ> 1，...，d is first calculated by the H gate (Hadamard Gate), and then the classical data is embedded in the form of phase into the RY gate to output a new quantum state Specifically, as shown in Figure 6 , the compressed holographic information matrix is divided into several 2x2 small blocks and embedded into the phase parameter of the RY gate; the H gate and the RY gate can be represented in matrix form as formula 8:

[0095]

[0096] In formula 8, θ is the parameter of the RY gate.

[0097] (2) The quantum state output from the embedding layer is sent to the entanglement layer of the quantum compensation module for calculation, and the final result is obtained by quantum entanglement on different quantum circuits through a series of unitary operators U; a d-bit U gate can be represented as U d (Θ), where Θ is the set of all free parameters of the U gate; these gate levels are cascaded to form a quantum neural network, which is similar to a classical deep neural network. The sequential cascade of L unitary gates can be represented as formula 9:

[0098]

[0099] In formula 9, represents the unitary gate of the i-th layer, and Θ = {Θ1,..., Θ L} is the set of all free parameters of .

[0100] The input of the entanglement layer is calculated by the quantum network of the entanglement layer to obtain the output of the entanglement layer as

[0101] Specifically, as shown in Figure 5 ​As shown, in one entanglement layer, a controlled RZ gate (CRZ Gate) is used to entangle the superposition states of different quantum bits, and then the phase of each quantum bit is adjusted through the RY gate; where the RY gate matrix is ​​expressed as shown in Equation 8, and the CRZ gate can be expressed in matrix form as Equation 10:

[0102]

[0103] Where q0 and q1 represent two interacting quantum bits, λ is the parameter of the CRZ gate, and i is the imaginary unit.

[0104] (3) The output of the entanglement layer The result is sent to the measurement layer of the quantum compensation module for measurement, and then upsampled to obtain the quantum compensation matrix B. compensation Among them, the measurement result of the Pauli operator for the i-th quantum bit is formula 11:

[0105]

[0106] Among them, y i represents the measurement result of the i-th quantum, Z is the Pauli matrix, and its matrix expression is as follows:

[0107]

[0108] 2.3 Training process of hybrid neural network based on quantum compensation

[0109] The training steps of the entire hybrid neural network based on quantum compensation include the following steps:

[0110] (1) dividing the computer-generated hologram data set generated in step 1) into several batches and sequentially inputting them into a hybrid neural network based on quantum compensation to obtain the forward propagation calculation results of the network;

[0111] (2) Record the error between the actual output of the entire network and the target output. Here, the mean square error function is used as the loss function L(target, output) expressed as Equation 13:

[0112]

[0113] Where B is the training batch size, N is the number of pixels of each hologram input to the neural network, and target n，b and output n，b Represents the target value and predicted value of the n-th pixel position in the b-th training sample in the same batch.

[0114] According to the gradient descent method, the parameters of each layer are adjusted through the back propagation of the neural network, which is specifically expressed as Equation 14:

[0115]

[0116] where w old is the parameter before update, w new is the parameter after update, L is the loss function, and η is the learning rate.

[0117] (3) Repeat steps (1) and (2) until the network training round reaches the set number of times.

[0118] 3) The holographic floating point matrix B compress obtained after the compression block of the quantum compensation-based hybrid neural network in step 2) is compressed, and each value of the matrix is quantized to an integer value between 0 and 255 by 8 bits, and then the quantized matrix is encoded into a binary bit stream B c for transmission;

[0119] The schematic diagram of the Huffman encoding algorithm is shown in Figure 7 (a), and the Huffman encoding is an entropy encoding (weight encoding) algorithm for lossless data compression. The Huffman encoding uses a variable-length encoding table to encode the source symbols, and the letters with high occurrence probability are encoded with shorter codes, and vice versa, which reduces the expected value of the length of the encoded string, thereby achieving the purpose of lossless encoding and compressed data.

[0120] Specifically, the source symbols are arranged in order of occurrence probability from large to small, the two smallest probability source symbols are combined and added, and this step is repeated, and the larger probability branch is always placed on the left side until only one source symbol is left and the probability reaches 1. For each pair of combined right one is designated as 1 and the left one is designated as 0, the path from each source symbol to the probability 1 is obtained, and the 1 and 0 along the path are recorded. For each source symbol, write the sequence of 1 and 0, and then traverse the sequence from right to left to obtain the non-equal-length Huffman code.

[0121] The 8-bit quantization operation of the holographic floating point matrix is shown in Figure 7 (b), which uses the rounding method to quantize the floating point to an integer.

[0122] 4) After receiving the bit stream B c sent by the sending end in step 3) at the receiving end of the transmission system, the integer matrix is obtained by decoding through the Huffman decoding algorithm. In order to counteract the loss caused by the 8-bit quantization in step 3), a to Quantized random noise is generated in a uniform distribution and added to the integer matrix to obtain a floating-point matrix, and the floating-point matrix is decompressed by the decompression block of the quantum compensation-based hybrid neural network trained in step 2) to obtain a recovered hologram Figure X recovered ;

[0123] 5) The hologram decompressed in step 4) is reproduced to obtain a reproduced image of the original object.

[0124] The recovered hologram is reproduced, and according to the Fresnel off-axis holographic algorithm, the computer-generated hologram is reproduced into the original image, which needs to be realized by the off-axis reference light irradiation and the Fresnel diffraction two processes.

[0125] According to the off-axis holographic reconstruction formula, i.e. formula 15, the hologram I(x, y) is irradiated by the reconstruction light R(x, y) consistent with the reference light for reconstruction:

[0126]

[0127] In formula 15, I(x, y) is the light intensity recorded by the hologram, R(x, y) is the wavefront function of the reference light and the reconstruction light, and U(x, y) is the reconstruction wavefront function of the original image. * (x, y) is the conjugate of the reconstructed image;

[0128] The diffraction process of the computer-generated hologram light wave irradiated by the reconstruction light is calculated by the Fresnel diffraction formula (formula 1~formula 2), and the three-dimensional reconstructed image of the original object is obtained on the diffraction plane.

[0129] In order to test the compression transmission speed (i.e. network training speed) and the quality of the reproduced image of the present application, the training rounds completed for convergence are used to measure the compression transmission speed, and the compression ratio (CR), the bit number per pixel (bpp), the mean square error (MSE), the peak signal-to-noise ratio (PSNR) and the structural similarity (SSIM) are used to measure the quality of the reproduced image, wherein the CR, bpp, MSE, PSNR and SSIM are defined as follows:

[0130]

[0131]

[0132]

[0133]

[0134]

[0135] X cis the size of the compressed hologram, X o is the size of the hologram before compression; B encode is the size of the data bit stream after compression and encoding, W and H are the width and height of the hologram respectively; x and y represent the original image before compression and the reconstructed image after recovery and reproduction respectively; B is the gray level of the gray scale hologram; μ x and μ y is the mean of x and y, σ x and σ y is the variance of x and y, σ xy is the covariance of x and y, c1 and c2 are constants.

[0136] In this embodiment, the corresponding holographic data set is constructed based on the BSDS200 (200 images) and Set14 (14 images) data sets, and the BSDS200 is used for training and the Set14 is used for model evaluation. Among them, Figure 8 is the comparison chart of the hologram and the reconstructed image recovered by the traditional neural network without quantum compensation and the hybrid neural network based on quantum compensation of the baboon image in the Set14 data set under seven different compression ratios, wherein the compression ratio CR takes values including {4, 9, 16, 25, 36, 49, 64}, wherein the left first column is the generated hologram and the reconstructed image of the original image without compression, the front two rows on the right are the hologram recovered by the traditional neural network without quantum compensation and the reconstructed image under different compression ratios, and the last two rows are the hologram recovered by the hybrid neural network based on quantum compensation and the reconstructed image under different compression ratios. It can be seen that under most compression ratios, the image quality recovered by the hybrid neural network based on quantum compensation is better than that recovered by the traditional neural network without quantum compensation, and the advantage is more obvious as the compression ratio increases. Therefore, the recovery effect of the computer generated hologram compression transmission method of the present application is better.

[0137] Table 1 compares the experimental results of the hybrid neural network based on quantum compensation (Quantum) and the traditional neural network without quantum compensation (No Quantum) under different compression rates on the Set14 data set

[0138]

[0139] Table 1 is a comparison of experimental results of two schemes under different compression ratios. Figure 9 The two schemes are shown on the Set14 data set, and the graph of the two key indicators PSNR and SSIM changes with bpp. It shows that the recovery effect of the hybrid neural network based on quantum compensation is better.

[0140] Figure 10The training loss Loss and training round Epoch of the traditional neural network without quantum compensation and the hybrid neural network based on quantum compensation on the BSDS200 dataset are compared. It can be seen that the convergence speed of the training loss of the hybrid neural network based on quantum compensation is faster with the increase of the compression ratio, and therefore, the speed of the computer-generated hologram compression transmission method is faster.

[0141] In summary, the scheme has good application prospects in three-dimensional holographic information compression and transmission. The scheme idea can achieve a relatively optimal level in the compression transmission speed and image recovery quality of the computer-generated hologram compression transmission technology.

[0142] It should be noted that the purpose of publishing the embodiments is to help further understand the present application, but those skilled in the art can understand that various replacements and modifications are possible without departing from the spirit and scope of the present application and the appended claims. Therefore, the present application should not be limited to the disclosed content of the embodiments, and the scope of the present application claimed is defined by the scope of the claims.

[0143] Reference:

[0144] [1]Ko H,Kim H Y.Deep learning-based compression for phase-only hologram[J].IEEE Access,2021,9:79735-79751.

[0145] [2]Shimobaba T,Blinder D,Schelkens P,et al.Deep-Learning-Based Dynamic Range Compression for 3D Scene Hologram[C] / / ICOL-2019:Proceedings of the International Conference on Optics and Electro-Optics,Dehradun,India.Springer Singapore,2021:41-44.

Claims

1. A digital holographic compression transmission method, the specific steps are as follows: 1) The image set containing the original 3D object is recorded and generated into a computer-generated hologram data set by a Fresnel off-axis holographic calculation method; 2) Construct a hybrid neural network based on quantum compensation, the network is composed of compression blocks and decompression blocks, wherein the compression block is composed of three layers of classical convolution layers, wherein the first layer of convolution and the third layer of convolution make the image matrix dimensions of the input and output consistent, only the function of feature extraction; the second layer of convolution will aggregate and compress the input image matrix by adjusting the convolution stride, so that the ratio of the input matrix size to the output matrix size of this layer of convolution is the square of the convolution stride, the input of the whole compression block is denoted as generated , and the output is denoted as B compress ; the floating point matrix B compress output by the compression block is output to the decompression block for calculation, the decompression block is composed of a main link and a compensation link, wherein the compensation link performs down-sampling on the matrix B compress to obtain the matrix B quantum , after calculation by the quantum compensation module, the matrix B compensation is obtained through the up-sampling module; the main link obtains the matrix B upsampling through up-sampling calculation, introduces the quantum compensation coefficient α, obtains the weighted result of quantum compensation (1-α)×B upsampling +α×B compensation , and obtains the final recovered output result X recovered after the compensation result is passed through a plurality of layers of convolution and residual; The computer-generated hologram data set obtained in step 1) is used in the model training stage to send into the hybrid neural network based on quantum compensation for joint training, and a trained hybrid neural network based on quantum compensation is obtained, which is divided into a compression block and a decompression block, wherein the compression block is placed at the sending end and the decompression block is placed at the receiving end; 3) The sending end obtains the holographic floating point matrix after compression by the compression block of the hybrid neural network based on quantum compensation in step 2), quantizes each value of the matrix to an integer value between 0 and 255 by 8 bits, and then encodes the quantized matrix into a binary bit stream by using the Huffman lossless encoding algorithm for transmission; 4) After receiving the bit stream sent by the sending end in step 3), the receiving end decodes the bit stream by using the Huffman decoding algorithm to obtain an integer matrix, adds quantization noise uniformly distributed from -1 / 2 to 1 / 2 to the integer matrix to obtain a floating point matrix, and decompresses the floating point matrix by using the decompression block of the hybrid neural network based on quantum compensation trained in step 2) to obtain a restored hologram; 5) The hologram obtained by decompression in step 4) is reproduced to obtain a reproduction image of the original object.

2. The method of claim 1, wherein the step of compressing the digital hologram is performed by using a compression algorithm. Step 1) specifically includes a Fresnel diffraction calculation and an off-axis reference light interference calculation method; The calculation formula of the Fresnel diffraction is formula 1: In formula 1, k is the wave number, j is the imaginary unit, (x0, y0) represents the object wave plane, (x, y) is the corresponding diffraction plane, U0(x0, y0) is the information of the original object, U(x, y) is the complex amplitude function of the object light wave front, and z represents the diffraction distance. The Fresnel diffraction integral is calculated by using a computer, and fast Fourier transform is performed by formula 2: In formula 2, F represents Fourier transform, z is the diffraction distance, k is the wave number, j is the imaginary unit, and L0 is the initial object wave plane sampling range. The number of sampling points of the initial object wave plane is N×N, that is, the number of pixels of the initial image. The sampling interval of the initial object wave plane sampling point is Δx0=Δy0=L0 / N. Δx=Δy is the sampling interval after the initial object wave plane discrete Fourier transform, which corresponds to the sampling interval of the diffraction plane in the spatial domain. The off-axis reference light interference calculation method records the wave front complex amplitude function to form a hologram I(x, y) by using the interference principle of light, as shown in formula 3: In Equation 3, I(x,y) is the recorded light intensity of the hologram, R(x,y) is the wavefront function of the reference wave, U(x,y) is the wavefront function of the original image, R * (x,y) and U * (x,y) correspond to the conjugate of R(x,y) and U(x,y), respectively.

3. The method of claim 1, wherein the step of compressing the digital hologram is performed by using a compression algorithm. The quantum compensation module is composed of an embedding layer, an entanglement layer and a measurement layer, the quantum compensation module has 4 quantum bits in |0> state as initial state input and is set as |ψ> 1,…,d , wherein d=4; |ψ> 1,…,d represents a superposition state of 2 d computational basis states in a d-dimensional Hilbert space H 2d , which is represented as formula 4: In Equation 4, H 2 represents a two-dimensional Hilbert space with basis {|0>, |1}, |ψ> 1,…,d is expressed as Equation 5: In formula 5, a i , i e {1,..., 2 d} denotes the complex amplitudes assigned to the computational basis states , The calculation of the entire quantum compensation module includes the following three calculation steps: 221) The input quantum bit state |ψ 1,…,d is sent into the embedding layer of the quantum compensation module, specifically, the input quantum bit state |ψ 1,…,d is first calculated by the H gate, and then the classical data is embedded into the RY gate in the form of phase to output a new quantum state 222) the quantum state output in step 221) is fed into the entanglement layer of the quantum compensation module for computation, the final result is obtained by quantum entanglement on different quantum circuits through a series of unitary operators U, a d-bit U gate is represented as U (Θ), where Θ is the set of all free parameters of the U gate; the sequential concatenation of L unitary gates is represented as formula 6: d (Θ), where Θ is the set of all free parameters of the U gate; the sequential concatenation of L unitary gates is represented as formula 6: in formula 6, denotes the unitary gate of the i-th layer, Θ = {Θ1,..., Θ L is the set of all free parameters; inputs to the entanglement layer quantum neural network representation through the entanglement layer the entanglement layer output is then obtained as 223) the output in step 222) is sent into the measurement layer of the quantum compensation module to measure and obtain the final output result of the entire quantum compensation block, and up-sampling is performed to obtain the matrix B in step 222) 223) the output in step 222) is sent into the measurement layer of the quantum compensation module to measure and obtain the final output result of the entire quantum compensation block, and up-sampling is performed to obtain the matrix B in step 222) compensation wherein the measurement result of the i-th quantum bit through the Pauli operator is formula 7: In formula 7, y i represents the measurement result of the i-th quantum, and Z is a Pauli matrix.

4. The method of claim 3, wherein the step of compressing the digital hologram is performed by using a wavelet transform. The embedding layer of step 221) is specifically that the compressed holographic information matrix is divided into several 2×2 small blocks and embedded into the phase parameter of the RY gate. The H gate and the RY gate are represented in matrix form as formula 8: In formula 8, θ is the parameter of the RY gate.

5. The method of claim 3, wherein the step of compressing the digital hologram is performed by using a wavelet transform. In the entanglement layer of step 222), the controlled CRZ gate is used to entangle the superposition state of different quanta, and then the phase of each quantum bit is adjusted by the RY gate. The RY gate matrix is shown in formula 8, and the CRZ gate is represented in matrix form as formula 9: Wherein q0, q1 represent two qubits of two interactions, λ is the parameter of CRZ gate, and i is the imaginary unit.

6. The method of claim 3, wherein the step of compressing the digital hologram is performed by using a wavelet transform. The measurement layer in step 223) uses the Pauli Z matrix for measurement, and the matrix expression is formula 10:

7. The method of claim 1, wherein the step of compressing the digital hologram is performed by using a compression algorithm. The training process of the hybrid neural network based on quantum compensation in step 2) includes: 231) The computer-generated hologram data set generated in step 1) is input into the hybrid neural network based on quantum compensation in batches to obtain the calculation results of the forward propagation of the network; 232) Record the error between the actual output and the target output of the entire network, and adjust the parameters of each layer through the back propagation of the neural network according to the gradient descent method; 233) Repeat steps 231) ~ 232) until the network training round reaches the set number of times.

8. The digital holographic compression transmission method according to claim 1, wherein: The reproduction in step 5) is specifically the reproduction of the Fresnel off-axis hologram through the processes of off-axis reference light irradiation and Fresnel diffraction, and specifically includes: According to the off-axis holographic reconstruction formula, that is, formula 8, the hologram I(x, y) is irradiated with the reconstruction light R(x, y) consistent with the reference light for reconstruction: In formula 11, I(x, y) is the light intensity of hologram recording, R(x, y) is the wave front function of the reference light and the reproduced light, U(x, y) is the reproduced wave front function of the original image; U * (x, y) is the conjugate term of the reproduced image; The diffraction process of the computer-generated hologram light wave irradiated by the reconstruction light is calculated by the Fresnel diffraction formula, and the three-dimensional reconstructed image of the original object is obtained on the diffraction plane.

Images