Quantum auto-encoder model, video prediction method and related device

Abstract

The invention discloses a quantum auto-encoder model, a video prediction method and a related device, and belongs to the technical field of quantum computing, the quantum auto-encoder model comprises an encoder, an intermediate module and a decoder which are sequentially connected in series, and the encoder, the intermediate module and the decoder all comprise different variable component sub-circuits; the encoder is used for extracting features of an input target image frame, and the target image frame is an image frame corresponding to a moment before a target moment; the intermediate module is used for generating an initial prediction image frame by using the features extracted by the encoder and a relationship between adjacent image frames constructed during prediction at a moment before a target moment; and the decoder is used for reconstructing the initial prediction image frame to obtain a target prediction image frame at a target moment. By applying the embodiment of the invention, the accuracy of video prediction is improved.

Description

Technical Field

[0001] This application belongs to the field of quantum computing technology, specifically a quantum autoencoder model, a video prediction method, and related devices. Background Technology

[0002] Video prediction technology requires analyzing the temporal dependencies within video sequences, understanding the relationships between consecutive frames, and predicting future frames. With the development of automation technologies, video prediction can be integrated into intelligent systems, such as autonomous vehicles and robot navigation, to achieve more intelligent decision-making and responses. Secondly, in the field of security monitoring, video prediction technology can provide early warnings of potential risks and threats, such as identifying suspicious behavior or predicting traffic accidents, thereby improving security. Furthermore, in the entertainment and gaming industry, video prediction can offer more immersive and interactive experiences, such as adjusting the game environment in real time by predicting user actions.

[0003] Video data contains a wealth of information, and video prediction technology can help analyze and understand this data to support decision-making. However, current video prediction models are limited by the computational power of classical computing, making it difficult to accurately capture dynamic changes and details. This leads to discrepancies between predicted results and actual events, resulting in reduced prediction accuracy. Summary of the Invention

[0004] The purpose of this application is to provide a quantum autoencoder model, a video prediction method, and related devices, with the aim of improving prediction accuracy.

[0005] One embodiment of this application provides a quantum autoencoder model, which includes an encoder, an intermediate module, and a decoder connected in series. Each of the encoder, the intermediate module, and the decoder includes different variable quantum network layers.

[0006] The encoder is used to extract features of the input target image frame, wherein the target image frame is the image frame corresponding to the time before the target time.

[0007] The intermediate module is used to generate an initial prediction image frame by utilizing the features extracted by the encoder and the relationship between adjacent image frames constructed during prediction at the time preceding the target time.

[0008] The decoder is used to reconstruct the initial predicted image frame to obtain the target predicted image frame at the target time.

[0009] Optionally, when both the target time and the time before the target time are future times, the target data is the target prediction image frame obtained by predicting the time before the target time;

[0010] When the time preceding the target time is not a future time, the target data is the image frame actually generated at the time preceding the target time.

[0011] Optionally, the encoder includes a first convolutional neural network (CNN) layer, a variable quantum network layer, and a second CNN layer connected in series.

[0012] The variable quantum network includes a first quantum circuit and a second quantum circuit, and the circuit structures of the first quantum circuit and the second quantum circuit are different.

[0013] The second CNN layer extracts features from the fused data, wherein the fused data is obtained by fusing the data output by the first quantum circuit and the second quantum circuit.

[0014] Optionally, the first quantum circuit includes a coding layer consisting of an H gate and a first RZ gate acting on each qubit, and a variable layer consisting of controlled modules;

[0015] Each controlled module includes a CNOT gate, a second RZ gate, a first RX gate, and a CNOT gate in sequence. The number of controlled modules is n*(n-1) / 2, where n is the number of qubits in the first quantum circuit.

[0016] Optionally, the second quantum circuit includes, in sequence, an H gate, an RY gate, a second RX gate, a Y gate, a Z gate, and a controlled RZZ gate and a CNOT gate acting on each qubit.

[0017] Optionally, the decoder includes a first deconvolutional neural network layer, a first quantum circuit, and a second deconvolutional neural network layer, wherein,

[0018] One output of the first deconvolutional neural network layer is connected to the input of the first quantum circuit;

[0019] The other output of the first deconvolutional neural network layer is connected to the output of the first quantum circuit and the input of the second deconvolutional neural network layer through a residual structure.

[0020] Optionally, the intermediate module includes M third quantum circuits and N fourth quantum circuits;

[0021] The M third quantum circuits are used to extract the time features and first feature data of the first input data. The first input data includes the features extracted by the encoder and the target time feature data. The target time feature data is the time feature data predicted based on the target data of the previous moment. The first feature data is the feature data extracted by the M quantum circuits and used as input to the fourth quantum circuit.

[0022] The N fourth quantum circuits are used to extract the spatial features and second feature data of the second input data. The second input data includes first feature data and target spatial feature data. The target spatial feature data is spatial feature data predicted based on the target data of the previous time step. The second feature data is feature data used to generate the initial predicted image frame.

[0023] The relationship between adjacent image frames includes the target temporal feature data and the target spatial feature data.

[0024] Optionally, the third quantum circuit and the fourth quantum circuit have the same structure, both including a coding layer composed of an H gate and a third RZ gate acting on each quantum bit and a variable layer composed of a CNOT gate, a U3 gate and a third RX gate.

[0025] One embodiment of this application provides a video prediction method, the method comprising:

[0026] Obtain the target image frame, wherein the target image frame is the image frame corresponding to the time before the target time.

[0027] The target image frame is input into the quantum autoencoder model in any of the above embodiments after training, to obtain the target predicted image frame.

[0028] One embodiment of this application provides a video prediction apparatus, the apparatus comprising:

[0029] The first acquisition module is used to acquire a target image frame, wherein the target image frame is the image frame corresponding to the time before the target time.

[0030] The second acquisition module is used to input the target image frame into the quantum autoencoder model in any of the above embodiments that has been trained, and to obtain the target predicted image frame.

[0031] Another embodiment of this application provides a computer device including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the quantum autoencoder or video prediction method described in any of the above embodiments.

[0032] Another embodiment of this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a computer, causes the computer to perform the quantum autoencoder or video prediction method in any of the above embodiments.

[0033] Compared with existing technologies, this application provides a quantum autoencoder model, a video prediction method, and related devices. The embodiments of this application utilize variable quantum circuitry and the parallelism of quantum computing to perform rapid multi-scale analysis of image frames, extracting features at different resolutions and levels of detail. Simultaneously, leveraging the high-dimensionality of quantum computing, it can better capture the complex features of image frames. Based on the relationship between the features extracted by the encoder and the previously predicted adjacent image frames, the intermediate module uses the learned trend of image frame changes to generate an initial predicted image frame. Then, the decoder reconstructs the data to obtain the target predicted image frame, thereby improving prediction accuracy. Attached Figure Description

[0034] Figure 1 A network block diagram for implementing a quantum autoencoder model or video prediction system is provided in the embodiments of this application;

[0035] Figure 2 A schematic diagram of the structure of a quantum autoencoder model provided in an embodiment of this application;

[0036] Figure 3 A schematic diagram of the overall architecture of a quantum autoencoder model provided in this application embodiment;

[0037] Figure 4 This is a schematic diagram of the structure of an encoder provided in an embodiment of this application;

[0038] Figure 5 This is a schematic diagram of the structure of a first quantum circuit provided in an embodiment of this application;

[0039] Figure 6 A schematic diagram of the structure of a second quantum circuit provided in this application embodiment.

[0040] Figure 7 A schematic diagram of the structure of a decoder provided in an embodiment of this application.

[0041] Figure 8 This is a schematic diagram of the structure of a third quantum circuit provided in an embodiment of this application;

[0042] Figure 9 A schematic diagram of the backbone network structure of an intermediate module provided in an embodiment of this application;

[0043] Figure 10 A flowchart illustrating a video prediction method provided in an embodiment of this application;

[0044] Figure 11 This is a schematic diagram of the structure of a video prediction device provided in an embodiment of this application;

[0045] Figure 12This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0046] The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0047] Figure 1 This is a network block diagram of a quantum autoencoder model or video prediction system provided in an embodiment of this application. The implementation of the quantum autoencoder system or video prediction system may include a network 110, a server 120, a wireless device 130, a client 140, storage 150, a classical computing unit 160, a quantum computing unit 170, and may also include additional memory, a classical processor, a quantum processor, and other devices not shown.

[0048] Network 110 is a medium used to provide communication links between various devices and computers connected together within a quantum autoencoder model or video prediction system, including but not limited to the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof, and the connection method can be wired, wireless communication links, or fiber optic cables.

[0049] Server 120, wireless device 130, and client 140 are conventional data processing systems that may contain data and application programs or software tools that perform conventional computational processes. Client 140 may be a personal computer or a network computer, so the data may also be provided by server 120. Wireless device 130 may be a smartphone, tablet, laptop, smart wearable device, etc. Storage unit 150 may include database 151, which can be configured to store data such as qubit parameters, quantum logic gate parameters, quantum circuits, and quantum programs.

[0050] The classical computing unit 160 (quantum computing unit 170) may include a classical processor 161 (quantum processor 171) for processing classical data (quantum data) and a memory 162 (memory 172) for storing classical data (quantum data). The classical data (quantum data) may be a boot file, an operating system image, and an application program 163 (application program 173). The application program 163 (application program 173) may be used to implement a quantum algorithm compiled according to the quantum autoencoder model or video prediction method provided in the embodiments of this application.

[0051] Any data or information stored or generated in the classical computing unit 160 (quantum computing unit 170) can also be configured to be stored or generated in another classical (quantum) processing system in a similar manner, and any application executed therein can also be configured to be executed in another classical (quantum) processing system in a similar manner.

[0052] It should be noted that a true quantum computer has a hybrid structure, which includes at least... Figure 1 The system consists of two main parts: the classical computing unit 160, which is responsible for performing classical calculations and control; and the quantum computing unit 170, which is responsible for running quantum programs to achieve quantum computing.

[0053] The aforementioned classical computing unit 160 and quantum computing unit 170 can be integrated into a single device or distributed across two different devices. For example, a first device including the classical computing unit 160 runs a classical computer operating system, providing quantum application development tools and services, as well as the storage and network services required for quantum applications. Users develop quantum programs using the quantum application development tools and services on the second device, and send these quantum programs to a second device including the quantum computing unit 170 via the network services. The second device runs a quantum computer operating system, which parses and compiles the quantum program's code into instructions that the quantum processor 170 can recognize and execute. The quantum processor 170 then implements the quantum algorithm corresponding to the quantum program based on these instructions.

[0054] The computing units of the classic processor 161 within the classic computing unit 160 are based on CMOS transistors on a silicon chip. These computing units are not limited by time or coherence; that is, they are available at any time without time constraints. Furthermore, the number of such computing units in a silicon chip is sufficient; currently, a single classic processor 161 contains tens of thousands of computing units. Given this sufficient number and the fixed selectable computing logic of the CMOS transistors (e.g., AND logic), computational performance is achieved by combining a large number of CMOS transistors with a limited set of logic functions during operation.

[0055] In the quantum computing unit 170, the basic computing unit of the quantum processor 171 is the qubit. The input of a qubit is limited by coherence and coherence time; that is, a qubit is limited by its available usage time and is not always readily available. Making full use of qubits within their available usage time is a key challenge in quantum computing. Furthermore, the number of qubits in a quantum computer is one of the representative indicators of its performance. Each qubit performs computational functions through on-demand configured logical functions. Given the limited number of qubits and the diverse logical functions available in quantum computing, such as Hadamard gates (H gates), Pauli-X gates (X gates), Pauli-Y gates (Y gates), Pauli-Z gates (Z gates), X gates, RY gates, RZ gates, CNOT gates, CR gates, iSWAP gates, Tofoli gates, etc., quantum computing requires combining a limited number of qubits with diverse logical function combinations to achieve computational effects.

[0056] Based on these differences, the design of classical logic functions applied to CMOS transistors and the design of quantum logic functions applied to qubits are significantly and fundamentally different. The design of classical logic functions applied to CMOS transistors does not need to consider the individuality of CMOS transistors. For example, the representation of a CMOS transistor in a silicon chip is its individual identifier, location, and usable time of each CMOS transistor. Therefore, classical algorithms composed of classical logic functions only express the operational relationship of the algorithm, not the algorithm's dependence on individual CMOS transistors.

[0057] Quantum logic functions applied to qubits need to consider the individuality of each qubit, such as its position within the quantum chip, its relationship with surrounding qubits, and the duration of its usable time. Therefore, quantum algorithms composed of quantum logic functions not only express the computational relationships within the algorithm but also its dependence on the individual qubits.

[0058] A quantum chip can include qubits and channels for controlling them. Quantum logic gates are implemented using analog signals. Different combinations of analog signals are applied to the qubits through these channels, thereby creating quantum circuits with different functions to process data. Therefore, the design of quantum logic functions in the qubits (including the design of whether qubits are used and the design of the efficiency of each qubit) is crucial for improving the computational performance of quantum computers and requires special design. This is the unique characteristic of quantum algorithms based on quantum logic functions, and it is fundamentally and significantly different from classical algorithms based on classical logic functions. The aforementioned design considerations for qubits are technical problems that ordinary computing devices do not need to consider or address.

[0059] See Figure 2 , Figure 2 This is a schematic diagram of a quantum autoencoder model provided in an embodiment of this application. The quantum autoencoder model includes an encoder 201, an intermediate module 202, and a decoder 203 connected in series. Each of the encoder 201, the intermediate module 202, and the decoder 203 includes different variable quantum network layers.

[0060] The encoder 201 is used to extract features of the input target image frame, wherein the target image frame is the image frame corresponding to the time before the target time.

[0061] The intermediate module 202 is used to generate an initial prediction image frame by utilizing the features extracted by the encoder and the relationship between adjacent image frames constructed during prediction at the time preceding the target time.

[0062] The decoder 203 is used to reconstruct the initial predicted image frame to obtain the target predicted image frame at the target time.

[0063] Encoder 201 is responsible for mapping high-dimensional input data to a low-dimensional latent space. This process compresses the input data into an encoded or latent representation that captures the core features of the input data. Intermediate module 202 acts as a bridge connecting encoder 201 and decoder 203, playing a crucial role in data compression, feature extraction, model regularization, and latent space construction. It maps the input data to a lower-dimensional space, achieving data compression. This process forces the quantum autoencoder to learn how to represent the input data with less information, thereby extracting the most salient features and helping to remove noise and unimportant details, resulting in a cleaner feature representation. The representation of intermediate module 202 can capture the essential features of the input data, which can be very useful for data reconstruction. In this embodiment, the relationship between the current input image frame and the predicted image frame can be captured based on the relationship between adjacent image frames constructed during the previous prediction. The data for the predicted image frame is then constructed based on the captured relationship. Decoder 203's main function is to reconstruct the low-dimensional representation generated by intermediate module 202 back to the original data or its approximate representation. It should be noted that, in the embodiments of this application, the image frame corresponds to a time, with one time corresponding to one image frame. The relationship between adjacent image frames refers to the relationship between image frames at adjacent time points.

[0064] The autoencoder model incorporates variational quantum circuits (VQCs) in its encoder 201, intermediate module 202, and decoder 203. These VQCs, with fewer parameters, achieve information transformation from the input image data and extract richer data features. In the decoder 203, the VQCs are used to reconstruct the output data from the intermediate module 202, generating highly similar predicted image frames. The overall architecture of the quantum autoencoder model is as follows: Figure 3 As shown, Figure 3 The process of data change is shown. When the output of encoder 201 is further processed by intermediate module 202, conditional information is added. This conditional information is based on the relationship between adjacent image frames constructed when predicting at the time before the target time to obtain initial prediction data. The initial prediction data is then reconstructed by decoder 203 to obtain the target prediction data.

[0065] This application embodiment can achieve data dimensionality reduction through the characteristics of the encoder itself. By using variable quantum circuits and leveraging the parallelism of quantum computing, it can perform rapid multi-scale analysis on image frames, extracting features at different resolutions and levels of detail. At the same time, by utilizing the high-dimensionality of quantum computing, it can also better capture the complex features of image frames. Based on the relationship between the features extracted by the encoder and the adjacent image frames predicted in the previous test, the intermediate module uses the learned trend of image frame changes to generate an initial predicted image frame. Then, the decoder is used to reconstruct the data to obtain the target predicted image frame, thereby improving the prediction accuracy.

[0066] In this embodiment, the encoder 201 can be implemented using a combination of quantum computing and classical computing. Specifically, the encoder 201 includes a first CNN layer, a variable quantum network layer, and a second CNN layer connected in series. The variable quantum network includes a first quantum circuit and a second quantum circuit, which have different circuit structures. The second CNN layer extracts features from the fused data, where the fused data is obtained by fusing the outputs of the first and second quantum circuits. The first and second CNN layers may also include group normalization and activation functions. The inputs to the first and second quantum circuits are the same, and the same data is processed. However, due to their different structures, their ability to capture data features differs. After processing by the variable quantum network, features of different dimensions of the data can be captured, allowing for a more comprehensive understanding of the data and thus improving the accuracy and predictive ability of the autoencoder. The outputs of the first and second quantum circuits are fused and then input into the second CNN layer to complete data transformation and feature extraction. In this embodiment, data fusion can be achieved by adding two sets of data. For example, the encoder can... Figure 4 As shown, the first CNN layer is a two-dimensional convolution with 3×3 and 1×1 kernels. The variable quantum network includes VQC1 and VQC2. The outputs of VQC1 and VQC2 are obtained by group normalization and LeakyReLU activation function, respectively. The two outputs are fused and input into the second CNN layer. The second CNN layer has the same structure as the first CNN layer, but the input data is different.

[0067] In this embodiment of the application, the first quantum circuit may include a coding layer consisting of an H gate and a first RZ gate acting on each qubit and a variable layer consisting of controlled modules. Each controlled module sequentially includes a CNOT gate, a second RZ gate, a first RX gate and a CNOT gate. The number of controlled modules is n*(n-1) / 2, where n is the number of qubits in the first quantum circuit.

[0068] The rotation angle of the first RZ gate in the encoding layer is determined by the input data. The rotation angles of the second RZ gate and the first RX gate in the controlled module are obtained by training the quantum autoencoder model. When the first quantum circuit has n qubits, the number of controlled modules with the first qubit as the control bit is n-1, the number of controlled modules with the second qubit as the control bit is n-2, and so on, resulting in a total number of controlled modules of n*(n-1) / 2. Taking a first quantum circuit with 4 qubits as an example, the first quantum circuit is as follows... Figure 5 As shown in the figure, the RZ gate next to the H gate is the first RZ gate, the RZ gate after the CNOT gate is the second RZ gate, and the RX gate after the second RZ gate is the first RX gate. Each controlled module has the same structure, but the parameters of the RZ gate and RX gate in the controlled module may be different, which is determined by training.

[0069] In this embodiment of the application, the second quantum circuit may sequentially include an H gate, an RY gate, a second RX gate, a Y gate, a Z gate, and a controlled RZZ gate and a CNOT gate acting on each qubit.

[0070] The RZZ gate is a two-qubit gate used in quantum computing to perform controlled Z-gate operations on two qubits. The RZZ gate can be viewed as applying a Z-gate operation to two qubits simultaneously, but this operation only occurs when both qubits are in the |1> state. The matrix form of the RZZ gate is as follows:

[0071]

[0072] Taking a second quantum circuit with 4 qubits as an example, the second quantum circuit is as follows: Figure 6 As shown, the RX gate between the RY gate and the Y gate is the second RX gate. The rotation angles of the RY gate and the RX gate are determined by the input data, while the rotation angle of the RZZ gate is determined during training.

[0073] In this embodiment, the decoder includes a first deconvolutional neural network layer, a first quantum circuit, and a second deconvolutional neural network layer, wherein...

[0074] One output of the first deconvolutional neural network layer is connected to the input of the first quantum circuit;

[0075] The other output of the first deconvolutional neural network layer is connected to the output of the first quantum circuit and the input of the second deconvolutional neural network layer through a residual structure.

[0076] The decoder consists of a two-dimensional deconvolutional network and a first quantum circuit. Deconvolution, also known as deconvolution or transpose convolution, is the inverse process of convolution. Through deconvolution, it attempts to recover the features of the original input from the output of the convolution operation. For example, the decoder structure can be as follows: Figure 7 As shown, the residual structure can be an adder. The difference in the first quantum circuit in encoder 201 is that, during the quantum circuit measurement phase, decoder 203 requires individual measurement of each qubit to calculate the expected value of the Hamiltonian observation, while encoder 201 calculates the expected value of the Hamiltonian observation for all qubits.

[0077] In this embodiment, the intermediate module 202, in addition to generating the initial predicted image frame, also needs to extract spatiotemporal features using variable quantum circuits. Therefore, the intermediate module can also be called a quantum spatiotemporal feature extraction unit. To extract spatiotemporal features and second feature data, the intermediate module may include M third quantum circuits and N fourth quantum circuits. The M third quantum circuits are used to extract the temporal features and first feature data of the first input data. The first input data includes features extracted by the encoder and target temporal feature data. The target temporal feature data is temporal feature data predicted based on target data from the previous moment. The first feature data is feature data extracted using the M quantum circuits and used as input to the fourth quantum circuit. The N fourth quantum circuits are used to extract the spatial features and second feature data of the second input data. The second input data includes first feature data and target spatial feature data. The target spatial feature data is spatial feature data predicted based on target data from the previous moment. The second feature data is feature data used to generate the initial predicted image frame.

[0078] The M third quantum circuits are connected in series, as are the N fourth quantum circuits. The third and fourth quantum circuits are also connected in series. The first characteristic data is input into the first fourth quantum circuit. The target time characteristic data and target space characteristic data are initially set to initial values ​​(which can be 0). After passing through M third quantum circuits, new target time characteristic data and the first characteristic data are obtained. Similarly, after passing through N fourth quantum circuits, new target space characteristic data and the second characteristic data are obtained. In some embodiments of this application, the third and fourth quantum circuits have the same structure, both including a coding layer composed of an H gate and a third RZ gate acting on each qubit, and a variable layer composed of a CNOT gate, a U3 gate, and a third RX gate. Taking 4 qubits as an example, the third quantum circuit can... Figure 8As shown, a U3 gate and an RX gate are distributed between the two CNOT gates. The qubits acted upon by the U3 and RX gates are the target qubits of the preceding CNOT gate. The third RZ gate is an RZ gate immediately adjacent to the H gate, and the third RZ gate is an RX gate immediately adjacent to the U3 gate. The rotation angle of the RZ gate is determined by the input data. The parameters of the U3 gate and the rotation angle of the RX gate are determined during training and are optimization parameters during training. Although the third and fourth quantum circuits can be the same, the parameters in the circuits are likely to be different, thus enabling the processing of features of different dimensions of the data. It should be noted that M and N are positive integers. During training, the number of M and N can be adjusted according to the training results. When training is complete, the number of M and N is fixed. In this application, the first, second, third, and fourth quantum circuits are all variable quantum circuits.

[0079] In this embodiment of the application, the intermediate module 202 processes the t-th frame X output by the encoder. ′ Generate the (t+1)th frame image Y ′ , t∈(0,T+N). When t≤T, X ′ For a subsequence X of the input image frame sequence X, t When t>T, X ′ To generate a subsequence Y of sequence Y t When both the target time and the time preceding the target time are future times, the target data is the predicted image frame obtained at the time preceding the target time. When the time preceding the target time is not a future time, the target data is the image frame actually generated at the time preceding the target time. For example, if T is 10 and N is 5, then using the image frames generated from time 0 to time 9, the data from time 10 to time 14 is predicted. When the target time is time 8, then the input X... ′ It is X7, which is the actual image frame generated at time 7, not the image frame predicted from the input at time 7. When the target time is time 12, the input is X. ′ It is Y 11 This refers to the image frame predicted at time 11. In this embodiment, the image Y is predicted when t ≤ T. ′ The main purpose is to obtain the relationship between adjacent image frames. The relationship between adjacent image frames includes target temporal feature data and target spatial feature data. Target temporal feature data reflects the trend of feature data changing over time, and target spatial feature data reflects the trend of feature data changing over space. These two data can reflect the relationship between the previous frame image and the next frame image, that is, they can reflect the trend of image change based on the current frame image.

[0080] The backbone network of intermediate module 202 can be as follows Figure 9 As shown, in addition to the input X′, it also includes S.t-1 and T t-1 Two hidden states for extracting spatiotemporal features, and a hidden state h′, S to assist in generating the prediction sequence. t-1 For target spatial feature data, T t-1 For the target time feature data, X′ and T t-1 The input is fed into an M-VQC to obtain T′. An M-VQC consists of M VQCs. When the next VQC in an M-VQC runs, it needs the output T′ of the previous VQC. Processing the output T′ of the last VQC yields T. t It should be noted that T′ and the feature data input to N-VQC are not directly obtained from M-VQC measurements, but rather from processed measurement results using conventional processing methods. The first feature data extracted from M-VQC and S... t-1 The input is fed into N-VQC to obtain S′ and second feature data. The extracted second feature data undergoes a series of processes shown in the figure to obtain the initial predicted image frame, where...

[0081] g t =tanh(W xg *X′+W hg *h′+b g )

[0082] i t =σ(W xi *X′+W hi *h′+b i )

[0083] c t =N t +g t ⊙i t

[0084] g′ t =tanh(W′) xg *X′+b′ g )

[0085] i′ t =σ(W′) xi *X′+b′ i )

[0086] f′ t =σ(W′) xf *X′+b′ f )

[0087] M t =f′ t ⊙h′+i′ t ⊙g′ t

[0088] o t =σ(W xo *X′+W ho *h′+W co *c t +W mo *M t +b o )

[0089] Y′=o t ⊙tanh(W 1×1 *[c t M t ])

[0090] σ represents the activation function sigmoid, W xg W hg W xi W hi 、W′ xi 、W′ xg 、W′ xf W xo W ho W co W mo b is the corresponding weight parameter. g b i b′ g b′ i b′ f b o For the corresponding bias parameter, N t This is the second feature data.

[0091] The quantum autoencoder model provided in this application can solve the multi-scale prediction problem of large-scale video image frame sequence datasets. Leveraging the parallelism of quantum computing, it allows for the simultaneous processing of large amounts of data, significantly improving processing speed and reducing computational resource requirements. This is particularly useful for processing large-scale video image frame sequences and brings revolutionary improvements to the fields of video analysis and image processing. Through quantum algorithms, image frames are analyzed rapidly at multiple scales, achieving efficient prediction at different resolutions and levels of detail. Simultaneously, the high-dimensionality of quantum computing enables better capture of complex image features in multi-scale prediction, thereby improving prediction accuracy and reliability. Combined with classical neural network techniques and mature architectures, a hybrid model is formed, further enhancing the performance and efficiency of video image processing.

[0092] This application also provides a video prediction method in its embodiments; see [link to relevant documentation]. Figure 10 The methods include:

[0093] S101: Obtain the target image frame, wherein the target image frame is the image frame corresponding to the time before the target time.

[0094] It should be noted that if the target time is a future time, i.e. the time that needs to be predicted, and the moment before the target time is also a future time, then the target predicted image frame predicted using the video prediction method provided in the embodiments of this application for the moment before the target time will be used as the target image frame. If the moment before the target time is not a future time, then the image frame actually generated at the moment before the target time will be used as the target image frame.

[0095] S102: Input the target image frame into the quantum autoencoder model in any of the above embodiments that has been trained to obtain the target predicted image frame.

[0096] If the target time is not the last time in the predicted time series, the video prediction method provided in this application embodiment needs to be executed again until the prediction of the last time in the predicted time series is completed. It should be noted that the video prediction method provided in this application embodiment is performed cyclically. When the target time is not the predicted time, the main purpose of prediction is to obtain the relationship between adjacent image frames. The target image frame input to the quantum autoencoder model is still the image frame actually generated at the corresponding time. Based on this, the quantum autoencoder model can understand the relationship between image frames and use quantum computing to overcome the limitations of computing power to capture the dynamic changes and details of image frames, reducing the deviation between the prediction result and the actual situation, thereby improving the prediction accuracy.

[0097] This application also provides a video prediction device, see [link to relevant documentation]. Figure 11 The device includes:

[0098] The first acquisition module 1101 is used to acquire a target image frame, wherein the target image frame is the image frame 1102 corresponding to the time before the target time.

[0099] The second acquisition module is used to input the target image frame into the quantum autoencoder model of any of the above embodiments that has been trained, and to obtain the target predicted image frame.

[0100] The specific functions and effects of the video prediction method and device described above can be explained by referring to other embodiments of this application, and will not be repeated here. Each module in the video prediction device can be implemented entirely or partially through software, hardware, or a combination thereof. Each module can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0101] Please see Figure 12This application also provides a computer device, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the quantum autoencoder model or video prediction method in any of the above embodiments. Please refer to [link to relevant documentation]. Figure 12 The computer device can be a classical computer or a quantum computer.

[0102] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a computer, causes the computer to perform the quantum autoencoder model or video prediction method in any of the above embodiments.

[0103] This application also provides a computer program product containing instructions that, when executed by a computer, cause the computer to perform the quantum autoencoder model or video prediction method in any of the above embodiments.

[0104] It is understood that the specific examples in this application are only intended to help those skilled in the art better understand the implementation methods of this application, and are not intended to limit the scope of this application.

[0105] It is understood that in the various embodiments of this application, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not limit the implementation process of the embodiments of this application in any way.

[0106] It is understood that the various implementation methods described in this application can be implemented individually or in combination, and the implementation methods in this application are not limited in this respect.

[0107] Unless otherwise stated, all technical and scientific terms used in the embodiments of this application have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The term "and / or" as used in this application includes any and all combinations of one or more of the associated listed items. The singular forms "a," "the," and "the" as used in the embodiments of this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0108] It is understood that the processor in the embodiments of this application can be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method embodiments can be completed by the integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory; the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0109] It is understood that the memory in the embodiments of this application may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. Specifically, non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may be random access memory (RAM). It should be noted that the memory in the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0110] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0111] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the aforementioned method implementations, and will not be repeated here.

[0112] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0113] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0114] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0115] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0116] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A quantum autoencoder model, characterized in that, The quantum autoencoder model includes an encoder, an intermediate module, and a decoder connected in series, and each of the encoder, the intermediate module, and the decoder includes different variable quantum circuits. The encoder is used to extract features of the input target image frame, wherein the target image frame is the image frame corresponding to the time before the target time. The intermediate module is used to generate an initial prediction image frame by utilizing the features extracted by the encoder and the relationship between adjacent image frames constructed during prediction at the time preceding the target time. The decoder is used to reconstruct the initial predicted image frame to obtain the target predicted image frame at the target time.

2. The model as described in claim 1, characterized in that, When both the target time and the time before the target time are future times, the target data is the target prediction image frame obtained by predicting the time before the target time; When the time preceding the target time is not a future time, the target data is the image frame actually generated at the time preceding the target time.

3. The model as described in claim 1 or 2, characterized in that, The encoder includes a first convolutional neural network (CNN) layer, a variable quantum network layer, and a second CNN layer connected in series. The variable quantum network includes a first quantum circuit and a second quantum circuit, and the circuit structures of the first quantum circuit and the second quantum circuit are different. The second CNN layer extracts features from the fused data, wherein the fused data is obtained by fusing the data output by the first quantum circuit and the second quantum circuit.

4. The model as described in claim 3, characterized in that, The first quantum circuit includes a coding layer consisting of an H gate and a first RZ gate acting on each qubit, and a variable layer consisting of controlled modules; Each controlled module includes a CNOT gate, a second RZ gate, a first RX gate, and a CNOT gate in sequence. The number of controlled modules is n*(n-1) / 2, where n is the number of qubits in the first quantum circuit.

5. The model as described in claim 3, characterized in that, The second quantum circuit includes, in sequence, an H gate, an RY gate, a second RX gate, a Y gate, a Z gate, a controlled RZZ gate, and a CNOT gate acting on each qubit.

6. The model as described in claim 3, characterized in that, The decoder includes a first deconvolutional neural network layer, a first quantum circuit, and a second deconvolutional neural network layer, wherein... One output of the first deconvolutional neural network layer is connected to the input of the first quantum circuit; The other output of the first deconvolutional neural network layer is connected to the output of the first quantum circuit and the input of the second deconvolutional neural network layer through a residual structure.

7. The model as described in claim 1, characterized in that, The intermediate module includes M third quantum circuits and N fourth quantum circuits; The M third quantum circuits are used to extract the time features and first feature data of the first input data. The first input data includes the features extracted by the encoder and the target time feature data. The target time feature data is the time feature data predicted based on the target data of the previous moment. The first feature data is the feature data extracted by the M quantum circuits and used as input to the fourth quantum circuit. The N fourth quantum circuits are used to extract the spatial features and second feature data of the second input data. The second input data includes first feature data and target spatial feature data. The target spatial feature data is spatial feature data predicted based on the target data of the previous time step. The second feature data is feature data used to generate the initial predicted image frame. The relationship between adjacent image frames includes the target temporal feature data and the target spatial feature data.

8. The model as described in claim 7, characterized in that, The third quantum circuit and the fourth quantum circuit have the same structure, both including a coding layer composed of an H gate and a third RZ gate acting on each quantum bit, and a variable layer composed of a CNOT gate, a U3 gate and a third RX gate.

9. A video prediction method, characterized in that, The method includes: Obtain the target image frame, wherein the target image frame is the image frame corresponding to the time before the target time. The target image frame is input into the quantum autoencoder model of any one of claims 1-8 after training to obtain the target predicted image frame.

10. A video prediction device, characterized in that, The device includes: The first acquisition module is used to acquire a target image frame, wherein the target image frame is the image frame corresponding to the time before the target time. The second acquisition module is used to input the target image frame into the quantum autoencoder model of any one of claims 1-8 after training, and obtain the target predicted image frame.

11. A computer device comprising a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the quantum autoencoder model of any one of claims 1-8 or the video prediction method of claim 9.

12. A computer-readable storage medium having a computer program stored thereon, which, when executed by a computer, causes the computer to perform the quantum autoencoder model of any one of claims 1-8 or the video prediction method of claim 9.

Images