Quantum self-attention neural network model and construction method thereof

Abstract

The invention discloses a quantum self-attention neural network model and a construction method thereof. The model comprises a quantum mixed state embedding module, a quantum self-attention module, a quantum position coding module and an output module. According to the method, the expression ability of a quantum system is expanded through mixed state operation, so that the model can capture the similarity between query and keys more comprehensively and finely, meanwhile, more efficient data representation and processing are achieved, the number of parameters is remarkably reduced under the same input sequence condition, and more excellent performance is shown; and higher learning ability is shown.

Description

Technical Field

[0001] The present invention relates to the field of quantum machine learning, and in particular to a quantum self-attention neural network model and a construction method thereof. Background Art

[0002] Over the past few decades, quantum computing has achieved unprecedented computing speed and efficiency in many fields such as biology, cryptography, and communication technology by leveraging the principles of quantum mechanics. This has promoted the integration of quantum methods with machine learning, especially in the field of quantum machine learning (QML). This emerging field combines the principles of quantum computing with traditional machine learning techniques to address complex challenges in data processing and pattern recognition. Initially, the field of natural language processing (NLP) also made significant progress through the development of deep learning models such as convolutional neural networks (CNNs) and recurrent neural networks (RNNs), which greatly improved tasks such as text classification, sentiment analysis, and machine translation. Recently, the introduction of the Transformer model and OpenAI's GPT series has further expanded the capabilities of NLP, enabling models to generate complex texts and interact naturally with humans, and even extending their creativity to composing, literary creation, and software development. However, as the model size and data volume grow, traditional deep learning methods face huge challenges in computing resource requirements and energy efficiency.

[0003] In the field of quantum natural language processing, researchers have proposed a variety of models to take advantage of quantum computing. Based on the weak measurement principle in quantum mechanics, Niu et al. proposed a parameter-free model that combines quantum attention mechanism with LSTM to improve the performance of sentence modeling. In addition, the Baidu team proposed QSANN (Quantum Self-Attention Neural Network), a model that uses Gaussian projection quantum self-attention for text classification, although it may encounter information loss problems when processing quantum queries and keys. Zhao et al. further proposed QSAN (Quantum Self-Attention Network), which uses quantum logical similarity (QLS) and quantum bit self-attention scoring matrix (QBSASM) to enhance information extraction capabilities. They also proposed QKSAN (Quantum Kernel Self-Attention Network), a model that combines quantum kernel methods and self-attention mechanisms to provide a broader data representation space.

[0004] Although these models compute similarities at the quantum level, they are primarily confined to the level of pure states and rely on unitary transformations of quantum circuits, which limits their expressive power. Summary of the invention

[0005] The present invention aims to provide a quantum self-attention neural network model and a construction method thereof, which is based on trainable quantum embedding, mixed-state quantum attention coefficients and untrainable quantum position information embedding, significantly reduces the number of parameters under the same input sequence conditions, and exhibits stronger data representation and information extraction capabilities.

[0006] According to one aspect of the present invention, a quantum self-attention neural network model is provided, which is applied to natural language processing, and includes: a quantum mixed state embedding module, which is used to convert classical data into quantum states through quantum feature mapping and variational quantum circuits, and obtain mixed state queries and keys; a quantum self-attention module, which is used to calculate the similarity between the mixed state query and the key through an exchange test quantum circuit within the quantum state dimension generated by the quantum mixed state embedding module; a quantum position encoding module, which encodes position information by introducing additional fixed quantum gates in the quantum self-attention module without the need for additional quantum bit resources; an output module, which calculates the self-attention coefficient of the quantum state through the mixed state query and the key through the exchange test circuit, multiplies it with the expected value of the observable quantity Z corresponding to the mixed state value, extracts the output encoding the semantic information of the input data, and constructs the quantum self-attention neural network model.

[0007] Preferably, the quantum hybrid state embedding module comprises: a data conversion unit for converting the classical data x into s Direct conversion to the quantum state |x s,q >、|x s,k > and |x s,v >, where 1≤s≤S, S is the number of input vectors in the data sample; the quantum state processing unit processes the quantum state through the trainable and adaptive variational quantum circuit to obtain the pure state query |x s,q > and key and |x s,k > vector; partial trace operation unit, which performs partial trace operation on the pure state query and key vector to obtain the mixed state query ρ q and key σ k , expressed as: ρ s,q =tr B (|x s,q > <x s,q |),σ s,k =tr B (|x s,k > <x s,k |) Among them, tr B (·) represents the partial trace operation on system B, ρ s,q and σ s,k represent the quantum mixed state density operators for query and key respectively.

[0008] Preferably, the variational quantum circuit comprises: a single quantum bit gate R x (x i ), which is used to convert the input data x=(x1,…,x N ) T Encoding into quantum circuits; two-qubit gates For entangled qubits and adding R y (θ2); The trainable parameter θ is used to control the transformation of the quantum circuit, which is: Where θ is a trainable parameter and L is a layer; each layer consists of a data encoding circuit block V(x) and a trainable parameter θ l The trainable circuit block W(θ l )composition.

[0009] Preferably, the quantum self-attention module comprises: an exchange test quantum circuit unit, which is used to calculate the quantum self-attention coefficient between the query and the key quantum state through an exchange test, expressed as: α s,j =tr(ρ s,q σ j,k ) Among them, ρ s,q and σ j,k are the mixed quantum states of query and key, respectively; A measurement unit to maintain the quantum information of the query and key at the quantum level; A similarity calculation unit is used to calculate the similarity between the query and the bond mixed quantum state based on the Hilbert-Schmidt distance, which is expressed as: D HS (ρ,σ)=tr((ρ-σ) 2 ) where ρ and σ represent the quantum states of the query and bond, respectively.

[0010] Preferably, the measurement unit comprises: a quantum bit for representing a query and a key; an auxiliary quantum bit for performing a swap test; a measurement device for measuring and calculating an attention coefficient; and the similarity calculation unit determines the quantum self-attention coefficient by normalization processing, which is expressed as: Among them, α s,j Represented as the unnormalized quantum self-attention coefficient between the sth and jth data, α s,m It is expressed as the unnormalized quantum self-attention coefficient between the sth and mth data, and S represents the total number of data in a sample.

[0011] Preferably, the quantum position encoding module comprises: a position encoding unit for encoding position information into a quantum circuit; a quantum gate array unit for implementing the encoding of the position information, including sinusoidal position encoding, expressed as: Among them, s represents the position, i represents the dimension, and d model Represents the dimension of the embedding.

[0012] Preferably, the model further comprises: a loss function for training the model on the dataset by minimizing the loss: Where Θ represents all trainable parameters in the quantum circuit, w and b represent the weight and bias of the fully connected layer, respectively. is the data set, N s represents the total number of samples, represents the predicted probability value, Indicates the actual label.

[0013] According to another aspect of the present invention, a method for constructing a self-attention neural network model is provided, which is applied to natural language processing, including: converting classical data into quantum states through quantum feature mapping and variational quantum circuits, and obtaining mixed state queries and keys; calculating the similarity between the mixed state query and the key by exchanging test quantum circuits within the quantum state dimension generated by the quantum mixed state embedding module; encoding position information by introducing additional fixed quantum gates in the quantum self-attention module without the need for additional quantum bit resources; and extracting the output encoding the semantic information of the input data according to the similarity between the mixed state query and the key through measurement and post-processing of the quantum state, thereby constructing the quantum self-attention neural network model.

[0014] Preferably, converting the classical data into a quantum state through quantum feature mapping and a variational quantum circuit, and obtaining a mixed state query and key comprises: using quantum feature mapping to convert the classical data x s Direct conversion to the quantum state |x s,q >、|x s,k > and |x s,v >, where 1≤s≤S, S is the number of input vectors in the data sample; the quantum state is processed by the trainable and adaptive variational quantum circuit to obtain the pure state query |x s,q > and key and |x s,k > vector; perform partial trace operations on the pure state query and key vector to obtain the mixed state query ρ q and key σ k , expressed as: ρ s,q =tr B (|x s,q > <xs,q |),σ s,k =tr B (|x s,k > <x s,k |) Among them, tr B (·) represents the partial trace operation on system B.

[0015] The invention discloses a quantum self-attention neural network model, including a quantum mixed state embedding module, which is used to convert classical data into quantum states through quantum feature mapping and variational quantum circuits, and obtain mixed state queries and keys; a quantum self-attention module, which is used to calculate the similarity between the mixed state query and the key through an exchange test quantum circuit within the quantum state dimension generated by the quantum mixed state embedding module; a quantum position encoding module, which encodes position information by introducing additional fixed quantum gates in the quantum self-attention module without the need for additional quantum bit resources; an output module, which calculates the self-attention coefficient of the quantum state through the mixed state query and the key through the exchange test circuit, multiplies it with the expected value of the observable quantity Z corresponding to the mixed state value, extracts the output encoding the semantic information of the input data, and constructs the quantum self-attention neural network model. The invention expands the expression ability of the quantum system through mixed state operations, enables the model to capture the similarity between the query and the key more comprehensively and finely, and realizes more efficient data representation and processing at the same time, and not only significantly reduces the number of parameters under the same input sequence conditions, but also shows more superior performance and stronger learning ability. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0017] Figure 1 A quantum hybrid state self-attention neural network framework diagram according to an embodiment of the present invention;

[0018] Figure 2 A quantum self-attention heat map according to an embodiment of the present invention;

[0019] Figure 3 Graphs showing test accuracy under different scaling forms of data according to an embodiment of the present invention;

[0020] Figure 4 Three quantum embedding circuit diagrams with different entanglement layers according to embodiments of the present invention, wherein (a) is an extended nearest neighbor (ENN) circuit diagram, (b) is a loop (CB) circuit diagram, and (c) is an all-connected (AA) circuit diagram;

[0021] Figure 5A quantum circuit diagram for implementing an exchange test according to an embodiment of the present invention;

[0022] Figure 6 A schematic diagram of introducing position coding into a quantum embedded circuit according to an embodiment of the present invention. DETAILED DESCRIPTION

[0023] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0024] Reference will now be made in detail to various embodiments of the present invention, examples of which are shown in the accompanying drawings and described below. For ease of interpretation and precise definition in the appended claims, the terms "upper", "lower", "inner" and "outer" are used to describe features of the exemplary embodiments shown in the drawings with reference to their positions.

[0025] The present invention protects a quantum self-attention neural network model, which can also be called a quantum self-attention neural network device.

[0026] Figure 1 , 2 3 respectively show the quantum mixed state self-attention neural network framework diagram, quantum self-attention heat map and test accuracy diagram under different data scaling forms according to the embodiment of the present invention. As shown in the figure, the embodiment of the present invention provides a quantum self-attention neural network model, which is applied to natural language processing, including: a quantum mixed state embedding module, which is used to convert classical data into quantum state through quantum feature mapping and variational quantum circuit, and obtain mixed state query and key; a quantum self-attention module, which is used to calculate the similarity between the mixed state query and the key by exchanging test quantum circuit within the quantum state dimension generated by the quantum mixed state embedding module; a quantum position encoding module, which encodes position information by introducing additional fixed quantum gates in the quantum self-attention module without additional quantum bit resources; an output module, which calculates the self-attention coefficient of the quantum state through the mixed state query and key through the exchange test circuit, multiplies it with the expected value of the observable quantity Z corresponding to the mixed state value, extracts the output encoding the semantic information of the input data, and constructs the quantum self-attention neural network model.

[0027] In the embodiment of the present invention, the quantum self-attention neural network model first converts the input data into quantum states through the quantum mixed state embedding module, then processes these quantum states through the quantum self-attention mechanism, calculates the attention weights, and combines the quantum position encoding scheme to enhance the model's understanding of sequence data. Finally, the model outputs a weighted quantum state representation that reflects the importance and position information of each element in the sequence.

[0028] Figure 4 The diagrams of three quantum embedding circuits with different entanglement layers according to an embodiment of the present invention are shown, wherein (a) is an extended nearest neighbor (ENN) circuit, (b) is a loop (CB) circuit, and (c) is an all-connected (AA) circuit. According to an embodiment of the present invention, the quantum mixed state embedding module includes: a data conversion unit for converting the classical data x into a quantum feature map using quantum feature mapping. s Direct conversion to the quantum state |x s,q >、|x s,k > and |x s,v >, where 1≤s≤S, S is the number of input vectors in the data sample; the quantum state processing unit processes the quantum state through the trainable and adaptive variational quantum circuit to obtain the pure state query |x s,q > and key and |x s,k > vector; partial trace operation unit, which performs partial trace operation on the pure state query and key vector to obtain the mixed state query ρ q and key σ k , expressed as: ρ s,q =tr B (|x s,q > <x s,q |),σ s,k =tr B (|x s,k > <x s,k |) Among them, tr B (·) represents the partial trace operation on system B, ρ s,q and σ s,k represent the quantum mixed state density operators for query and key respectively.

[0029] According to an embodiment of the present invention, the variational quantum circuit includes: a single quantum bit gate R x (x i ), which is used to convert the input data x=(x1,…,x N ) T Encoding into quantum circuits; two-qubit gates For entangled qubits and adding R y (θ2); The trainable parameter θ is used to control the transformation of the quantum circuit, which is: Where θ is a trainable parameter and L is a layer; each layer consists of a data encoding circuit block V(x) and a trainable parameter θ l The trainable circuit block W(θ l )composition.

[0030] Figure 5A quantum circuit diagram for implementing an exchange test according to an embodiment of the present invention is shown. As shown in the figure, according to an embodiment of the present invention, the quantum self-attention module includes: an exchange test quantum circuit unit, which is used to calculate the quantum self-attention coefficient between the query and the key quantum state through the exchange test, expressed as: α s,j =tr(ρ s,q σ j,k ) Among them, ρ s,q and σ j,k are the mixed quantum states of query and key, respectively; A measurement unit to maintain the quantum information of the query and key at the quantum level; A similarity calculation unit is used to calculate the similarity between the query and the bond mixed quantum state based on the Hilbert-Schmidt distance, which is expressed as: D HS (ρ,σ)=tr((ρ-σ) 2 ) where ρ and σ represent the quantum states of the query and bond, respectively.

[0031] According to an embodiment of the present invention, the measurement unit includes: a quantum bit for representing a query and a key; an auxiliary quantum bit for performing an exchange test; a measurement device for measuring and calculating an attention coefficient; the similarity calculation unit determines the quantum self-attention coefficient by normalization processing, which is expressed as: Among them, α s,j Represented as the unnormalized quantum self-attention coefficient between the sth and jth data, α s,m It is expressed as the unnormalized quantum self-attention coefficient between the sth and mth data, and S represents the total number of data in a sample.

[0032] Figure 6 A schematic diagram of introducing position coding in a quantum embedded circuit according to an embodiment of the present invention is shown. As shown in the figure, according to an embodiment of the present invention, the quantum position coding module includes: a position coding unit, which is used to encode position information into a quantum circuit; a quantum gate array unit, which is used to implement the encoding of the position information, including sinusoidal position coding, which is expressed as: Among them, s represents the position, i represents the dimension, and d model Represents the dimension of the embedding.

[0033] According to an embodiment of the present invention, the model further includes: a loss function, which is used to train the model on the data set by minimizing the loss: Where Θ represents all trainable parameters in the quantum circuit, w and b represent the weight and bias of the fully connected layer, respectively. is the data set, N s represents the total number of samples, represents the predicted probability value, Indicates the actual label.

[0034] According to another embodiment of the present invention, a method for constructing a self-attention neural network model is provided, which is applied to natural language processing and is characterized in that it includes: converting classical data into quantum states through quantum feature mapping and variational quantum circuits, and obtaining mixed state queries and keys; calculating the similarity between the mixed state query and the key by exchanging test quantum circuits within the quantum state dimension generated by the quantum mixed state embedding module; encoding position information by introducing additional fixed quantum gates in the quantum self-attention module without the need for additional quantum bit resources; and extracting the output encoding the semantic information of the input data based on the similarity between the mixed state query and the key through measurement and post-processing of the quantum state to construct the quantum self-attention neural network model.

[0035] According to an embodiment of the present invention, converting classical data into a quantum state through quantum feature mapping and a variational quantum circuit, and obtaining a mixed state query and key includes: using quantum feature mapping to convert the classical data x s Direct conversion to the quantum state |x s,q >、|x s,k > and |x s,v >, where 1≤s≤S, S is the number of input vectors in the data sample; the quantum state is processed by the trainable and adaptive variational quantum circuit to obtain the pure state query |x s,q > and key and |x s,k > vector; perform partial trace operations on the pure state query and key vector to obtain the mixed state query ρ q and key σ k , expressed as: ρ s,q =tr B (|x s,q > <x s,q |),σ s,k =tr B (|x s,k > <x s,k |) Among them, tr B (·) represents the partial trace operation on system B.

[0036] This invention extends the concept of quantum embedding to mixed state representation, providing a more general and powerful data representation framework. Mixed states provide enhanced representation capabilities by enabling the system to encode and process superpositions of multiple quantum states simultaneously. This capability is particularly advantageous in the context of quantum self-attention mechanisms, as the complexity and diversity of data relationships require robust and flexible representation methods.

[0037] When designing a quantum self-attention network, a straightforward and natural way to compute the similarity between a query and a key is to use the inner product: α s,j =| ,q |x j,k >| 2 (1)

[0038] However, in quantum circuits, since only unitary transformations are performed, for the same number of qubits, |x s,q > and |x j,k > can be viewed as a rotation operation, since they are both in the same Hilbert space. In contrast, in a classic self-attention network, the query vector and key vector are: q=xW q ,k=xW k (2) Where x is the input vector, W q and W k is the corresponding coefficient matrix. W q and W k changes the direction and length of x. The relationship between q and k involves rotation and scaling. Therefore, this makes |x s,q >It is difficult to simultaneously associate those |x that are far away s,j >. This direct approach is not the best way to calculate the quantum attention coefficient.

[0039] Based on the above analysis, the present invention proposes a hybrid state-based quantum self-attention network to solve these shortcomings. By performing partial trace operations on pure state queries and key vectors, the present invention obtains the hybrid state query ρ q and key σ k Although this operation reduces their dimensionality, it also introduces changes in the direction and magnitude of the Bloch vectors in the Bloch sphere.

[0040] Based on the principle of using hybrid states, the embedding circuits are designed to transform classical data inputs into quantum states, employing quantum feature maps, followed by trainable, adaptive variational quantum circuits to facilitate their integration into the quantum self-attention mechanism.

[0041] ​Typically, quantum machine learning methods focus on using fixed quantum feature maps and then applying trainable, adaptive variational quantum circuits to adjust the measurement basis. This architecture, including data encoding circuits and learnable quantum neural networks (QNNs), is widely used in various scenarios, such as quantum convolutional neural networks, quantum recurrent neural networks, strongly entangled circuit architectures, and search architectures. However, this approach requires careful design of the encoding circuits because the fixed quantum feature maps significantly affect the generalization performance of the algorithm, and most of the computing resources are used for the QNN. However, references show that this may not be the most effective approach.

[0042] If the data has been well mapped to the Hilbert space, then subsequent tasks can be accomplished by a shallow quantum classifier circuit. This is similar to the "feature extractor" in classical machine learning, where the network is trainable and its core purpose is to convert or encode input data (such as images, text, or audio) into a new feature space. This feature space more effectively represents the key information of the data, thereby facilitating subsequent tasks such as classification, regression, or clustering. Therefore, the present invention can focus the adaptive training of quantum circuits on training a trainable quantum feature map. This maps classical data to Hilbert space. The present invention calls this process quantum embedding.

[0043] The present invention introduces a repetitive quantum embedding architecture, such as Figure 4 As shown. In the classical self-attention network, there are three parts: query, key and value. Similarly, in the quantum self-attention network of the present invention, the present invention trains three quantum embeddings. Through these embeddings, the present invention converts the classical data x s Represented as a quantum state |x s,q >、|x s,k > and |x s,v >, where 1≤s≤S, S represents the number of input vectors in the data sample.

[0044] Specifically, these three mappings to |x q >、|x k > and |x v >Using the same quantum circuit structure, different parameters θ are used q ,θ k and θ v The quantum circuit uses single-qubit and double-qubit quantum gates. First, the present invention uses a single-qubit gate R x (x i ) Input data x=(x1,…,x N ) T Then, the present invention uses gate to entangle the qubits and add R y(θ2). In order to enhance the expressiveness of the quantum circuit, this structure can include L layers. Finally, the present invention adds a single quantum bit gate R again in the last layer. x (x i ) to encode data. Therefore, the entire circuit can be expressed as U emb (x,θ), where x is the input data and θ is a trainable parameter. Each layer consists of a data encoding circuit block V(x) and a trainable parameter θ l The trainable circuit block W(θ l )composition.

[0045] Therefore, through three trainable quantum embeddings, the present invention converts the input data x s Embedded into three quantum states:

[0046] For |x s,q > and |x s,k >, they are transformed by unitary transformation U emb (x,θ) from the initial state Get, so |x s,q > and |x s,k > are all pure states. In order to obtain a mixed state, the present invention extracts information from the first n / 2 qubit subsystem A of the entire n-qubit quantum system by performing a partial trace operation on the quantum system and discarding the remaining n / 2 qubit subsystem B. Specifically, this operation converts the pure state of the entire system |x q > and |x k >Converted to the mixed state ρ of the corresponding subsystem respectively q and σ k : ρ s,q =tr B (|x s,q > <x s,q |)σ s,k =tr B (|x s,k > <x s,k |) (5) Among them, tr B (·) represents the partial trace operation on system B.

[0047] The obtained state ρ s,q and σ s,k In the same dimensional space represented by Bloch vectors, they differ not only in the direction of the vectors but also in the magnitude of the vectors.

[0048] The quantum self-attention mechanism of the present invention directly calculates the similarity of queries and keys within the quantum state dimension, maintaining its native quantum form throughout the calculation process until the final measurement step. This method ensures the complete preservation of quantum information and utilizes the unique properties of quantum mechanics to enhance the efficiency and accuracy of similarity calculations between network elements. By keeping quantum information at the quantum level without converting it to classical data prematurely, the method of the present invention avoids the information loss typically associated with methods based on observables. This not only prevents the degradation of quantum data fidelity, but also bypasses the limitations of traditional methods where significant quantum details are lost when transitioning to classical representations.

[0049] The method of calculating the attention coefficient for mixed state query and key is inspired by the Hilbert-Schmidt distance. The Hilbert-Schmidt distance is an important distance metric in quantum information theory. It can be measured and optimized by a small quantum circuit, which is of great significance for near-term quantum computing. It is defined as follows: D HS (ρ,σ)=tr((ρ-σ) 2 ) (6)

[0050] Expanding the above equation yields three different terms: tr(ρσ), tr(σ 2 ) and tr(ρ 2 ). The term tr(ρσ) quantifies the similarity of two quantum systems in Hilbert space by the similarity between the systems; when tr(ρσ) = 1, it means that the two quantum systems are composed of the same pure state, and tr(ρσ) = 0 means that all two quantum systems are completely orthogonal. 2 ) and tr(σ 2 ) evaluates the degree of mixing within the quantum system. In the quantum self-attention network, because the present invention pays more attention to the similarity between the query and the key, the present invention ignores tr(σ 2 ) and tr(ρ 2 ), and finally only use tr(ρσ) as a method to calculate the similarity between query and key in quantum self-attention network.

[0051] The present invention defines the quantum self-attention coefficient between the sth and jth mixed states, calculated from the corresponding query and key parts: α s,j =tr(ρ s,q σ j,k ) (7)

[0052] The above equation can be easily implemented by exchanging test quantum circuits, such as Figure 5 Next, the present invention introduces the basic principle of the circuit to implement formula (7).

[0053] Assume that the present invention has a pair of mixed states ρ and σ of n qubits, where ρ = ∑ i p i |e i > <e i | and σ = ∑ i q i |f i > <f i |Through their respective orthogonal bases|e i > and |f i > decomposition. If the present invention measures the auxiliary quantum bit and obtains the result |0>, the exchange test passes, otherwise it fails. Mixed state The probability of passing the swap test is:

[0054] Therefore, the present invention uses an exchange test quantum circuit to realize the calculation of the quantum self-attention coefficient between the query and the key. This can effectively estimate the proximity of two mixed states. If the two mixed states are the same, that is, ρ = σ, the test always passes with p = 1. When the states are different, the finite probability p of passing the test depends on the similarity tr(ρσ) between the two states; the closer they are, the greater the probability of passing the test. The output solution process is described in matrix form, and the coefficient matrix can be expressed as: in, Represents the normalized quantum self-attention coefficient:

[0055] For the value part, the present invention uses an n-dimensional vector to represent it, and measures the observable quantity Z for each quantum bit to obtain a vector with the same dimension as the number of quantum bits.

[0056] Finally, output y s The classic form of can be calculated by the following formula. The present invention adopts the structure of residual network to design the output to prevent network degradation:

[0057] The QMSAN of the present invention is similar to the classic self-attention network, which can model the relationship between the word segments in the sequence and capture the contextual representation of a given word segment, and has an excellent ability to capture long-range dependencies. However, an inherent limitation of the self-attention network is that they cannot capture the order of the input word segments. Therefore, in order to enable the model to utilize the sequential information of the word segments, the present invention must inject some information about the position of the word segment in the sequence.

[0058] For quantum position coding, using the entanglement characteristics of quantum systems for encoding is an effective strategy to encode classical position information into one or more additional qubits through quantum embedding. Subsequently, a trainable QNN circuit fully entangles the qubit containing the position information with the data qubit, so that the output quantum state of the entire system contains the position information. However, this method requires additional qubit resources. In this study, the inventors adopted a different approach, such as Figure 6 As shown. The present invention designs a quantum circuit that eliminates the need for auxiliary quantum bits by sacrificing circuit depth, thereby saving quantum bit resources. The present invention introduces more quantum gate operations into the quantum circuit to achieve effective encoding of position information.

[0059] The present invention introduces sinusoidal position coding into quantum circuits. In classical sinusoidal position coding, the vector values ​​corresponding to even and odd positions are: Among them, s is the position, i is the dimension, and the dimension of the position encoding is the same as the dimension of the embedding d model same.

[0060] The present invention scales the positional encoding data to the range [0,2π]:

[0061] For input data x, the present invention scales its range to [0,π]: Among them, x min and x max Represents the minimum and maximum values ​​of all elements in all vectors in the input data set, PE min and PE max are the minimum and maximum values ​​of the position-encoded data.

[0062] Since the quantum bit revolving door R x The characteristics of (θ) are used to encode the position PE (s) and input data x s This gate rotates the quantum state counterclockwise around the X-axis by an angle θ, where θ is valid in the range [0,2π], representing a complete cycle. For a periodic PE(s), its period naturally conforms to R x The present invention thus ensures that PE (s) The value is converted to the range [0,2π] to match R x The operation cycle of the gate is consistent. However, for the original input data x s, since it is not periodic, directly scaling it to [0,2π] may cause changes in physical properties. Specifically, if it is encoded in the range of [0,2π], the encoded quantum state has R x (2π-x′ s )=-ZR x (x′ s )Z this specific transformation relationship, which in the original data x s Therefore, special considerations need to be paid attention to when processing different types of data in the quantum coding process to ensure that the physical meaning of the data and the encoding effect are properly reflected. Through this delicate data processing and using the same single-qubit gates to process traditional data and position information, the present invention can more efficiently convert classical information into states in quantum circuits.

[0063] The present invention minimizes the loss function in the data set The model of the present invention is trained on s Indicates the total number of samples and the label of each sample Indicates its category.

[0064] For each sample, S m Indicates the number of words it contains, each input data x m,s is an n-dimensional vector.

[0065] The feature vector of each sample is outputted by m,s Sum and average to get, where 1≤m≤N s And 1≤s≤S m :

[0066] The output y for each sample m is fed into a fully connected layer to produce a binary prediction for each sample Among them, w and b represent the weight and bias of the fully connected layer respectively, and σ represents the sigmoid activation function.

[0067] For classification problems, there are many loss functions to choose from, such as cross entropy loss and mean square error (L2 loss). In the current work, the present invention uses a simple and effective mean square error as the loss function: Here, Θ represents all trainable parameters in the quantum circuit.

[0068] In addition, the present invention demonstrates simulation experiments on text classification tasks conducted on the Tensorcircuit platform. These experiments are intended to evaluate the effectiveness of the model of the present invention in processing text data of various types and sizes, as well as its robustness to noise perturbations. To this end, the present invention selects two types of public data sets to verify the effectiveness of the model of the present invention. The first type of data set includes simple sentences or phrases, which are intended to evaluate the performance of the model in understanding the meaning and grammatical structure of sentences. Such data sets include MC and RP data sets. The second type of data set is the Sentiment Labeled Sentences Data Set, which contains three subsets of real user reviews, mainly used to evaluate the model's ability to analyze sentence sentiment, including Yelp, IMDb and Amazon data sets. The study first evaluated the model that did not contain position encoding information on the two types of data sets. Subsequently, an in-depth analysis was conducted on the model that integrated position encoding information on the second type of data set. Considering that it is impossible to completely eliminate noise in a real-noise intermediate-scale quantum (NISQ) device, the model will be affected by various errors. This study simulated the effects of three types of noise, depolarization, phase damping and amplitude damping, on the model results in a simulated environment to evaluate the robustness of the model to noise.

[0069] In the MC (Meaning Classification) task, there are 130 sentences (70 training + 30 development + 30 test), each with 3 or 4 words. Half of the sentences are related to food and the other half are related to information technology (IT). The vocabulary of the task consists of 17 words, some of which are shared in both categories.

[0070] In the RP (RELPRON) task, there are 105 noun phrase sentences (74 training + 31 testing) containing relative clauses, each with 4 words. The vocabulary of the task has 115 words, and the phrases are selected to ensure that each word appears at least three times in the dataset. The goal is to determine whether a noun phrase contains a subject or object relative clause. The larger vocabulary and the resulting word sparsity make this task a more challenging benchmark compared to the MC task.

[0071] The Sentiment Labelled Sentences Data Set includes reviews from three websites: Amazon, IMDb, and Yelp, each of which provides 1,000 sentiment-labeled sentences. The average length of sentences in this dataset is 10.2 words, the shortest sentence is 1 word, and the longest sentence is 30 words. The IMDb dataset contains movie reviews from the imdb.com website, with an average sentence length of 14.4 words. The shortest sentence in this dataset is 1 word and the longest sentence is 71 words. The Yelp dataset contains restaurant reviews, with an average sentence length of 10.9 words, the shortest is 1 word, and the longest is 32 words. In all datasets, reviews with ratings of 4 and 5 are considered positive, while reviews with ratings of 1 and 2 are considered negative. In each subset, positive and negative sentiment reviews each account for 50%. The dataset randomly selects 80% of the data as the training set and the remaining 20% ​​as the test set. Table 1 Experimental configuration Where LR represents the learning rate, -P represents the model with quantum position encoding, and -NP represents the model without quantum position encoding.

[0072] In order to fairly compare the experimental results of the present invention with those in the existing literature, the present invention sets the number of qubits of quantum embedding input to be the same as theirs. Specifically, the present invention uses n = 2 qubits in the MC task and n = 4 qubits in other tasks. The detailed hyperparameter settings are shown in Table 1.

[0073] In order to explore the potential advantages of the quantum bit topology in the QMSAN, the present invention compares three different entanglement modes in the quantum embedding module: extended nearest neighbor (ENN) quantum circuit, loop circuit block (CB) quantum circuit and fully interconnected (AA) quantum circuit, such as Figure 4 As shown. The CB quantum circuit is characterized by a ring-shaped arrangement of qubits, which is suitable for efficient closed-loop circuit operations. The extended nearest neighbor (ENN) quantum circuit evolved from the nearest neighbor (NN) quantum circuit and has the same number of parameters as the CB quantum circuit, but introduces a different entanglement strategy. This change allows the results produced by different entanglement methods to be compared experimentally. In contrast, the AA quantum circuit is based on a fully connected network of qubits, allowing direct interaction between any pair of qubits, thereby enhancing the entanglement capability.

[0074] For the sake of clarity, the present invention represents these entanglement-based QMSAN variants as QMSAN-ENN, QMSAN-CB, and QMSAN-AA, respectively. In addition, the present invention introduces suffixes to indicate whether the novel quantum position coding of the present invention is included: '-P' indicates a model including quantum position coding, and '-NP' indicates a model not including quantum position coding. For example, QMSAN-ENN-P indicates a QMSAN model using ENN quantum circuits and position coding, while QMSAN-ENN-NP indicates a variant that does not include position coding.

[0075] In addition, the present invention proposes a variant called QMSAN-D2π, in which the input data is scaled to the range of [0,2π]. This variant is used as a control model to evaluate the impact of different scaling methods on quantum embedding performance. Similarly, the present invention proposes a quantum pure state attention network (QPSAN) to compare different calculation methods for quantum queries and keys. QPSAN uses pure state inner products to calculate the similarity between quantum queries and keys. Apart from this, the structures of QPSAN and QMSAN remain consistent. For ease of comparison, the present invention uses the same naming convention to represent QPSAN and QMSAN-D2π variants, such as QPSAN-ENN-P and QMSAN-D2π-ENN-P.

[0076] Assuming the quantum embedding circuit has n qubits and L layers, the AA quantum circuit has 3n((n-1) / 2+n)L parameters, while the ENN and CB quantum circuits have 6nL parameters. The quantum circuit depths for query, key, and value are the same. It is worth noting that the AA quantum circuit has more two-qubit gates than the ENN and CB quantum circuits, and the total number is greater, providing stronger entanglement capabilities.

[0077] All quantum circuit parameters Θ and the weights w of the classical part of the network are initialized from a normal distribution with a mean of 0 and a variance of 0.1. The bias b of the classical network is initialized to 0. For the value quantum circuit, the present invention measures the expected value under the observable quantity Z. For the attention matrix, the present invention measures the probability of the output state |0> under the observable quantity Z. Through these measurements, the present invention converts quantum data into classical data that can be used by subsequent classical networks. In the work of the present invention, the present invention uses the Tensorcircuit framework for quantum circuit simulation and the Tensorflow framework for parameter optimization, and the optimizer is Adam. The batch size is 64, and training is stopped when convergence is reached or after a fixed number of cycles. In the MC and RP tasks, the present invention repeats each experiment 9 times with different parameter initializations. For the sentiment labeling sentence dataset task, the present invention uses cross-validation for experiments.

[0078] In this experiment, two model architectures are used for numerical experiments: a model without position encoding and a model with position encoding, which can deeply analyze the performance differences between models and explore how position information enhances the model's understanding and processing capabilities of data. The experimental design aims to compare with representative models in classical networks and quantum networks. For the MC and RP datasets, the present invention compares the experimental results with the classical quantum model based on syntactic analysis and the QSANN model. At the same time, for three public sentiment analysis datasets (Yelp, IMDb, and Amazon), this model will be compared and analyzed with the classical self-attention neural network (CSANN) and the QSANN model. Through these comparative experiments, it is intended to comprehensively evaluate the performance and potential of quantum self-attention networks on different models and datasets. Table 2 Comparison of test accuracy for MC and RP tasks The above table compares the test accuracy of the method proposed in this invention and DisCoCat and CSANN for MC and RP tasks. Table 3 Comparison of test accuracy for Yelp, IMDb, and Amazon tasks The above table is a comparison of the test accuracy between the method proposed in the present invention and CSANN and QSANN for Yelp, IMDb and Amazon tasks.

[0079] For the MC dataset: The present invention first experiments on models without position information, and the results are detailed in Table 2. On the MC dataset, no matter which entanglement structure is used, the QMSAN-NP series models can perfectly distinguish sentences related to food and information technology, significantly outperforming the quantum models based on syntactic analysis mentioned in the existing literature. Since the tasks of the MC dataset are relatively simple, different entanglement structures can effectively capture the semantic information of the sentences in this case. It is worth noting that in terms of the number of parameters, the QMSAN-NP series models use fewer parameters than QSANN and the classic deep learning model DisCoCat, but achieve the same performance level, which shows the potential application value of the QMSAN-NP series models in resource-limited environments.

[0080] For the RP dataset: The present invention evaluates the performance of the model when processing more complex data. The experimental results are shown in Table 2. Different variants of the QMSAN-NP series models show different performances to varying degrees, among which the QMSAN-AA-NP model has the highest test accuracy of 77.42%. The experimental results of the other two entangled structures have declined, which may be because the fully connected structure provides richer information entanglement, which helps the model better capture the complex relationships in the sentence. The experimental results of the QMSAN-NP series models all exceed the 72.30% test accuracy of DisCoCat based on syntactic analysis, and are significantly better than the 67.74% of the QSANN model. Although the complexity of the RP dataset is significantly higher than that of the MC dataset, the present model can still effectively parse and infer the complex relationships in the dataset, further confirming the great potential of quantum self-attention networks in enhancing the model's understanding ability. In particular, QMSAN-ENN-NP and QMSAN-CB-NP improve performance while maintaining a low number of parameters and computational cost, indicating that the QMSAN model has potential in processing natural language processing tasks with a certain degree of complexity.

[0081] At the same time, the experimental results of QMSAN-NP and QPSAN-NP on this dataset were compared and analyzed. The QMSAN-NP series models performed better than the QPSAN-NP series models on the RP dataset, among which the QMSAN-AA-NP model had the highest test accuracy of 77.42%, which was 3.23% higher than the highest test accuracy of 74.19% of the QPSAN-NP series models. The specific results are shown in Table 2. This shows that the QMSAN model is able to capture the similarity characteristics between queries and keys more accurately through the hybrid state method, especially when dealing with complex semantic relationships.

[0082] For sentiment labeling sentence datasets: it covers more complex and diverse sentiment analysis tasks. The sentiment labeling sentence datasets include three subsets: Yelp, IMDb, and Amazon, covering different categories of reviews. Each category has its own unique language usage and sentiment expression, which increases the difficulty and complexity of sentiment analysis and provides a broader and more challenging test platform for the QMSAN-NP series models. The experimental results are detailed in Table 3.

[0083] Compared with the classic model CSANN, the methods of the QMSAN-NP series of models have achieved comprehensive improvements in accuracy, with the maximum improvement of QMSAN-ENN-NP on IMDb being 4.45%. Compared with the quantum model QSANN, the accuracy on most datasets has been significantly improved, with the maximum increase of 3.84% on the IMDb dataset and 2.47% on the Amazon dataset. The accuracy on the Yelp dataset is also similar, indicating that the QMSAN-NP series of models can more effectively capture the intrinsic characteristics and complex relationships of the data by using hybrid state quantum attention calculations and trainable embedded quantum modules, providing richer data representation for complex sentiment analysis tasks, thereby improving the representation ability of the model. It can effectively adapt to sentiment classification tasks in different fields and data distributions, not only significantly improving performance in most datasets, but also maintaining stable performance in different application scenarios, showing better generalization capabilities.

[0084] In terms of the number of parameters, QMSAN-ENN-NP and QMSAN-CB-NP have 29 parameters, which is significantly less than CSANN's 785 and QSANN's 49 parameters, but show significant performance improvements on most datasets. Being able to capture enough information with fewer parameters is very beneficial for reducing computing resources and improving efficiency. This shows that this model has a clear advantage in capturing sentiment features in text data. Among the three QMSAN-NP series models, QMSAN-AA-NP has slightly more parameters and stronger quantum entanglement capabilities, but its test accuracy on the Yelp dataset is only slightly higher than the other two models. This suggests that these more complex and different datasets may require different entanglement methods.

[0085] In order to further improve the model performance, the experimental part focuses on analyzing the impact of position information on the QMSAN model. Considering the importance of position information in text sequence processing, fixed quantum gates are used to encode position information. Since the average number of words per sentence in the MC and RP datasets is relatively small, the impact of position information on model performance may not be significant. Therefore, the experiment chooses to evaluate the performance of the QMSAN series models that introduce position information on the Yelp, IMDb, and Amazon data subsets to more accurately measure the impact of position information. The experimental results show that the QMSAN-P series models with fixed position information have improved performance across the board compared to models without position information. The detailed experimental results are shown in Table 3. The test accuracy on the Yelp dataset has increased by up to 0.71%, 1.08% on the IMDb dataset, and 0.92% on the Amazon dataset. In addition, Figure 2The data attention heat map is provided, in which the horizontal and vertical axes represent the vocabulary in the input sentence, and each word has a corresponding position in the horizontal and vertical axes. The colors presented in the heat map range from light to dark red, and the color intensity represents the quantum self-attention coefficient of one word to another word. For example, in the first sentence "The phone loads super!", phone and super are the main sources of information in the sentence, and the colors presented on their horizontal axes are darker than those of other words. The quantum self-attention coefficient is relatively high, which means that the importance of the two is relatively strong, proving the ability of this model to identify the important parts of the sentence, and intuitively demonstrating the effectiveness of the method of the present invention.

[0086] The above experimental results show that the method of the present invention can effectively encode position information in text sequences and comprehensively improve the accuracy of the QMSAN model in different sentiment analysis tasks. By using fixed quantum gates with position information, the model obtains important information about the relative positions of words in the text and enhances the understanding of the entire text structure. The advantage of this method is that it provides a fixed and efficient way for quantum models to integrate position information. We do not need to add additional quantum bit resources or trainable parameters, thus saving computing resources and time during training. In addition, since a stable and consistent way is used to represent the position information of words, this helps the generalization ability of the model.

[0087] At the same time, the experiments are compared with the experiments of the QMSAN-D2π-P series models, which scale the input data to [0,2π]. Figure 3 As shown in the figure, the test accuracy of the QMSAN-D2π-P series models is lower than that of the QMSAN-P series models, and even performs lower than the QMSAN-NP model without position information on some data sets. This also supports the above-mentioned scaling method for traditional data and position information encoding.

[0088] In the practical application of quantum computing, the influence of quantum noise is an important factor, because NISQ devices are sensitive to the environment and easily interfered by noise. In order to evaluate the robustness of the QMSAN-P series model in a quantum noise environment, a series of experiments were conducted using Tensorcircuit simulation software, which not only considered common noise channels such as depolarization noise, amplitude damping noise, and phase damping noise, but also explored the effects of different quantum circuit structures on anti-noise.

[0089] The depolarization channel (DP) depolarizes the qubit with probability p. For a single qubit, it is replaced by a fully mixed state I / 2 and remains unchanged with probability 1-p. The depolarization channel can be represented as the following density matrix mapping: Among them, ρ is the original density matrix, X, Y, Z are Pauli matrices.

[0090] Amplitude damping (AD) describes the process by which a quantum system loses energy, while phase damping (PD) describes the process by which a quantum system loses phase information without losing energy. The noise mapping of a single quantum bit density matrix can be uniformly expressed as: For amplitude damping, and For phase damping, and E0 and E1 represent the Kraus operator, and p represents the noise level.

[0091] In this experiment, these single-qubit noise channels are added to the embedded layer circuit for noise addition. When the noise level is 0.01, 0.1 and 0.2, the experimental results are shown in Table 4. The performance of the QMSAN-P series models has slightly decreased, where the maximum accuracy under the PD (0.1) noise model on the Yelp dataset has decreased to 1.54%; the maximum accuracy under the DP (0.2) noise model on the IMDb dataset has decreased to 1.48%; the maximum accuracy under the DP (0.2) noise model on the Amazon dataset has decreased to 1.38%. The test accuracy drop caused by noise did not exceed 1.6%, indicating that the QMSAN series models can maintain high performance stability in a low-level quantum noise environment, showing good robustness to common quantum noise, and verifying the feasibility of running the QMSAN model in a practical quantum computing environment. Table 4 Test accuracy of QMSAN-P series models under different noise models

[0092] The table above shows the test accuracy of the QMSAN-P series models on the Yelp, IMDb and Amazon datasets under different noise models. DP is the depolarization channel, AD is the amplitude damping, and PD is the phase damping.

[0093] This paper proposes a novel quantum multi-head self-attention network (QMSAN) model, which combines the characteristics of quantum computing with the advantages of classical self-attention networks to enhance the processing power and efficiency of NLP tasks. This model processes queries and keys in a mixed state through quantum gate operations, and directly generates similarity scalars through measurement. Compared with traditional pure state unitary transformations, the method of the present invention expands the expressive power of quantum systems through mixed state operations, enabling the model to capture the similarities between queries and keys more comprehensively and finely. At the same time, a trainable quantum embedding module is introduced to map classical data to quantum states, achieving more efficient data representation and processing. In addition, a novel quantum position encoding scheme is proposed, which improves the accuracy of encoding without increasing additional quantum bit resources by introducing additional fixed quantum gates in the quantum circuit to encode position information.

[0094] Experimental results on various datasets verify the effectiveness of QMSAN. Compared with the classic self-attention network, this model not only significantly reduces the number of parameters under the same input sequence conditions, but also shows superior performance, proving its stronger learning ability.

[0095] The present invention has the following advantages:

[0096] (1) Improve data processing efficiency: The present invention adopts a quantum attention mechanism based on mixed states to achieve efficient and direct estimation of the similarity between queries and keys in the quantum field, thereby obtaining a more effective attention coefficient and being able to capture richer information and intrinsic data associations. At the same time, the similarity between queries and keys is directly estimated at the quantum level, and there is no need to degenerate the quantum information of queries and keys into classical information for processing, thus maintaining the efficiency and parallelism of quantum computing, and ultimately achieving improved NLP task processing efficiency.

[0097] (2) Improve model accuracy: On the one hand, the present invention adopts a new quantum attention coefficient calculation mechanism based on mixed states to avoid the decrease in accuracy due to information loss. On the other hand, the present invention implements an innovative quantum position encoding scheme by introducing additional fixed quantum gates in the quantum circuit. The scheme adopts the absolute position encoding form to improve the encoding accuracy without adding additional quantum bit resources.

[0098] (3) Improve the overall performance of the model: This paper further optimizes the implementation framework of the quantum self-attention network by integrating the originally independent fixed quantum embedding and trainable quantum neural network structures into a unified trainable quantum embedding model. Not only does it significantly reduce the number of parameters, but it also demonstrates powerful data representation and information extraction capabilities, as well as excellent robustness in low-noise environments, significantly improving the overall performance of the quantum self-attention neural network model.

[0099] The above embodiments are only examples for clearly explaining the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications may be made based on the following description, and these various changes, modifications, substitutions and deformations derived from the principles and spirit of the present invention still fall within the scope of protection of the present invention.

Claims

1. A quantum self-attention neural network model, applied to natural language processing, characterized in that: include: Quantum hybrid state embedding module, which is used to convert classical data into quantum states through quantum feature mapping and variational quantum circuits, and obtain hybrid state queries and keys; A quantum self-attention module, configured to calculate the similarity between the mixed state query and the key by exchanging a test quantum circuit within the quantum state dimension generated by the quantum mixed state embedding module; A quantum position encoding module, which encodes position information by introducing additional fixed quantum gates in the quantum self-attention module without requiring additional quantum bit resources; The output module calculates the self-attention coefficient of the quantum state through the mixed state query and key exchange test circuit, multiplies it with the expected value of the observable quantity Z corresponding to the mixed state value, extracts the output encoding the semantic information of the input data, and constructs the quantum self-attention neural network model.

2. The model according to claim 1, characterized in that The quantum mixed state embedding module comprises: A data conversion unit is used to convert the classical data x into s Direct conversion to the quantum state |x s，q >、|x s，k > and |x s，v >, where 1≤s≤S, S is the number of input vectors in the data sample; A quantum state processing unit processes the quantum state through the trainable and adaptive variational quantum circuit to obtain a pure state query |x s，q > and key and |x s，k > vector; A partial trace operation unit performs a partial trace operation on the pure state query and the key vector to obtain a mixed state query ρ q and key σ k , expressed as: ρ s，q =tr B (|x s，q > <x s，q |),σ s，k =tr B (|x s，k > <x s，k |) Among them, tr B (·) represents the partial trace operation on system B, ρ s，q and σ s，k represent the quantum mixed state density operators for query and key respectively.

3. The model according to claim 2, characterized in that The variational quantum circuit comprises: Single-qubit gate R x (x i ), used to convert the input data x=(x1, ..., x N ) T Encoded into quantum circuits; Two-qubit gate For entangled qubits and adding R y (θ2); The trainable parameter θ is used to control the transformation of the quantum circuit, which is: Where θ is a trainable parameter and L is a layer; each layer consists of a data encoding circuit block V(x) and a trainable parameter θ l The trainable circuit block W(θ l )composition.

4. The model according to claim 1, characterized in that The quantum self-attention module includes: The exchange test quantum circuit unit is used to calculate the quantum self-attention coefficient between the query and the key quantum state through the exchange test, which is expressed as: a s，j =tr(ρ s，q s j，k ) Among them, ρ s，q and σ j，k are the mixed quantum states of query and key, respectively; A measurement unit to maintain the quantum information of the query and key at the quantum level; A similarity calculation unit is used to calculate the similarity between the query and the bond mixed quantum state based on the Hilbert-Schmidt distance, expressed as: D HS (ρ,σ)=tr((ρ-σ) 2 ) where ρ and σ represent the quantum states of the query and bond, respectively.

5. The model according to claim 4, characterized in that The measuring unit comprises: qubits, used to represent queries and keys; Auxiliary qubits, used to perform the swap test; Measuring equipment for measuring and calculating attention coefficient; The similarity calculation unit determines the quantum self-attention coefficient by normalization processing, which is expressed as: Among them, α s，j Represented as the unnormalized quantum self-attention coefficient between the sth and jth data, α s，m It is expressed as the unnormalized quantum self-attention coefficient between the sth and mth data, and S represents the total number of data in a sample.

6. The model according to claim 1, characterized in that The quantum position encoding module comprises: A position encoding unit for encoding position information into the quantum circuit; The quantum gate array unit is used to realize the encoding of the position information, including sinusoidal position encoding, which is expressed as: Among them, s represents the position, i represents the dimension, and d model Represents the dimension of the embedding.

7. The model according to claim 1, characterized in that: The model also includes: Loss function, used to train the model on the dataset by minimizing the loss: Where Θ represents all trainable parameters in the quantum circuit, W and b represent the weight and bias of the fully connected layer, respectively. is the data set, N S represents the total number of samples, represents the predicted probability value, Indicates the actual label.

8. A method for constructing a self-attention neural network model, applied to natural language processing, characterized in that: include: Convert classical data into quantum states through quantum feature mapping and variational quantum circuits, and obtain mixed-state queries and keys; In the quantum state dimension generated by the quantum mixed state embedding module, the similarity between the mixed state query and the key is calculated by exchanging the test quantum circuit; By introducing additional fixed quantum gates in the quantum self-attention module to encode position information, no additional quantum bit resources are required; According to the similarity between the mixed state query and the key, the output encoding the semantic information of the input data is extracted through the measurement and post-processing of the quantum state, and the quantum self-attention neural network model is constructed.

9. The method according to claim 8, characterized in that: Convert classical data into quantum states through quantum feature mapping and variational quantum circuits, and obtain mixed state queries and keys including: The classical data x is transformed into s Direct conversion to the quantum state |x s，q >、|x s，k > and |x s，v >, where 1≤s≤S, S is the number of input vectors in the data sample; The quantum state is processed by the trainable and adaptive variational quantum circuit to obtain a pure state query |x s，q > and key and |x s，k > vector; Perform partial trace operations on the pure state query and key vector to obtain the mixed state query ρ q and key σ k , expressed as: ρ s，q =tr B (|x s，q > <x s，q |),σ s，k =tr B (|x s，k > <x s，k |) Among them, tr B (·) represents the partial trace operation on system B.

Images