A text recognition method based on quantum transfer learning

Abstract

The application provides a text classification method based on quantum transfer learning, and belongs to the field of quantum computing and artificial intelligence fusion. Considering the complex feature space and nonlinear characteristics in text classification, the quantum circuit design, self-attention mechanism and quantum transfer learning process are optimized to improve the learning efficiency and classification accuracy of the model. The high-dimensional feature representation and complex nonlinear decision process are converted into a quantum self-attention network structure, an optimal strategy for text classification is obtained by using a complex quantum kernel self-attention network (CQKSAN) model, and a quantum superposition and entanglement mechanism is introduced to enhance the expression ability of the model. Simulation results show that the proposed algorithm has higher classification accuracy and learning efficiency compared with traditional deep learning algorithms and other hybrid quantum-classical algorithms. The method not only shows the potential of hybrid quantum-classical methods, but also provides a foundation for further research on future all-quantum learning systems.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of quantum transfer learning and artificial intelligence, and specifically relates to a text classification method based on quantum transfer learning, and more specifically relates to a quantum transfer learning model combined with a complex quantum kernel self-attention network (CQKSAN) for improving text classification performance. BACKGROUND

[0002] In recent years, quantum computing, as a revolutionary technology, has shown great potential in various fields through the superposition and entanglement of qubits. It is particularly useful in solving high-dimensional and complex data problems that are difficult for classical computers to handle. Due to this characteristic, quantum computers are expected to bring revolutionary changes to the field of artificial intelligence (AI) in the future.

[0003] Machine learning in artificial intelligence has made significant progress through classical methods, especially in natural language processing (NLP) and other fields. Classical deep learning models such as BERT and Transformer have performed well in tasks such as text classification, machine translation, and named entity recognition. However, the limitations of these classical methods lie in their low efficiency in handling high-dimensional complex data and the need for large amounts of computational resources. To overcome these problems, the combination of quantum transfer learning and machine learning has become a promising solution.

[0004] In the current era of noisy intermediate-scale quantum (NISQ) devices, the number of qubits and error rates still limit the application of pure quantum transfer learning. Therefore, hybrid classical-quantum models have become an effective solution. By using classical neural networks for feature extraction and combining quantum circuits for classification, these models can optimize performance under the limitations of current quantum hardware.

[0005] Transfer learning, as an effective method to improve the efficiency and accuracy of machine learning, has been widely applied in NLP tasks, especially in handling low-resource languages and cross-language adaptation problems. Quantum transfer learning is the integration of quantum transfer learning and transfer learning, aiming to leverage the advantages of quantum processing and pattern recognition to optimize solutions for specific tasks.

[0006] In existing research, classical-to-quantum transfer learning methods combine classical feature extractors with quantum classifiers to demonstrate performance improvements in various fields, especially in handling small data sets and high-dimensional complex tasks. Quantum self-attention networks (QSANN), quantum kernel self-attention mechanisms (QKSAM), and quantum mixed-state self-attention networks (QMSAN) have been proposed and applied in the NLP field, demonstrating the potential advantages of quantum transfer learning in text classification and language modeling.

[0007] The application provides a complex quantum kernel self-attention network (CQKSAN) model based on quantum transfer learning, which combines quantum circuits and self-attention mechanisms to improve the efficiency and accuracy of text classification. The CQKSAN model takes advantage of pre-trained classical models and enhances them through quantum transfer learning. In experiments, the CQKSAN model outperforms traditional deep learning models in terms of learning efficiency and accuracy, especially on small sample datasets, with faster convergence speed and higher model stability. SUMMARY

[0008] The application aims to provide a complex quantum kernel self-attention network (CQKSAN) model based on quantum transfer learning to address the inefficiency and inaccuracy of existing text classification tasks.

[0009] To achieve the above-mentioned purpose, the application provides the following solutions:

[0010] The application provides a text classification method based on a complex quantum kernel self-attention network (CQKSAN) model using quantum transfer learning, which includes the following steps:

[0011] S1: Construct a pre-trained classical neural network (such as BERT) as a feature extractor to extract features from input text data using the pre-trained model.

[0012] S2: Process the extracted features through a quantum circuit, which optimizes the feature representation capability based on quantum superposition and entanglement characteristics. Use angle encoding to map classical features to quantum states.

[0013] S3: Apply a self-attention mechanism in the quantum circuit to convert input features into high-dimensional quantum states, calculate queries, keys, and values, and execute the self-attention mechanism through the quantum circuit to obtain better feature weight distribution.

[0014] S4: Use the complex quantum kernel method (CQKSAN) model to calculate the similarity between different input samples through quantum feature mapping and kernel function, and complete text classification.

[0015] S5: Use the quantum transfer learning framework to combine the advantages of classical neural networks and quantum circuits to improve the learning efficiency and accuracy of the model.

[0016] Further, in S1, a pre-trained classical neural network (such as BERT) is constructed as a feature extractor to extract features from input text data using the pre-trained model, specifically:

[0017] The pre-trained model is a BERT model that performs context-aware feature extraction on text data through a bidirectional encoder and is fine-tuned in downstream tasks to adapt to specific text classification tasks. The pre-trained BERT model provides powerful context-aware sentence representations that can be fine-tuned for various downstream NLP tasks. Fine-tuning requires adjusting multiple hyperparameters, the values of which directly affect the performance of the model. The BERTbase model consists of 12 layers of hidden layers, each with 768 hidden units and 12 attention heads, totaling 110 million parameters. The BERTbase model accepts input sequences of up to 512 tokens, each represented by a specific embedding. These embeddings are used for various NLP tasks, including syntax analysis. The tokens in the input sequence can be case-sensitive or case-insensitive, resulting in two different pre-trained models. The bidirectionality of BERT enables it to internally encode rich contextual information and adapt to specific tasks during the fine-tuning phase, making it a foundational model in recent years.

[0018] Before inputting the token sequence into BERT, a special [CLS] token is added at the beginning of the sequence to obtain a vector representation of the entire sequence. Another special token [SEP] is placed at the end of each sentence. Given the input sequence t = (t1, t2, …, t m ), the output of BERT is h = (h0, h1, h2, …, h m ), where is the final hidden state of the [CLS] token, providing a summary representation of the entire input sequence, and h1, h2, …, h m are the final hidden states of the other tokens. To classify the input token sequence into K different text categories, the final hidden state h0 is input into a classification layer, followed by a softmax operation to convert the scores of each text category into probabilities: P = softmax(CW T ). Where is the parameter matrix of the classification layer. In this way, BERT can effectively perform various NLP tasks.

[0019] Further, in S2, the extracted features are processed through a quantum circuit, which optimizes the feature representation capability based on the quantum superposition and entanglement characteristics. The classical features are mapped to quantum states using angle encoding, specifically:

[0020] The quantum circuit performs angle encoding through a rotation gate, which is a common way to map classical information to quantum states. The classical feature h i is mapped to a quantum state through a rotation gate R y (θ): where θ ih i is determined. In this way, the classical vectors extracted by BERT are embedded into quantum states. The multiple qubits are entangled through quantum entanglement gates such as CNOT gates, so as to enhance the feature expression capability, and more efficient coding of data is achieved by using the quantum superposition and entanglement characteristics.

[0021] First, quantum processing is performed: the BERT model extracts features from the input text, and then processes them through a classical neural network layer to obtain the real part of the quantum circuit input. The real part of the input is encoded into a quantum state using ansatz. The quantum circuit subsequently performs the following operations: applying a rotation gate (RY) to encode the real part into a quantum state. Different quantum circuits are applied according to the ansatz type: 1. HEAnsatz: applying a Hadamard gate to initialize the quantum bits to a superposition state. Applying a rotation gate (RZ) to each quantum bit based on the parameters. Applying a controlled rotation gate (CRY) between adjacent quantum bits. 2. QAOAAnsatz: applying a Hadamard gate to initialize the quantum bits to a superposition state. Applying a rotation gate (RY) to each quantum bit. Applying a controlled non gate (CNOT) gate between adjacent quantum bits, followed by a rotation gate (RZ) and another set of CNOT gates.

[0022] Further, in S3, a self-attention mechanism is applied in the quantum circuit to convert the input features into a high-dimensional quantum state, calculate the query, key, and value, and perform the self-attention mechanism through the quantum circuit to obtain better feature weight distribution, specifically:

[0023] Calculation of query, key, and value. In the classical self-attention mechanism, the attention distribution of query, key, and value is calculated by the following formula In the quantum version, the quantum states of query, key, and value are calculated by the quantum circuit, and each input quantum state is represented as ψ i = α|0> + β|1>. The quantum inner product of query and key is used to calculate the attention distribution: The attention value is calculated by the quantum circuit, and combined with the classical attention mechanism to improve the accuracy and efficiency of the self-attention mechanism.

[0024] Further, in S4, the complex quantum kernel method (CQKSAN) model is used to calculate the similarity between different input samples through quantum feature mapping and kernel function, and the text classification is completed, specifically:

[0025] The quantum kernel function calculates the inner product of the two quantum states after quantum feature mapping: κ(ψ(x i ), ψ(x j )) = |<ψ(x i )|ψ(x j)| 2 The inner product is used to measure the similarity between samples, and after feature mapping by the quantum circuit, the quantum kernel function can better capture the complex relationship between high-dimensional data. CQKSAN realizes higher classification accuracy in the text classification task through the optimized quantum kernel function.

[0026] Further, in S5, through the quantum transfer learning framework, the advantages of classical neural networks and quantum circuits are combined to improve the learning efficiency and accuracy of the model, specifically:

[0027] The feature extraction ability of the classical pre-trained model (such as BERT) and the efficient processing ability of the quantum circuit are combined to optimize the training process and convergence speed of the model. The output value of the quantum state is obtained by measuring the final generated quantum state. The classical loss function is calculated by cross-entropy: Where y i is the label, and p i is the probability predicted by the model. In this framework, the classical model is responsible for preliminary feature extraction, while the quantum circuit encodes and enhances the features of the quantum state, ultimately improving the accuracy of text classification. Quantum transfer learning accelerates the convergence of quantum models by pre-training classical models and improves classification results using quantum transfer learning.

[0028] The present application has the following beneficial technical effects relative to the prior art:

[0029] 1. The text classification method based on quantum transfer learning proposed in the present application combines the efficiency of quantum transfer learning and the feature extraction ability of classical pre-trained models, greatly improving the efficiency of text classification.

[0030] 2. Through the complex quantum kernel method, the model of the present application can process high-dimensional feature representation, significantly improving the accuracy of text classification and the robustness of the model. It shows superior learning ability when processing small-scale data sets, significantly shortens the convergence time of the model, and has higher generalization ability, suitable for various natural language processing tasks. BRIEF DESCRIPTION OF DRAWINGS

[0031] Figure 1 The flowchart of the text recognition method algorithm based on quantum transfer learning provided for the embodiments of the present application;

[0032] Figure 2 The quantum circuit framework diagram of the text recognition based on quantum transfer learning provided for the embodiments of the present application;

[0033] Figure 3 The complex quantum kernel self-attention network (CQKSAN) model framework diagram of the text recognition based on quantum transfer learning provided for the embodiments of the present application;

[0034] Figure 4 A simulation result graph of a complex quantum kernel self-attention network (CQKSAN) for text recognition based on quantum transfer learning is provided for an embodiment of the present application. DETAILED DESCRIPTION

[0035] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0036] The purpose of the present application is to provide a complex quantum kernel self-attention network (CQKSAN) model based on quantum transfer learning, which is used to solve the problems of insufficient efficiency and accuracy in existing text classification tasks.

[0037] In order for those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0038] Figure 1 A flowchart of a text recognition method algorithm based on quantum transfer learning provided for an embodiment of the present application is shown in FIG. 1, which includes the following steps: Figure 1

[0039] S1: A pre-trained classical neural network (such as BERT) is constructed as a feature extractor, and the input text data is extracted by the pre-trained model:

[0040] The pre-trained model is a BERT model, which extracts features with context awareness through a bidirectional encoder for text data and fine-tunes in downstream tasks to adapt to specific text classification tasks. The pre-trained BERT model provides powerful context-aware sentence representation, which can be fine-tuned for various downstream NLP tasks. Fine-tuning requires adjusting multiple hyperparameters, the values of which directly affect the performance of the model. The BERTbase model consists of 12 layers of hidden layers, each layer having 768 hidden units and 12 attention heads, totaling 110 million parameters. The BERTbase model accepts input sequences of up to 512 tokens, each token represented by a specific embedding. These embeddings are used for various NLP tasks, including syntax analysis. The tokens in the input sequence can be case-sensitive or case-insensitive, resulting in two different pre-trained models. The bidirectionality of BERT enables it to internally encode rich contextual information and adapt to specific tasks during the fine-tuning phase, making it a foundation model in recent years.

[0041] ​Before the token sequence is input into BERT, a special [CLS] token is added in front of the sequence to obtain the vector representation of the entire sequence. Another special token [SEP] is placed at the end of each sentence. Given the input sequence t = (t1, t2, …, t m ), the output of BERT is h = (h0, h1, h2, …, h m ), where is the final hidden state of the [CLS] token, providing a summary representation of the entire input sequence, and h1, h2, …, h m are the final hidden states of other tokens. To classify the input token sequence into K different text categories, the final hidden state h0 is input into a classification layer, followed by a softmax operation to convert the score of each text category into a probability: P = softmax(W T ). Where is the parameter matrix of the classification layer. In this way, BERT can effectively perform various NLP tasks.

[0042] S2: The extracted features are processed through a quantum circuit, which optimizes the feature representation capability based on the superposition and entanglement characteristics of quantum. The classical features are mapped to quantum states using angle encoding:

[0043] The quantum circuit performs angle encoding through the rotation gate of the quantum bit, which is a common way to map classical information to quantum states. The classical feature h i is mapped to a quantum state through the rotation gate R y (θ): where θ i is determined by the classical feature h i . In this way, the classical vector extracted by BERT is embedded into a quantum state. Multiple quantum bits are entangled through quantum entanglement gates (such as CNOT gates), thereby enhancing the feature expression capability and using the superposition and entanglement characteristics of quantum to encode data more effectively.

[0044] Specifically, as Figure 2As shown, in this embodiment, first, the quantum processing: the BERT model extracts features from the input text, and then processes them through the classical neural network layer to obtain the real part of the quantum circuit input. The real part of the input is encoded into a quantum state using ansatz. The quantum circuit then performs the following operations: apply the rotation gate (RY) to encode the real part into a quantum state. According to the type of ansatz, different quantum circuits HEAnsatz are applied: apply the Hadamard gate to initialize the quantum bits to the superposition state. Apply the rotation gate (RZ) to each quantum bit based on the parameters. Apply the controlled rotation gate (CRY) between adjacent quantum bits. QAOAAnsatz: apply the Hadamard gate to initialize the quantum bits to the superposition state. Apply the rotation gate (RY) to each quantum bit. Apply the controlled non gate (CNOT) gate between adjacent quantum bits, followed by the rotation gate (RZ) and another set of CNOT gates.

[0045] S3: Apply the self-attention mechanism in the quantum circuit to convert the input features into a high-dimensional quantum state, calculate the query, key and value, and perform the self-attention mechanism through the quantum circuit to obtain better feature weight distribution.

[0046] Calculation of query (Query), key (Key) and value (Value). In the classical self-attention mechanism, the attention distribution of query, key and value is calculated by the following formula In the quantum version, the quantum state of query, key and value is calculated by the quantum circuit, and each input quantum state is represented as ψ i = α|0> + β|1>. The quantum inner product of query and key is used to calculate the attention distribution: The attention value is calculated by the quantum circuit, and combined with the classical attention mechanism to improve the accuracy and efficiency of the self-attention mechanism.

[0047] S4: Use the complex quantum kernel method (CQKSAN) model to calculate the similarity between different input samples through quantum feature mapping and kernel function, and complete text classification:

[0048] Specifically, as Figure 3 As shown in the model structure diagram, in this embodiment, the quantum kernel function is calculated by the inner product of the two quantum states after quantum feature mapping: κ(ψ(x i ), ψ(x j )) = |<ψ(x i )| ψ(x j )>| 2 . This inner product is used to measure the similarity between samples. After feature mapping by the quantum circuit, the quantum kernel function can better capture the complex relationship between high-dimensional data. CQKSAN realizes higher classification accuracy in the text classification task through the optimized quantum kernel function.

[0049] S5: Through the quantum transfer learning framework, the advantages of classical neural networks and quantum circuits are combined to improve the learning efficiency and accuracy of the model:

[0050] The final generated quantum state is measured to obtain the output value of the quantum state. The classical loss function is calculated by cross-entropy: Where y i is the label, and p i is the probability predicted by the model. In this framework, the classical model is responsible for preliminary feature extraction, while the quantum circuit encodes and enhances the features of these features in quantum state, ultimately improving the accuracy of text classification. Quantum transfer learning accelerates the convergence of quantum models by pre-training classical models, and improves the classification effect by using quantum transfer learning.

[0051] Specifically, as Figure 4 The simulation results of text recognition under different algorithms are described, and the complex quantum kernel self-attention network of the text recognition based on quantum transfer learning can obtain the best recognition performance. When processing the same data set, the accuracy reaches a peak training accuracy of 97.97%, which is 1.12% better than the BERT model.

[0052] The above describes in detail the text recognition method based on quantum transfer learning provided by the present application. Each embodiment in the specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts of each embodiment can be referred to.

[0053] The contents not described in detail in the specification of the present application belong to the existing technology known to those skilled in the art.

Claims

1.A method for text recognition based on quantum transfer learning, characterized in that, The method comprises the following steps: S1: constructing a pre-trained classical neural network as a feature extractor to extract features of input text data through the pre-trained model; S2: processing the extracted features through a quantum circuit, which optimizes the feature representation capability based on the quantum superposition and entanglement characteristics, and uses angle encoding to map the classical features to quantum states, specifically including: first, processing by the quantum circuit: the BERT model extracts features from the input text, and then processes through a classical neural network layer to obtain the real part of the input of the quantum circuit; the real part of the input is encoded into a quantum state using ansatz, and the quantum circuit subsequently performs the following operations: applying a rotation gate RY to encode the real part into a quantum state; according to the ansatz type, different quantum circuits are applied: (1) HEAnsatz: applying a Hadamard gate to initialize the quantum bits to a superposition state, applying a rotation gate RZ to each quantum bit based on parameters, and applying a controlled rotation gate CRY between adjacent quantum bits; (2) QAOAAnsatz: applying a Hadamard gate to initialize the quantum bits to a superposition state, applying a rotation gate RY to each quantum bit, and applying a controlled non gate CNOT gate between adjacent quantum bits, followed by a rotation gate RZ and another set of CNOT gates; S3: applying a self-attention mechanism in the quantum circuit to convert the input features into high-dimensional quantum states, calculating the query, key and value, and performing the self-attention mechanism through the quantum circuit to obtain better feature weight distribution; S4: using the complex quantum kernel method model, the similarity between different input samples is calculated through quantum feature mapping and kernel function, and the text classification is completed; the quantum kernel function is calculated by the inner product of two quantum states after quantum feature mapping: κ(ψ(x i ),ψ(x j ))=|<ψ(x i )|ψ(x j )>| 2 , the inner product is used to measure the similarity between samples, and the quantum kernel function can better capture the complex relationship between high-dimensional data after feature mapping of the quantum circuit; CQKSAN realizes higher classification accuracy in the text classification task through the optimized quantum kernel function; S5: Through the quantum transfer learning framework, combine the advantages of classical neural networks and quantum circuits to improve the learning efficiency and accuracy of the model, including: measuring the final generated quantum state to get the output value of the quantum state; the classical loss function is calculated by cross entropy: In this framework, the classical model is responsible for preliminary feature extraction, while the quantum circuit encodes and enhances the features in the quantum state, ultimately improving the accuracy of text classification; quantum transfer learning accelerates the convergence of quantum models through pre-trained classical models and improves classification results using quantum computing. 2.The quantum transfer learning based text recognition method of claim 1, wherein: In S1, the pre-trained classical neural network as a feature extractor is specifically constructed as: The pre-trained model is a BERT model, which performs context-aware feature extraction on text data through a bidirectional encoder, and is fine-tuned in downstream tasks to adapt to specific text classification tasks; the pre-trained BERT model provides powerful context-aware sentence representation, which can be fine-tuned for various downstream NLP tasks; Fine-tuning requires adjusting multiple hyperparameters, the values of which directly affect the performance of the model; the BERTbase model consists of 12 layers of hidden layers, each with 768 hidden units and 12 attention heads, totaling 110 million parameters; the BERTbase model accepts input sequences of up to 512 tokens, each token represented by a specific embedding; these embeddings are used for various NLP tasks, including syntax analysis; the tokens in the input sequence can be case-sensitive or case-insensitive, resulting in two different pre-trained models; the bidirectionality of BERT enables it to internally encode rich contextual information and adapt to specific tasks during the fine-tuning phase; Before inputting the token sequence into BERT, a special [CLS] token is added in front of the sequence to obtain the vector representation of the whole sequence; another special token [SEP] is placed at the end of each sentence; given the input sequence t = (t1, t2, …, t m ), the output of BERT is h = (h0, h1, h2, …, h m ), where h0 is the final hidden state of the [CIS] token, providing a summary representation of the entire input sequence, and h1, h2, …, h m are the final hidden states of other tokens; in order to classify the input token sequence into K different text categories, the final hidden state h0 is input into the classification layer, followed by the softmax operation to convert the score of each text category into a probability: P = softmax(CW T ); where is the parameter matrix of the classification layer. 3.The quantum transfer learning based text recognition method of claim 1, wherein: In S2, the feature data output by the classical BERT is converted into a quantum state, specifically including: Quantum circuits are angle encoded by rotation gates of qubits; classical features h i Through the rotation gate R y (θ) is mapped to a quantum state: Where, θ i Is determined by the classical feature h i In this way, the classical vector extracted by BERT is embedded into a quantum state; the entanglement between multiple qubits is realized through quantum entanglement gates, and the data is more effectively encoded by using the superposition and entanglement characteristics of quantum, thereby enhancing the feature expression ability. 4.The quantum transfer learning based text recognition method of claim 1, wherein: In S3, the input features are converted into high-dimensional quantum states, the query, key and value are calculated, and the self-attention mechanism is executed through the quantum circuit to obtain better feature weight distribution, specifically including: First, the calculation of query, key and value; in the classical self-attention mechanism, the attention distribution of query, key and value is calculated by the following formula In the quantum version, the quantum states of query, key and value are calculated by quantum circuits, and each input quantum state is represented as ψi=α|0>+β|1>; the quantum inner product of query and key is used to calculate the attention distribution: The attention value is calculated by the quantum circuit, and combined with the classical attention mechanism, the accuracy and efficiency of the self-attention mechanism are improved.

Images