Text classification method and device, electronic equipment and computer readable storage medium

Abstract

Embodiments of the present application provide a text classification method and device, electronic equipment and computer readable storage medium, relating to the field of natural language processing. Since the input matrix is obtained by adding the text embedding matrix corresponding to the text to be classified and the quantum position encoding matrix corresponding to the text to be classified, and the first feature extraction network and the second feature extraction network used for extracting features both include variational quantum circuits, the third feature matrix input into the classifier actually contains quantum position encoding information corresponding to the text to be classified, embedding encoding information of the word itself, and more dimensional semantic information such as deeper semantic information obtained by entanglement between different words through the variational quantum circuit, so as to effectively alleviate the problems of semantic information loss and redundancy caused by limiting the fixed dimension of word embedding encoding in the prior art, thereby achieving better text classification effect.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of natural language processing, and in particular, to a text classification method and device, electronic equipment and a computer readable storage medium. BACKGROUND

[0002] Text classification is to classify text according to different categories (such as sports, entertainment, etc.). The common method of existing text classification is to convert text sequence information into a word embedding matrix through word embedding, then input the word embedding matrix into a Transformer to learn sentence semantic features, extract deep semantic features of the sentence, and finally output classification labels through a full connection.

[0003] The traditional text classification based on the Transformer uses its Encoder module, and the limitation of fixed dimension of word embedding coding will cause semantic information loss and redundancy, resulting in poor final classification effect. SUMMARY

[0004] Therefore, the purpose of the present application is to provide a text classification method, device, electronic equipment and computer readable storage medium to solve the problem of semantic information loss and redundancy caused by the limitation of fixed dimension of word embedding coding in the prior art, resulting in poor final classification effect.

[0005] To achieve the above purpose, the technical solutions adopted by the embodiments of the present application are as follows:

[0006] In a first aspect, the present application provides a text classification method, which comprises:

[0007] inputting a text to be classified into a pre-trained text classification network, processing the text to be classified by using a word vector layer in the text classification network, and obtaining an input matrix corresponding to the text to be classified; the input matrix is obtained by adding a text embedding matrix corresponding to the text to be classified and a quantum position encoding matrix corresponding to the text to be classified;

[0008] using a first feature extraction network in the text classification network to extract features of the input matrix, and obtaining a plurality of first feature matrices;

[0009] using a multi-head attention layer in the text classification network to perform attention calculation on the plurality of first feature matrices, and obtaining a second feature matrix;

[0010] inputting the input matrix and the second feature matrix into a second feature extraction network in the text classification network to extract features, and obtaining a third feature matrix; the first feature extraction network and the second feature extraction network both include a variational quantum circuit.

[0011] inputting the third feature matrix into a classifier in the text classification network to obtain a category corresponding to the text to be classified.

[0012] In an optional implementation, the word vector layer comprises a word embedding layer and a quantum position encoding layer; and the processing of the text to be classified by using the word vector layer in the text classification network to obtain the input matrix corresponding to the text to be classified comprises:

[0013] performing word embedding encoding on the text to be classified by using the word embedding layer to obtain a text embedding matrix;

[0014] performing quantum position encoding on the text to be classified by using the quantum position encoding layer to obtain a quantum position encoding matrix;

[0015] adding the text embedding matrix and the quantum position encoding matrix to obtain the input matrix corresponding to the text to be classified.

[0016] In an optional implementation, the first feature extraction network comprises a plurality of variational quantum circuits, and the plurality of variational quantum circuits each correspond to different parameters; and the feature extraction of the input matrix by using the first feature extraction network in the text classification network to obtain a plurality of first feature matrices comprises:

[0017] inputting the input matrix into each variational quantum circuit in the first feature extraction network to perform feature extraction to obtain a first feature matrix output by each variational quantum circuit.

[0018] In an optional implementation, the second feature extraction network comprises a first data processing layer, a variational quantum circuit and a second data processing layer; and the feature extraction of the input matrix and the second feature matrix by using the second feature extraction network in the text classification network to obtain a third feature matrix comprises:

[0019] performing addition and normalization processing on the input matrix and the second feature matrix by using the first data processing layer to obtain a first intermediate matrix;

[0020] performing feature extraction on the first intermediate matrix by using the variational quantum circuit to obtain a second intermediate feature matrix;

[0021] performing addition and normalization processing on the first intermediate matrix and the second intermediate feature matrix by using the second data processing layer to obtain a third feature matrix.

[0022] In an optional implementation, the variational quantum circuit comprises a data encoding layer, a variational layer and a quantum measurement layer, and the parameter optimization step of the variational quantum circuit comprises:

[0023] inputting the training sample into a pre-constructed variational quantum circuit, encoding the training sample into quantum state feature data through the data encoding layer;

[0024] inputting the quantum state feature data into the variational layer, performing unitary operation on the quantum state feature data through the variational layer to obtain multi-qubit entanglement data;

[0025] inputting the multi-qubit entanglement data into the quantum measurement layer, measuring the expected value on each qubit through the quantum measurement layer to obtain prediction information;

[0026] updating the parameters of the variational quantum circuit according to the prediction information.

[0027] In a second aspect, the present application provides a text classification device, the device comprising:

[0028] a word vector processing module, configured to input a text to be classified into a pre-trained text classification network, and process the text to be classified by using a word vector layer in the text classification network to obtain an input matrix corresponding to the text to be classified; the input matrix is obtained by adding a text embedding matrix corresponding to the text to be classified and a quantum position encoding matrix corresponding to the text to be classified;

[0029] a first feature extraction module, configured to extract features of the input matrix by using a first feature extraction network in the text classification network to obtain a plurality of first feature matrices;

[0030] an attention calculation module, configured to calculate attention of the plurality of first feature matrices by using a multi-head attention layer in the text classification network to obtain a second feature matrix;

[0031] a second feature extraction module, configured to input the input matrix and the second feature matrix into a second feature extraction network in the text classification network to extract features to obtain a third feature matrix; the first feature extraction network and the second feature extraction network both comprise a variational quantum circuit;

[0032] a classification module, configured to input the third feature matrix into a classifier in the text classification network to obtain a category corresponding to the text to be classified.

[0033] In an optional implementation, the word vector layer comprises a word embedding layer and a quantum position encoding layer; the word vector processing module is configured to perform word embedding coding on the text to be classified by using the word embedding layer to obtain a text embedding matrix; perform quantum position coding on the text to be classified by using the quantum position encoding layer to obtain a quantum position encoding matrix; and add the text embedding matrix and the quantum position encoding matrix to obtain the input matrix corresponding to the text to be classified.

[0034] In an optional implementation, the second feature extraction network comprises a first data processing layer, a variational quantum circuit, and a second data processing layer; the second feature extraction module is configured to add and normalize the input matrix and the second feature matrix by using the first data processing layer to obtain a first intermediate matrix; perform feature extraction on the first intermediate matrix by using the variational quantum circuit to obtain a second intermediate feature matrix; and add and normalize the first intermediate matrix and the second intermediate feature matrix by using the second data processing layer to obtain a third feature matrix.

[0035] In a third aspect, the present application provides an electronic device comprising a processor, a memory, and a computer program stored on the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the text classification method according to any one of the preceding embodiments.

[0036] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program, when executed by a processor, implements the steps of the text classification method according to any one of the preceding embodiments.

[0037] The text classification method, device, electronic device, and computer-readable storage medium provided by the embodiments of the present application can effectively alleviate the problems of semantic information loss and redundancy caused by limiting the fixed dimension of word embedding coding in the prior art, thereby achieving better text classification effect. In addition, after the multi-head attention layer outputs the second feature matrix, the initial input matrix and the second feature matrix are input into the second feature extraction network again for feature extraction, which can further reduce the loss of semantic information, thereby improving the robustness of the text classification network.

[0038] In order to make the above objectives, characteristics and advantages of the present application more apparent, the following preferred embodiments are specifically described below with reference to the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0039] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments, and it should be understood that the following drawings only show some embodiments of the present application, and should not be regarded as a limitation to the scope. For those skilled in the art, other related drawings can also be obtained without creative labor.

[0040] Figure 1 A structural schematic diagram of a conventional Transformer is shown;

[0041] Figure 2 A block schematic diagram of an electronic device provided by an embodiment of the present application is shown;

[0042] Figure 3 A flow schematic diagram of a text classification method provided by an embodiment of the present application is shown;

[0043] Figure 4 A structural schematic diagram of a text classification network is shown;

[0044] Figure 5 Another structural schematic diagram of a text classification network is shown;

[0045] Figure 6 A structural schematic diagram of a variational quantum circuit is shown;

[0046] Figure 7 Another structural schematic diagram of a variational quantum circuit is shown;

[0047] Figure 8 A functional module diagram of a text classification device provided by an embodiment of the present application is shown.

[0048] Icon: 100-electronic device; 110-memory; 120-processor; 130-communication module; 800-text classification device; 810-word vector processing module; 820-first feature extraction module; 830-attention calculation module; 840-second feature extraction module; 850-classification module. DETAILED DESCRIPTION

[0049] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. The components of the embodiments of the present application described and shown in the drawings can be arranged and designed in various different configurations.

[0050] Therefore, the following detailed description of the embodiments of the application provided in the drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of the application. Based upon this description of the embodiments of the application, all other embodiments obtained by those of ordinary skill in the art without departing from the spirit of the application are intended to be within the scope of the application.

[0051] It should be noted that the relational terms herein such as first, second, and the like are used solely to distinguish one from another entity or action, without necessarily requiring or implying any actual relationship or order between such entities or actions. Moreover, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without more limitations, an element preceded by "comprises... a" does not, without more limitations, foreclose the existence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0052] The structure of the conventional Transformer can refer to Figure 1 The Transformer follows the architecture of Encoder-Decoder, the Encoder is composed of N x = 6 identical layers (i.e. the units on the left side in Figure 1 Each layer is composed of two sub-layers, multi-head self-attention and fully connected feed-forward network, and each sub-layer is added with residual connection and normalisation. The Decoder is also composed of N x = 6 identical layers (i.e. the units on the right side in Figure 1 In addition to the self-attention layer and the fully connected layer, there is also a Masked self-attention layer, and the use of mask on the multi-head attention mechanism is mainly to ensure that the prediction result of the current position only depends on the previous output; the Decoder is generally used for seq2seq models, such as machine translation and other modules.

[0053] The multi-head attention mechanism in the Transformer can be represented as: MultiHead(Q, K, V) = Concat(head1, head2,..., head h)W o , head i =Attention(QW i Q ,KW i K ,VW i V ),

[0054] Calculating self-attention first creates three vectors from the input vector, namely the query vector query (Q), the key vector key (K) and the value vector value (V). These vectors can be generated by multiplying the word embedding by the training matrix created during the training process; head i Indicates the attention calculation method of each head. W is three different weight parameter matrices. Since there are h heads, there are h dimensions, and each dimension is a vector. W i Q Refers to the i-th dimension of the W matrix, which is a vector. The result of the multi-head attention calculation can be obtained by concatenating the h-time attention calculation results.

[0055] In the encoder's multi-head self-attention, Q, K, and V all come from the same place (equal), and they are the output of the previous layer of encoder; for the first layer of encoder, they are the input obtained by adding word embedding and positional encoding.

[0056] In the decoder's multi-head self-attention, Q, K, and V all come from the same source (equally), representing the output of the previous decoder layer. For the first-layer decoder, these are simply the sum of the word embedding and positional encoding. However, since the decoder is not expected to receive the next time step (i.e., future information), a mask is used in the multi-head attention mechanism.

[0057] In the Encoder-Decoder attention, Q comes from the output of the previous layer of the Decoder, K and V come from the output of the Encoder, and K and V are the same. The dimensions of Q, K, and V are the same, that is, dq = dk = dv.

[0058] The position encoding in Transformer is mainly achieved through the following calculation method:

[0059]

[0060] Wherein, PE is a two-dimensional matrix, the size is the same as the dimension of the input embedding, the row represents the word, and the column represents the word vector; pos represents the position of the word in the sentence; d model represents the dimension of the word vector; i represents the position of the word vector. Therefore, the above formula represents that the sin variable is added at the even position of the word vector of each word, and the cos variable is added at the odd position, so as to fill the entire PE matrix.

[0061] Therefore, the traditional text classification based on the Transformer uses the Encoder module thereof, the current Transformer text classification network has the problem of slow convergence speed, the fixed dimension of the word embedding coding limits will cause semantic information loss and redundancy, and the classification effect of the text with a longer length (greater than 512) will be poor.

[0062] Based on this, the embodiment of the present application provides a text classification method, the input matrix is obtained by adding the text embedding matrix corresponding to the text to be classified and the quantum position coding matrix corresponding to the text to be classified, and the first feature extraction network and the second feature extraction network used for extracting features both include a variational quantum circuit, so that the third feature matrix input into the classifier actually contains the quantum position coding information corresponding to the text to be classified, the embedding coding information of the word itself, and the more deep semantic information obtained by entanglement between different words through the variational quantum circuit, and the like. Therefore, the problem of semantic information loss and redundancy caused by the fixed dimension of the word embedding coding limit in the prior art can be effectively alleviated, so that better text classification effect can be realized. In addition, after the multi-head attention layer outputs the second feature matrix, the initial input matrix and the second feature matrix are input into the second feature extraction network again for feature extraction, which can further reduce the loss of semantic information, thereby improving the robustness of the text classification network. It can be understood that the quantum position coding matrix is a quantum position coding matrix obtained by using a quantum circuit to code on the basis of the traditional position coding, and is position information represented by quantum state information.

[0063] Please refer to Figure 2 is a block schematic diagram of an electronic device 100 provided by the embodiment of the present application. The electronic device 100 includes a memory 110, a processor 120 and a communication module 130. The memory 110, the processor 120 and the communication module 130 are directly or indirectly electrically connected to each other to realize the transmission or interaction of data. For example, these elements can be electrically connected to each other through one or more communication buses or signal lines.

[0064] The memory 110 is configured to store data or programs. The memory 110 can be, but is not limited to, a Random Access Memory (RAM), a Read Only Memory (ROM), a Programmable Read-Only Memory (PROM), an Erasable Programmable Read-Only Memory (EPROM), an Electric Erasable Programmable Read-Only Memory (EEPROM), etc.

[0065] The processor 120 is configured to read / write the data or programs stored in the memory 110, and perform corresponding functions. For example, when the processor 120 executes the computer program stored in the memory 110, the text classification method disclosed in the embodiments of the present application can be implemented.

[0066] The communication module 130 is configured to establish a communication connection between the electronic device 100 and other communication terminals through a network, and configured to receive / transmit data through the network.

[0067] In the embodiments, the electronic device 100 can be, but is not limited to, a server, a PC (Personal Computer), a smart phone, a tablet computer, a navigation device, a game device, etc.

[0068] It should be understood that, Figure 2 The structure shown is only a structural schematic diagram of the electronic device 100, and the electronic device 100 can further include more or less components than those shown in the embodiments, or have a different configuration from that shown in the embodiments. Figure 2 The components shown in the embodiments can be implemented in hardware, software or a combination thereof. Figure 1 Figure 2 The computer program stored in the computer readable storage medium can be executed by the processor 120 to implement the text classification method disclosed in the embodiments of the present application.

[0069] The computer program stored in the computer readable storage medium can be executed by the processor 120 to implement the text classification method disclosed in the embodiments of the present application.

[0070] Please refer to Figure 3 for a flowchart of the text classification method provided by the embodiments of the present application. It should be noted that the text classification method provided by the embodiments of the present application is not limited to the flowchart shown in the embodiments. Figure 3 ​and the specific sequence below is limited, it should be understood that in other embodiments, part of the steps of the text classification method provided by the embodiments of the present application can be exchanged according to actual needs, or part of the steps can be omitted or deleted. The text classification method can be applied to Figure 2 The electronic device 100 shown in the figure will be described in detail below. Figure 3 The specific process shown in the figure will be described in detail below.

[0071] Step S301, input the text to be classified into the pre-trained text classification network, and process the text to be classified using the word vector layer in the text classification network to obtain an input matrix corresponding to the text to be classified. The input matrix is obtained by adding the text embedding matrix corresponding to the text to be classified and the quantum position encoding matrix corresponding to the text to be classified.

[0072] In one embodiment, as Figure 4 The text classification network can include a word vector layer, a first feature extraction network, a multi-head attention layer, a second feature extraction network, and a classifier. The first feature extraction network and the second feature extraction network each include a variational quantum circuit (VQC).

[0073] The text embedding matrix corresponding to the text to be classified is a word-level sentence vector representation that includes embedding encoding information of each word in the text. The quantum position encoding matrix corresponding to the text to be classified is a matrix that includes quantum position encoding information of the text. Adding the text embedding matrix that includes word embedding encoding information and the quantum position encoding matrix that includes quantum position encoding information of the text, the obtained input matrix will be used as the input of the next layer.

[0074] Step S302, using the first feature extraction network in the text classification network to extract features from the input matrix to obtain a plurality of first feature matrices.

[0075] In this embodiment, since the first feature extraction network includes a variational quantum circuit, when the input matrix is extracted, the variational quantum circuit can be used for quantum-enhanced deep semantic extraction, thereby obtaining deeper semantic information obtained by entanglement between different words through the variational quantum circuit.

[0076] Step S303, using the multi-head attention layer in the text classification network to perform attention calculation on the plurality of first feature matrices to obtain a second feature matrix.

[0077] In the embodiment, the multi-head attention layer can adopt the multi-head attention mechanism in the Transformer, take the plurality of first feature matrices as inputs of the multi-head attention layer, the multi-head means that the attention layer calculates a plurality of groups of attention, each group of attention focuses on different parts of the input information, realizes information extraction from different angles, and obtains the second feature matrix.

[0078] In step S304, the input matrix and the second feature matrix are input into a second feature extraction network in the text classification network for feature extraction to obtain a third feature matrix; the first feature extraction network and the second feature extraction network each include a variational quantum circuit.

[0079] In the embodiment, considering that there may be a case of loss of part of semantic information in the process of obtaining the second feature matrix, after obtaining the second feature matrix, the second feature matrix and the original input matrix are input into the second feature extraction network again for feature extraction to obtain the third feature matrix. Since the second feature extraction network also includes a variational quantum circuit, the second feature extraction network can also perform quantum-enhanced deep semantic extraction through the variational quantum circuit when performing feature extraction, so as to obtain deeper semantic information obtained by entanglement between different words through the variational quantum circuit.

[0080] In step S305, the third feature matrix is input into a classifier in the text classification network to obtain a category corresponding to the text to be classified.

[0081] In the embodiment, a softmax classifier can be used for text classification, and after obtaining the third feature matrix, the third feature matrix is input into the softmax classifier, the softmax classifier outputs a probability corresponding to each category, and the category corresponding to the text to be classified can be determined according to the probability corresponding to each category.

[0082] The text classification method provided in the embodiment can effectively alleviate the problems of semantic information loss and redundancy caused by the fixed dimension of word embedding coding in the prior art, so as to realize better text classification effect; in addition, after the multi-head attention layer outputs the second feature matrix, the initial input matrix and the second feature matrix are input into the second feature extraction network again for feature extraction, which can further reduce the loss of semantic information, thereby improving the robustness of the text classification network.

[0083] In one embodiment, see Figure 5 The above-mentioned word vector layer may include a word embedding layer and a quantum position coding layer. The above-mentioned step S301 may specifically include: using the word embedding layer to perform word embedding coding on the text to be classified to obtain a text embedding matrix; using the quantum position coding layer to perform quantum position coding on the text to be classified to obtain a quantum position coding matrix; adding the text embedding matrix and the quantum position coding matrix to obtain an input matrix corresponding to the text to be classified.

[0084] Among them, the word embedding layer can use methods such as Word2Vec and GlovVe to represent each word in the text to be classified as a word vector, and then concatenate them to obtain the word-level sentence vector representation, that is, the text embedding matrix.

[0085] For example, the dictionary corresponding to "Women's singles lost the bronze medal" is {"Women": [0.1, 0.3], "Single": [0.01, 0.1], "Pain": [0.01, 0.7], "Loss": [0.01, 0.2], "Bronze": [0.01, 0.8], "Card": [0.01, 0.3]}. This converts the above text into a 6*2 text embedding matrix. It should be noted that in actual applications, the embedding dimension of each word should be set to be consistent with the actual number of qubits used, and the initial text embedding remains unchanged throughout the network.

[0086] The quantum position encoding layer can be implemented using the aforementioned variational quantum circuit. This involves applying position encoding to the text to be classified using the Transformer to obtain a PE matrix. This PE matrix is ​​then used as the input to the quantum position encoding layer, which then performs quantum position encoding to ultimately obtain a quantum position encoding matrix. After obtaining the text embedding matrix and quantum position encoding matrix corresponding to the text to be classified, the text embedding matrix and the quantum position encoding matrix are added together to obtain the input matrix corresponding to the text to be classified.

[0087] It should be noted that although the quantum position encoding layer, the first feature extraction network, and the second feature extraction network in this embodiment all use variational quantum circuits, the parameters of each variational quantum circuit after optimization (training) are different. It is understandable that the variational quantum circuits used in the text classification network of this embodiment have the same structure, but the parameters of the iterative optimization are different.

[0088] In one embodiment, still referring to Figure 5 The above-mentioned first feature extraction network includes multiple variational quantum circuits, and the parameters corresponding to each of the multiple variational quantum circuits are different; the above-mentioned step S302 may specifically include: inputting the input matrix into each variational quantum circuit in the first feature extraction network for feature extraction, and obtaining a first feature matrix output by each variational quantum circuit.

[0089] In this embodiment, three variational quantum circuits are taken as an example, the parameters of which are iteratively optimized, and each of the three variational quantum circuits performs feature extraction on the input matrix to output a corresponding first feature matrix. Therefore, the first feature extraction network outputs three first feature matrices, which are input into the multi-head attention layer for attention calculation. In this case, the three first feature matrices output by the first feature extraction network actually correspond to the three vectors Q, K, and V of the input multi-head attention module in the Transformer.

[0090] In an implementation, still referring to Figure 5 The second feature extraction network described above includes a first data processing layer, a variational quantum circuit, and a second data processing layer. The step S304 can specifically include: adding and normalizing the input matrix and the second feature matrix by using the first data processing layer to obtain a first intermediate matrix; performing feature extraction on the first intermediate matrix by using the variational quantum circuit to obtain a second intermediate feature matrix; and adding and normalizing the first intermediate matrix and the second intermediate feature matrix by using the second data processing layer to obtain a third feature matrix. It can be understood that, in this embodiment, the variational quantum circuit performs deep semantic coding on the first intermediate matrix, so that the memory depth of the network is wider.

[0091] In this embodiment, the first data processing layer adds the second feature matrix output by the multi-head attention layer to the input matrix, and normalizes the matrix obtained after the addition to obtain the first intermediate matrix. The normalization processing is to subtract the mean of the data in the matrix obtained after the addition to centralize the data, and then divide the centralized data by the standard deviation to obtain the standardized data, i.e., the first intermediate matrix.

[0092] The variational quantum circuit in the second feature extraction network performs quantum-enhanced deep semantic extraction on the first intermediate matrix output by the first data processing layer, so as to obtain deeper semantic information between different words entangled by the variational quantum circuit, i.e., the second intermediate feature matrix.

[0093] The second data processing layer adds the second intermediate feature matrix output by the variational quantum circuit to the first intermediate matrix output by the first data processing layer, and normalizes the matrix obtained after the addition to obtain the third feature matrix. Similar to the first data processing layer, the second data processing layer also normalizes the matrix, which is to subtract the mean of the data in the matrix obtained after the addition to centralize the data, and then divide the centralized data by the standard deviation to obtain the standardized data, i.e., the third feature matrix.

[0094] The text classification method provided by the embodiment of the application can effectively reduce the loss of semantic information, improve the robustness of the text classification network, reduce the difference between vectors, accelerate the convergence speed of the network, and achieve better convergence effect through matrix addition and standardization processing.

[0095] In one embodiment, referring to Figure 6 , the variational quantum circuit can include a data encoding layer, a variational layer, and a quantum measurement layer, each of the variational quantum circuits used in the text classification network can adopt the structure of Figure 6 , and each needs to be used after parameter optimization.

[0096] In the embodiment, the parameter optimization step of the variational quantum circuit includes: inputting a training sample into a pre-constructed variational quantum circuit, encoding the training sample into quantum state feature data through the data encoding layer; inputting the quantum state feature data into the variational layer, performing unitary operation on the quantum state feature data through the variational layer to obtain multi-qubit entanglement data; inputting the multi-qubit entanglement data into the quantum measurement layer, measuring the expected value on each quantum bit through the quantum measurement layer to obtain prediction information; and updating the parameters of the variational quantum circuit according to the prediction information.

[0097] Wherein, the quantum bit is a basic concept of quantum computing and quantum information, and the quantum state of a single quantum bit can be represented as:

[0098] Wherein, α and β are complex numbers, and |α| 2 +|β| 2 =1.

[0099] Generally, the quantum state of n quantum bits can be represented as: Wherein is the complex amplitude of the ground state, x i ∈{0,1}, and

[0100] The data encoding layer is to encode the data after encoding the text semantics into a quantum state, that is, to convert the classical input vector into a rotation angle to guide the rotation of a single quantum bit. As shown in Figure 7 , assuming that the embedded representation of a certain Chinese character is [x1, x2, x3, x4], after encoding by the data encoding layer, R x (x1) is the quantum gate of x1 angle conversion of the first dimension, R x (x2) is the quantum gate of x2 angle conversion of the first dimension, R x (x3) is the quantum gate of x3 angle conversion of the first dimension, and R x(x4) are x4-angle converted quantum gates of the first dimension, which will be input to the next layer and will not be optimized in the training of the whole network.

[0101] After encoding the training sample into quantum state feature data, a series of unitary operations are needed to be input into the variational layer. Figure 7 For example, the variational layer generally includes a plurality of cycles of single-qubit rotation gates Rz and CNOT gates to increase the depth of the variational layer. The CNOT gate is applied to each pair of single-qubit rotation gates with fixed adjacent 1 and 2 to generate multi-qubit entanglement. The rotation angle of the single-qubit rotation gate is randomly initialized and will be updated in the iterative optimization process based on the gradient descent method.

[0102] The last layer of the variational quantum circuit is the quantum measurement layer, where the expected value on each qubit is measured. The VQC layer described above is implemented using QPanda, so the dimension of the returned result is the number of qubits set, that is, the returned result is a fixed-length vector that needs to be further processed on a classical computer.

[0103] In this embodiment, the optimization of the variational quantum circuit is performed by the parameter shift method:

[0104]

[0105] where x represents the input training sample, represents the expectation of the observed value (i.e. the prediction information described above), I(x) represents the quantum state feature data encoded from x into a quantum state, I H (x) represents the conjugate matrix of I(x); i represents the index of the variational layer, θ i represents the parameters that can be optimized in the variational quantum circuit (i.e. the rotation angle of the single-qubit rotation gate); G i (θ i ) is a single-qubit rotation gate, so the gradient of f(x; θ i ) with respect to θ i is:

[0106]

[0107] Specifically, the parameter optimization process of the variational quantum circuit can be: input the training sample, shift the optimizable parameters in the variational quantum circuit forward by to obtain a prediction information, then shift the optimizable parameters in the variational quantum circuit backward by to obtain another prediction information; based on the two obtained prediction information and the respective corresponding optimizable parameters, calculate and to obtain the gradient of f(x; θ i ) with respect to θi Gradient Then, the gradient descent algorithm is used to optimize the parameters, so that the network's prediction information can be close to the actual results. Specifically, after calculating the gradient, the following formula is used to update the parameter θ to θ′:

[0108]

[0109] Among them, lr is the learning rate, To find the gradient.

[0110] It can be understood that for each variational quantum circuit in the text classification network, parameter optimization will be performed separately in the above manner. In the parameter optimization process, different training samples can be selected to finally obtain variational quantum circuits with different parameters for application in the text classification network.

[0111] In order to execute the corresponding steps in the above embodiments and various possible methods, a method for implementing a text classification device is given below. Figure 8 , which is a functional module diagram of a text classification device 800 provided in an embodiment of the present invention. It should be noted that the basic principles and technical effects of the text classification device 800 provided in this embodiment are the same as those of the above-mentioned embodiments. For the sake of simplicity, any parts not mentioned in this embodiment can be referred to the corresponding contents in the above-mentioned embodiments. The text classification device 800 includes a word vector processing module 810, a first feature extraction module 820, an attention calculation module 830, a second feature extraction module 840, and a classification module 850.

[0112] Optionally, the above modules can be stored in the form of software or firmware. Figure 2 The memory 110 shown in FIG. 110 or the operating system (OS) of the electronic device 100 may be fixed and may be used by Figure 2 Meanwhile, the data, program codes, etc. required to execute the above modules may be stored in the memory 110.

[0113] The word vector processing module 810 is used to input the text to be classified into a pre-trained text classification network, and use the word vector layer in the text classification network to process the text to be classified to obtain an input matrix corresponding to the text to be classified; the input matrix is ​​obtained by adding the text embedding matrix corresponding to the text to be classified and the quantum position encoding matrix corresponding to the text to be classified.

[0114] It can be understood that the word vector processing module 810 can execute the above step S301.

[0115] The first feature extraction module 820 is configured to perform feature extraction on the input matrix by using a first feature extraction network in the text classification network to obtain a plurality of first feature matrices.

[0116] It can be understood that the word vector processing module 810 can perform the step S302.

[0117] The attention calculation module 830 is configured to perform attention calculation on the plurality of first feature matrices by using a multi-head attention layer in the text classification network to obtain a second feature matrix.

[0118] It can be understood that the attention calculation module 830 can perform the step S303.

[0119] The second feature extraction module 840 is configured to perform feature extraction on the input matrix and the second feature matrix by using a second feature extraction network in the text classification network to obtain a third feature matrix; and the first feature extraction network and the second feature extraction network each include a variational quantum circuit.

[0120] It can be understood that the second feature extraction module 840 can perform the step S304.

[0121] The classification module 850 is configured to input the third feature matrix into a classifier in the text classification network to obtain a category corresponding to the text to be classified.

[0122] It can be understood that the classification module 850 can perform the step S305.

[0123] Optionally, the word vector layer includes a word embedding layer and a quantum position encoding layer, and the word vector processing module 810 is specifically configured to: perform word embedding coding on the text to be classified by using the word embedding layer to obtain a text embedding matrix; perform quantum position coding on the text to be classified by using the quantum position encoding layer to obtain a quantum position encoding matrix; and add the text embedding matrix and the quantum position encoding matrix to obtain the input matrix corresponding to the text to be classified.

[0124] Optionally, the first feature extraction network includes a plurality of variational quantum circuits, and the plurality of variational quantum circuits each correspond to different parameters; and the first feature extraction module 820 is specifically configured to perform feature extraction on the input matrix by inputting the input matrix into each variational quantum circuit in the first feature extraction network to obtain a first feature matrix output by each variational quantum circuit.

[0125] Optionally, the second feature extraction network comprises a first data processing layer, a variational quantum circuit and a second data processing layer; and the second feature extraction module 840 is specifically configured to: perform addition and normalization processing on the input matrix and the second feature matrix by using the first data processing layer to obtain a first intermediate matrix; perform feature extraction on the first intermediate matrix by using the variational quantum circuit to obtain a second intermediate feature matrix; and perform addition and normalization processing on the first intermediate matrix and the second intermediate feature matrix by using the second data processing layer to obtain the third feature matrix.

[0126] Optionally, the variational quantum circuit comprises a data encoding layer, a variational layer and a quantum measurement layer, and the variational quantum circuit can be parameter-optimized in the following manner: inputting a training sample into a pre-constructed variational quantum circuit, encoding the training sample into quantum state feature data through the data encoding layer; inputting the quantum state feature data into the variational layer, performing unitary operation on the quantum state feature data through the variational layer to obtain multi-qubit entanglement data; inputting the multi-qubit entanglement data into the quantum measurement layer, measuring the expected value on each qubit through the quantum measurement layer to obtain prediction information; and updating the parameters of the variational quantum circuit according to the prediction information.

[0127] The text classification device provided by the embodiment of the application inputs the text to be classified into a pre-trained text classification network through a word vector processing module, processes the text to be classified by using a word vector layer in the text classification network to obtain an input matrix corresponding to the text to be classified; the input matrix is obtained by adding a text embedding matrix corresponding to the text to be classified and a quantum position encoding matrix corresponding to the text to be classified. A first feature extraction module extracts features of the input matrix by using a first feature extraction network in the text classification network to obtain a plurality of first feature matrices. An attention calculation module calculates attention of the plurality of first feature matrices by using a multi-head attention layer in the text classification network to obtain a second feature matrix. A second feature extraction module inputs the input matrix and the second feature matrix into a second feature extraction network in the text classification network to extract features to obtain a third feature matrix; the first feature extraction network and the second feature extraction network both comprise a variational quantum circuit. A classification module inputs the third feature matrix into a classifier in the text classification network to obtain a category corresponding to the text to be classified.

[0128] Since the input matrix is obtained by adding the text embedding matrix corresponding to the text to be classified and the quantum position encoding matrix corresponding to the text to be classified, and the first feature extraction network and the second feature extraction network used for extracting features both include a variational quantum circuit, the third feature matrix input into the classifier actually contains quantum position encoding information corresponding to the text to be classified, embedding encoding information of the word itself, and more dimensional semantic information such as deeper semantic information obtained by entanglement between different words through the variational quantum circuit, so as to effectively alleviate the problem of semantic information loss and redundancy caused by limiting the fixed dimension of word embedding encoding in the prior art, thereby realizing better text classification effect. In addition, after the multi-head attention layer outputs the second feature matrix, the initial input matrix and the second feature matrix are input into the second feature extraction network again for feature extraction, which can further reduce the loss of semantic information, thereby improving the robustness of the text classification network.

[0129] In several embodiments provided in the present application, it should be understood that the disclosed apparatus and method can also be implemented in other ways. The apparatus embodiments described above are only schematic, for example, the flowcharts and block diagrams in the drawings show the possible implementation architectures, functions and operations of the apparatus, method and computer program product according to the embodiments of the present application. In this regard, each block in the flowcharts or block diagrams can represent a module, a program segment or a part of code, which contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur in different orders than that shown in the drawings. For example, two consecutive blocks can actually be executed substantially in parallel, and sometimes they can be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and the combination of blocks in the block diagrams and / or flowcharts, can be implemented by a dedicated hardware-based system for implementing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0130] In addition, each functional module in the embodiments of the present application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0131] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application or the parts of the technical solutions that essentially contribute to the prior art or the parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in the various embodiments of the present application. The aforementioned storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0132] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A text classification method, characterized in that: The method comprises: Inputting a text to be classified into a pre-trained text classification network, processing the text to be classified using a word vector layer in the text classification network to obtain an input matrix corresponding to the text to be classified; the input matrix is ​​obtained by adding a text embedding matrix corresponding to the text to be classified and a quantum position encoding matrix corresponding to the text to be classified, which represents position information of the text to be classified based on quantum state information; Using a first feature extraction network in the text classification network to perform feature extraction on the input matrix to obtain a plurality of first feature matrices; Performing attention calculation on the multiple first feature matrices using a multi-head attention layer in the text classification network to obtain a second feature matrix; Inputting the input matrix and the second feature matrix into a second feature extraction network in the text classification network for feature extraction to obtain a third feature matrix; both the first feature extraction network and the second feature extraction network include variational quantum circuits; The third feature matrix is ​​input into the classifier in the text classification network to obtain the category corresponding to the text to be classified.

2. The method according to claim 1, characterized in that The word vector layer includes a word embedding layer and a quantum position encoding layer; the word vector layer in the text classification network is used to process the text to be classified to obtain an input matrix corresponding to the text to be classified, including: Using the word embedding layer to perform word embedding encoding on the text to be classified to obtain a text embedding matrix; Performing quantum position encoding on the text to be classified using the quantum position encoding layer to obtain a quantum position encoding matrix; The text embedding matrix and the quantum position encoding matrix are added together to obtain an input matrix corresponding to the text to be classified.

3. The method according to claim 1, characterized in that The first feature extraction network includes a plurality of variational quantum circuits, each of which corresponds to a different parameter. The first feature extraction network in the text classification network is used to extract features from the input matrix to obtain a plurality of first feature matrices, including: The input matrix is ​​input into each variational quantum circuit in the first feature extraction network for feature extraction, so as to obtain a first feature matrix output by each variational quantum circuit.

4. The method according to claim 1, wherein The second feature extraction network includes a first data processing layer, a variational quantum circuit, and a second data processing layer; the input matrix and the second feature matrix are input into the second feature extraction network in the text classification network for feature extraction to obtain a third feature matrix, including: Using the first data processing layer to add and normalize the input matrix and the second feature matrix to obtain a first intermediate matrix; Performing feature extraction on the first intermediate matrix using the variational quantum circuit to obtain a second intermediate feature matrix; The second data processing layer is used to add and normalize the first intermediate matrix and the second intermediate feature matrix to obtain a third feature matrix.

5. The method according to claim 1, wherein The variational quantum circuit includes a data encoding layer, a variational layer, and a quantum measurement layer. The parameter optimization step of the variational quantum circuit includes: Inputting a training sample into a pre-built variational quantum circuit, and encoding the training sample into quantum state feature data through the data encoding layer; Inputting the quantum state characteristic data into the variation layer, and performing a unitary operation on the quantum state characteristic data through the variation layer to obtain multi-qubit entangled data; Inputting the multi-qubit entangled data into the quantum measurement layer, and measuring the expected value of each qubit through the quantum measurement layer to obtain prediction information; The parameters of the variational quantum circuit are updated according to the prediction information.

6. A text classification device, characterized in that: The device comprises: A word vector processing module is used to input the text to be classified into a pre-trained text classification network, and use the word vector layer in the text classification network to process the text to be classified to obtain an input matrix corresponding to the text to be classified; the input matrix is ​​obtained by adding a text embedding matrix corresponding to the text to be classified and a quantum position encoding matrix corresponding to the text to be classified based on quantum state information representing the position information of the text to be classified; A first feature extraction module, configured to perform feature extraction on the input matrix using the first feature extraction network in the text classification network to obtain a plurality of first feature matrices; an attention calculation module, configured to perform attention calculation on the plurality of first feature matrices using a multi-head attention layer in the text classification network to obtain a second feature matrix; a second feature extraction module, configured to input the input matrix and the second feature matrix into a second feature extraction network in the text classification network for feature extraction to obtain a third feature matrix; wherein both the first feature extraction network and the second feature extraction network include a variational quantum circuit; The classification module is used to input the third feature matrix into the classifier in the text classification network to obtain the category corresponding to the text to be classified.

7. The device according to claim 6, characterized in that The word vector layer includes a word embedding layer and a quantum position encoding layer; the word vector processing module is used to use the word embedding layer to perform word embedding encoding on the text to be classified to obtain a text embedding matrix; use the quantum position encoding layer to perform quantum position encoding on the text to be classified to obtain a quantum position encoding matrix; add the text embedding matrix and the quantum position encoding matrix to obtain an input matrix corresponding to the text to be classified.

8. The device according to claim 6, characterized in that The second feature extraction network includes a first data processing layer, a variational quantum circuit, and a second data processing layer; the second feature extraction module is used to use the first data processing layer to add and normalize the input matrix and the second feature matrix to obtain a first intermediate matrix; Performing feature extraction on the first intermediate matrix using the variational quantum circuit to obtain a second intermediate feature matrix; The second data processing layer is used to add and normalize the first intermediate matrix and the second intermediate feature matrix to obtain a third feature matrix.

9. An electronic device, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, the steps of the text classification method according to any one of claims 1 to 5 are implemented.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the text classification method according to any one of claims 1 to 5.

Images