Text Classification Method, Device, Equipment and Storage Medium

Abstract

Embodiments of the present disclosure disclose a text classification method, apparatus, device, and storage medium. The method includes: determining word vectors corresponding to each word in the text to be classified; sequentially inputting the word vectors corresponding to each word into a neuron model of a spiking neural network for processing in the order of the text; when the current word vector is the last word vector in the text to be classified, determining a classification result of the text to be classified according to the output result of the neuron model; wherein the neuron model determines a leakage coefficient according to the output result at the (i-1)-th moment and a target weight matrix, and the leakage coefficient is used to determine the output result of the neuron model at the i-th moment. The technical solution of the embodiments of the present disclosure realizes the classification of long-sequence texts with fewer computing resources.

Description

Technical Field

[0001] Embodiments of the present invention relate to the technical field of natural language processing, and in particular, to a text classification method, apparatus, device, and storage medium. Background Art

[0002] Natural Language Processing (NLP) has a development history of more than a decade since its rise. In the field of NLP, Long Short-Term Memory (LSTM) is a classic choice for dealing with the problem of dynamic input sequences commonly existing in NLP.

[0003] However, in related technologies, a relatively large amount of computing resources are required to classify long-sequence texts. Summary of the Invention

[0004] The present disclosure provides a text classification method, apparatus, device, and storage medium to enable classifying long-sequence texts with fewer computing resources.

[0005] In a first aspect, embodiments of the present disclosure provide a text classification method, including:

[0006] Determine word vectors corresponding to each word in the text to be classified;

[0007] Input the word vectors corresponding to each word into the neuron model of the spiking neural network for processing in sequence according to the text order;

[0008] When the current word vector is the last word vector in the text to be classified, determine the classification result of the text to be classified according to the output result of the neuron model;

[0009] Wherein, the neuron model determines a leakage coefficient according to the output result at the (i - 1)-th moment and the target weight matrix, and the leakage coefficient is used to determine the output result of the neuron model at the i-th moment.

[0010] Further, the signal types of the word vectors include spiking signal types and analog signal types.

[0011] Further, inputting the word vectors corresponding to each word into the neuron model of the spiking neural network for processing in sequence according to the text order includes:

[0012] For the word vector at the i-th moment input into the neuron model, obtain the pre-trained first weight matrix and the membrane potential information at the (i - 1)-th moment;

[0013] Determine a temporary membrane potential according to the word vector at the i-th moment, the first weight matrix, and the membrane potential information at the (i - 1)-th moment;

[0014] Determine the leakage coefficient according to the output result at the (i - 1)-th moment and the target weight matrix;

[0015] Determine the firing result at the i-th moment and reset the membrane potential according to the temporary membrane potential and the preset threshold;

[0016] Determine the membrane potential information at the i-th moment according to the reset membrane potential and the leakage coefficient.

[0017] Furthermore, the output result at the (i - 1)-th moment includes the membrane potential information at the (i - 1)-th moment and the firing result at the (i - 1)-th moment.

[0018] Among them, determining the leakage coefficient according to the output result at the (i - 1)-th moment and the target weight matrix includes one of the following methods:

[0019] Determine the leakage coefficient according to the membrane potential information at the (i - 1)-th moment and the target weight matrix;

[0020] Determine the leakage coefficient according to the firing result at the (i - 1)-th moment and the target weight matrix.

[0021] Furthermore, the target weight matrix includes a second weight matrix and a third weight matrix.

[0022] Among them, determining the leakage coefficient according to the membrane potential information at the (i - 1)-th moment and the target weight matrix includes:

[0023] Determine a first result according to the word vector at the i-th moment and the second weight matrix;

[0024] Determine a second result according to the membrane potential information at the (i - 1)-th moment and the third weight matrix;

[0025] Determine the leakage coefficient according to the first result, the second result and the activation function.

[0026] Furthermore, determining the membrane potential information at the i-th moment according to the reset membrane potential and the leakage coefficient includes:

[0027] Determine the first membrane potential information according to the reset membrane potential and the leakage coefficient;

[0028] Determine the second membrane potential information according to the membrane potential information at the (i - 1)-th moment and the leakage coefficient;

[0029] Determine the membrane potential information at the i-th moment according to the first membrane potential information and the second membrane potential information.

[0030] Furthermore, determining the firing result at the i-th moment according to the temporary membrane potential and the preset threshold includes one of the following methods:

[0031] Determine the firing result at the i-th moment according to the temporary membrane potential, the preset threshold, and the simulated activation function, where the firing result is a simulated value;

[0032] If the temporary membrane potential is greater than or equal to the preset threshold, determine that the firing result is 1. If the temporary membrane potential is less than the preset threshold, determine that the firing result is 0, where the firing result is a pulse value.

[0033] Further, determine the reset membrane potential at the i-th moment according to the temporary membrane potential and the preset threshold, including:

[0034] If the temporary membrane potential is greater than or equal to the preset threshold, determine the preset resting potential as the reset membrane potential;

[0035] If the temporary membrane potential is less than the preset threshold, determine the temporary membrane potential as the reset membrane potential.

[0036] Further, determine the temporary membrane potential according to the word vector at the i-th moment, the first weight matrix, and the membrane potential information at the (i - 1)-th moment, including:

[0037] Perform an integration operation according to the word vector at the i-th moment and the first weight matrix to obtain an intermediate vector;

[0038] Sum the intermediate vector and the membrane potential information at the (i - 1)-th moment to obtain the temporary membrane potential.

[0039] In a second aspect, an embodiment of the present disclosure further provides a text classification device, where the text classification device includes:

[0040] A word vector determination module, configured to determine word vectors corresponding to each word in the text to be classified;

[0041] A word vector processing module, configured to sequentially input the word vectors corresponding to each word into the neuron model of the spiking neural network for processing according to the text sequence;

[0042] A classification result determination module, configured to determine the classification result of the text to be classified according to the output result of the neuron model when the current word vector is the last word vector in the text to be classified;

[0043] Wherein, the neuron model determines a leakage coefficient according to the output result at the (i - 1)-th moment and the target weight matrix, and the leakage coefficient is used to determine the output result of the neuron model at the i-th moment.

[0044] In a third aspect, an embodiment of the present disclosure further provides a device, where the device includes:

[0045] One or more processors;

[0046] A storage device, configured to store one or more programs;

[0047] When one or more programs are executed by one or more processors, such that the one or more processors implement the text classification method provided in any embodiment of the present disclosure.

[0048] In a fourth aspect, embodiments of the present disclosure further provide a storage medium containing computer-executable instructions, which are used to execute the text classification method provided in any embodiment of the present disclosure when executed by a computer processor.

[0049] Embodiments of the present disclosure determine word vectors corresponding to each word in the text to be classified; sequentially input the word vectors corresponding to each word into the neuron model of the spiking neural network for processing in the order of the text; when the current word vector is the last word vector in the text to be classified, determine the classification result of the text to be classified according to the output result of the neuron model; wherein, the neuron model determines a leakage coefficient according to the output result at the (i - 1)-th moment and the target weight matrix, and the leakage coefficient is used to determine the output result of the neuron model at the i-th moment. The spiking neural network is used to classify the text to be classified. The spiking neural network has characteristics such as low power consumption and low storage requirements, reducing the computing resources required in the classification process; the leakage coefficient used by the neuron model when processing the word vector input at the i-th moment is determined according to the output result at the (i - 1)-th moment and the target weight matrix, making the connection relationship between the output result corresponding to any word vector in the determined text to be classified and the output results of its previous word vectors closer, and further making the classification result determined according to the output result of the neuron model more accurate. It realizes the classification of long-sequence text with less computing resources without causing gradient disappearance and explosion. Description of the Drawings

[0050] Figure 1 is a flowchart of a text classification method in an exemplary embodiment of the present disclosure;

[0051] Figure 2 is a flowchart of a text classification method in an exemplary embodiment of the present disclosure;

[0052] Figure 3 is a flowchart of determining a temporary membrane potential in an exemplary embodiment of the present disclosure;

[0053] Figure 4 is a schematic diagram of information processing of a neuron model in an exemplary embodiment of the present disclosure;

[0054] Figure 5 is a schematic structural diagram of a text classification device in an exemplary embodiment of the present disclosure;

[0055] Figure 6 is a schematic structural diagram of a device in an exemplary embodiment of the present disclosure. Detailed implementation manners

[0056] The present disclosure will be further described in detail below with reference to the accompanying drawings and embodiments. It can be understood that the specific embodiments described herein are only used to explain the present disclosure, rather than limiting the present disclosure. In addition, it should be noted that, for the sake of description, only parts related to the present disclosure rather than all structures are shown in the accompanying drawings. In addition, the embodiments in the present disclosure and the features in the embodiments may be combined with each other without conflict.

[0057] Figure 1 The figure is a flowchart of a text classification method provided for an exemplary embodiment of the present disclosure. This embodiment is applicable to the case of classifying long-sequence text content. This method can be executed by a text classification device, which can be implemented by software and / or hardware, and the text classification device can be configured on a computing device, and includes the following steps:

[0058] S101. Determine word vectors corresponding to each word in the text to be classified.

[0059] Among them, the text to be classified can be understood as a set of multiple words that need to judge the content expressed by the text according to the overall semantics. Optionally, the text to be classified can be various comments on the Internet in the case of big data, such as movie reviews, music reviews, and shopping reviews, etc. The word vector can be understood as a vector of words used to form the text to be classified, and the word can be a single character or multiple characters, and the embodiments of the present disclosure do not limit this.

[0060] A possible implementation manner is to obtain the text to be classified that needs to be text-classified. The text to be classified can be comments with different numbers of words. Different word segmentation methods are used to segment the text to be classified according to the language type of the text to be classified, and it is divided into multiple word vectors. Exemplarily, if the text to be classified is an English comment, the text to be classified can be divided by means of splitting by space, by symbol, or stemming, etc., and each word obtained by the division is used as a word vector; if the text to be classified is a Chinese comment, the text to be classified can be divided by means of dictionary matching-based, statistics-based, or deep learning-based methods, etc., and each phrase obtained by the division is used as a word vector.

[0061] S102. Input the word vectors corresponding to each word into a neuron model in a spiking neural network for processing in the order of the text.

[0062] Among them, a spiking neuron network (SNN) can be understood as an abstraction of the human brain neuron network from the perspective of information processing, establishing a certain simple model, and forming a certain network model according to different connection methods. It encodes the input data in the form of electrical pulses to achieve the processing of the input data, and it can be composed of one or more neuron models. The spiking neuron network has the characteristics of low power consumption, high computing speed, and low storage requirements. Moreover, compared with other artificial neural networks, its simulated neurons are closer to reality and also take into account the influence of time information. Among them, the neuron model can be implemented by a hardware processing unit with input and output in a neuromorphic circuit.

[0063] Furthermore, the signal types of the word vectors include a pulse signal type and an analog signal type.

[0064] Furthermore, the neuron model determines the leakage coefficient according to the output result at the (i - 1)-th moment and the target weight matrix, and the leakage coefficient is used to determine the output result of the neuron model at the i-th moment.

[0065] Among them, the output result can be understood as the result obtained after the neuron model processes the word vector input at the (i - 1)-th moment. This output result can be membrane potential information or a firing result determined according to different word vector signal types.

[0066] The target weight matrix can be understood as a matrix value obtained by training with training samples to adapt to the usage environment. The target weight matrix can be continuously trained during the text classification process to make the weight matrix more suitable for the usage conditions of text classification. The leakage coefficient can be understood as a coefficient used to simulate the loss of membrane potential information when a biological neuron transmits information, so as to determine the retention degree of its membrane potential information when processing the word vector at the i-th moment, and further determine the output result of the neuron model at the i-th moment.

[0067] A possible implementation method is that after dividing the text to be classified into multiple word vectors, the obtained word vectors are sequentially input into the spiking neuron network in the order of the text, so that the neuron models in the spiking neuron network process the sequentially input word vectors in the chronological order from first to last. Taking the word vector input at the i-th moment as an example, it is input into the neuron model to obtain a processing result, and the leakage coefficient for this word vector determined by the neuron model according to the output result at the (i - 1)-th moment and the target weight is obtained. The retention degree of the above processing result is determined using this leakage coefficient, and the processing result processed by the leakage coefficient is used as the output result at the i-th moment.

[0068] In the embodiments of the present disclosure, the word vector signal type of the input neuron model can be either a pulse signal type or an analog signal type, which enriches the types of information that can be processed by the neural network model in the text classification method. At the same time, a leakage coefficient determined based on the output result of the previous word vector dependent on the input word vector is introduced, making the connection between the output result corresponding to any word vector in the text to be classified and the output results of the previous word vectors closer, and thus making the output result of the word vector processed by the input neuron model more accurate.

[0069] S103. When the current word vector is the last word vector in the text to be classified, determine the classification result of the text to be classified according to the output result of the neuron model.

[0070] Among them, the classification result can be understood as the result determined according to the content bias expressed in the text to be classified, and can include positive and negative. Exemplarily, when the text to be classified is a movie review, the movie review can be classified, and the obtained classification result can illustrate whether the review expresses a positive view praising the movie or a negative view criticizing the movie.

[0071] A possible implementation manner is that when the current word vector is the last word vector in the text to be classified, it indicates that the text to be classified has been completely input into the spiking neural network for processing at the current moment, and the output result obtained after the current word vector is processed by the neuron model is related to all the information in the entire text to be classified. After inputting the output result into the fully connected layer for processing, the classification result of the text to be classified can be obtained.

[0072] In the embodiments of the present disclosure, the word vectors corresponding to each word in the text to be classified are determined; the word vectors corresponding to each word are sequentially input into the neuron model of the spiking neural network for processing according to the text sequence; when the current word vector is the last word vector in the text to be classified, determine the classification result of the text to be classified according to the output result of the neuron model; among them, the neuron model determines the leakage coefficient according to the output result at the (i - 1)th moment and the target weight matrix, and the leakage coefficient is used to determine the output result of the neuron model at the ith moment. The spiking neural network has characteristics such as low power consumption, high computing speed, and low storage requirements. By classifying the text to be classified through the spiking neural network, the computing resources required in the classification process can be reduced, and the problems in the related art, such as a large number of model weights and high requirements for computing resources when performing text classification, can be solved.

[0073] When processing the word vector input at the $i$-th moment, the leakage coefficient used by the neuron model is determined based on the output result at the $(i - 1)$-th moment and the target weight matrix, making the connection relationship between the output results corresponding to any word vector in the text to be classified more compact. As a result, the classification result determined based on the output result of the neuron model is more accurate. Moreover, by dynamically determining the leakage coefficient at each moment, the problem of gradient disappearance and explosion in long sequence text data can be effectively solved, achieving the classification of long sequence text with less computational resources without causing gradient disappearance and explosion, improving the speed of long sequence text classification, and reducing the power consumption and storage requirements for long sequence text classification.

[0074] Figure 2 The following is a flowchart of a text classification method provided by an exemplary embodiment of the present disclosure. The technical solution of this embodiment is further refined on the basis of the above technical solution and may include the following steps:

[0075] S201. Determine the word vectors corresponding to each word in the text to be classified.

[0076] S202. For the word vector at the $i$-th moment input to the neuron model, obtain the pre-trained first weight matrix and the membrane potential information at the $(i - 1)$-th moment.

[0077] Among them, the pre-trained first weight matrix can be understood as a weight coefficient matrix obtained by training according to training samples for processing input word vectors. The membrane potential information can be understood as the value obtained after the word vector information is processed by the neuron model in a spiking neural network, and a membrane potential vector can be formed among multiple membrane potentials according to the time relationship.

[0078] In a possible implementation, for the word vector input to the neuron model at the $i$-th moment, to process it more accurately, it can be associated with the processing result of the word vector input at the previous moment. Therefore, when inputting the word vector at the $i$-th moment, obtain the pre-trained first weight matrix and the membrane potential information obtained after the word vector input at the $(i - 1)$-th moment is processed.

[0079] S203. Determine the temporary membrane potential according to the word vector at the $i$-th moment, the first weight matrix, and the membrane potential information at the $(i - 1)$-th moment.

[0080] Among them, the temporary membrane potential can be understood as an intermediate value obtained by the neuron from receiving the input word vector for processing until obtaining the processing result.

[0081] A possible implementation manner is that the first weight matrix is a weight matrix to be trained for processing word vectors. According to the word vector input at the i-th moment and the obtained first weight matrix, an intermediate vector can be determined. Combining the intermediate vector and the membrane potential information at the (i - 1)-th moment, the temporary membrane potential after the word vector at the i-th moment passes through the neuron model can be determined.

[0082] Further, Figure 3 The following is a flowchart for determining the temporary membrane potential provided by an exemplary embodiment of the present disclosure, which may include the following steps:

[0083] S2031: Perform an integration operation according to the word vector at the i-th moment and the first weight matrix to obtain an intermediate vector.

[0084] A possible implementation manner is to determine the signal type of the word vector input at the i-th moment according to the data type required by the neuron model, as well as the corresponding calculation method. The word vector signal types received by the neuron model in the present disclosure include any one of the pulse signal type and the analog signal type (such as floating-point numbers, integers, etc.). According to the selected signal type, perform a matrix multiplication or convolution operation on the word vector at the i-th moment and the first weight matrix to obtain the intermediate vector.

[0085] Exemplarily, assume X t is used to represent the word vector input at the t-th moment, W is the set first weight matrix, and I t is the intermediate vector corresponding to the t-th moment. Then, according to the different data types required, the calculation method of I t can be as follows:

[0086]

[0087] That is, when the required data type is in the fully connected (linear) form, perform matrix multiplication on the word vector input at the t-th moment and the first weight matrix to obtain the intermediate vector; when the required data type is in the convolution form, perform a convolution operation on the word vector input at the t-th moment and the first weight matrix to obtain the intermediate vector.

[0088] S2032: Sum the intermediate vector and the membrane potential information at the (i - 1)-th moment to obtain the temporary membrane potential.

[0089] Continuing with the above example, assume is used to represent the membrane potential information at the (t - 1)-th moment, is the temporary membrane potential corresponding to the t-th moment. Then, the calculation method of can be as follows:

[0090]

[0091] S204. Determine the leakage coefficient according to the output result at the (i - 1)-th moment and the target weight matrix.

[0092] Among them, the output result at the (i - 1)-th moment includes the membrane potential information at the (i - 1)-th moment and the firing result at the (i - 1)-th moment.

[0093] Among them, the firing result can be understood as the firing output result obtained after the word vector is processed by the neuron model. According to the different signal types of the input word vector, there are different calculation methods.

[0094] A possible implementation method is that according to the different data types required by the neuron model, the input signal type can be a pulse signal type or an analog signal type. Furthermore, the calculation method of the leakage coefficient used to simulate the loss of membrane potential information during information transmission by biological neurons is also different. When processing the word vector input at the i-th moment, the leakage coefficient determined by the output result of the neuron model at the (i - 1)-th moment is required. Since the output result of the neuron model can be the membrane potential information or the firing result determined according to the word vector signal type, the membrane potential information or the firing result can be subjected to matrix multiplication or convolution operation with the target weight matrix according to the different signal types, and the operation result is substituted into the activation function to finally obtain the leakage coefficient determined according to the output result of the neuron model at the (i - 1)-th moment.

[0095] Furthermore, if the output result at the (i - 1)-th moment is the firing result at the (i - 1)-th moment, the firing result can be subjected to matrix multiplication or convolution operation with the target weight matrix according to the signal type of the firing result, and the operation result is substituted into the activation function to obtain the leakage coefficient determined according to the firing result at the (i - 1)-th moment when processing the word vector input at the i-th moment.

[0096] Furthermore, if the output result at the (i - 1)-th moment is the membrane potential information at the (i - 1)-th moment, determine the leakage coefficient according to the membrane potential information at the (i - 1)-th moment and the target weight matrix.

[0097] Among them, the target weight matrix includes a second weight matrix and a third weight matrix.

[0098] Among them, the second weight matrix and the third weight matrix can be understood as two different weight coefficient matrices obtained through training in the pulse neural network.

[0099] The process of determining the leakage coefficient may include the following steps:

[0100] S2041. Determine the first result according to the word vector at the i-th moment and the second weight matrix.

[0101] A possible implementation method is to perform matrix multiplication or convolution operation on the word vector at the i-th moment with the second weight matrix according to the signal type of the word vector at the i-th moment, and determine the operation result as the first result.

[0102] S2042. Determine a second result according to the membrane potential information at the (i - 1)-th moment and the third weight matrix.

[0103] A possible implementation method is that the selected membrane potential information at the (i - 1)-th moment has the same signal type as the input word vector at the i-th moment. Perform matrix multiplication or convolution operation on the membrane potential information at the (i - 1)-th moment with the third weight matrix, and determine the operation result as the second result.

[0104] S2043. Determine a leakage coefficient according to the first result, the second result, and the activation function.

[0105] Among them, the activation function can be understood as a non-linear function often used in neural networks. Using a non-linear function can enable the network to have the ability to simulate non-linear equations, so that the network can map more complex relationships. Optionally, the activation function used in the present disclosure can be the Sigmoid function, which is a common S-shaped function in deep learning and is also called the S-shaped growth curve, and has the properties of monotonic increase and monotonic increase of the inverse function.

[0106] A possible implementation method is to add the first result and the second result, substitute the obtained result after addition into the activation function to obtain a value mapped between 0 and 1, and use this value as the finally obtained leakage coefficient to control the retention degree of the membrane potential information at the i-th moment.

[0107] Exemplarily, assume X t is used to represent the word vector input at the t-th moment, is used to represent the membrane potential information at the (t - 1)-th moment. Assume W in is the second weight matrix, W m is the third weight matrix, and α is the leakage coefficient. Then the calculation method of α can be shown as follows:

[0108]

[0109] S205. Determine the firing result at the i-th moment according to the temporary membrane potential and the preset threshold.

[0110] Among them, the preset threshold can be understood as a preset potential threshold.

[0111] A possible implementation method is to determine the signal type of the firing result at the i-th moment according to the signal type required when processing the word vector in the neuron model. If the required signal type is an analog value, the temporary membrane potential and the preset potential threshold can be substituted into the analog activation function, and the firing result at the i-th moment in the form of an analog value can be obtained by mapping the input temporary membrane potential and the preset threshold to the output end; if the required signal type is a pulse value, the magnitude relationship between the temporary membrane potential and the preset threshold is compared. When the temporary membrane potential is greater than or equal to the preset threshold, the firing result is determined to be 1, and when the temporary membrane potential is less than the preset threshold, the firing result is determined to be 0.

[0112] Continuing with the above example, assume F t is the signal type at the t-th moment and is a pulse value, V th is the preset threshold, f(x, V th ) is the analog activation function, and Y t is the firing result at the t-th moment. Then the calculation method of Y t can be shown as follows:

[0113]

[0114] That is, when the firing result to be output is a pulse value, then output F determined according to the magnitude relationship between the temporary membrane potential th and the preset threshold V t . When the firing result to be output is an analog value, then substitute the temporary membrane potential and the preset threshold V th into the analog activation function, and determine the output result of the analog activation function as the firing result.

[0115] Among them, the analog activation function f(x, V th ) can be related to the preset threshold potential (TR mode) or not related to the preset threshold potential (NTR mode), and can be expressed as:

[0116]

[0117] Among them, the activation function Act(x) is a non-linear function.

[0118] S206. Determine the reset membrane potential at the i-th moment according to the temporary membrane potential and the preset threshold.

[0119] Among them, the reset membrane potential can be understood as the potential value obtained after judging whether the temporary membrane potential at the i-th moment needs to be reset according to the temporary membrane potential.

[0120] A possible implementation method is to determine the relationship between the temporary membrane potential and a preset threshold. According to the determination result, the method result of the pulse type in the firing result can be determined. Based on the pulse firing result, the temporary membrane potential, and the preset resting potential, the reset membrane potential at the i-th moment can be determined.

[0121] Further, if the temporary membrane potential is greater than or equal to the preset threshold, the preset resting potential is determined as the reset membrane potential; if the temporary membrane potential is less than the preset threshold, the temporary membrane potential is determined as the reset membrane potential.

[0122] S207. Determine the membrane potential information at the i-th moment according to the reset membrane potential and the leakage coefficient.

[0123] A possible implementation method is that after determining the reset membrane potential according to the temporary membrane potential, since the value of the leakage coefficient is within the range of [0, 1], the first product of the reset membrane potential and the leakage coefficient can be obtained, and according to a preset leakage constant determined in advance, the sum of the first product and the leakage constant is determined as the membrane potential information at the i-th moment.

[0124] Further, determine the membrane potential information at the i-th moment according to the reset membrane potential, the membrane potential information at the (i - 1)-th moment, and the leakage coefficient.

[0125] A possible implementation method is to determine the first membrane potential information according to the reset membrane potential and the leakage coefficient;

[0126] Determine the second membrane potential information according to the membrane potential information at the (i - 1)-th moment and the leakage coefficient;

[0127] Determine the membrane potential information at the i-th moment according to the first membrane potential information and the second membrane potential information.

[0128] For example, take the product of the reset membrane potential and the leakage coefficient as the first membrane potential information, obtain the difference from the leakage coefficient, and take the product of the difference and the membrane potential information at the (i - 1)-th moment as the second membrane potential information. Determine the sum of the first membrane potential information and the second membrane potential information as the membrane potential information at the i-th moment.

[0129] Continuing with the above example, assume that the reset membrane potential at the t-th moment can be represented by Then the membrane potential information at the i-th moment can be expressed as:

[0130]

[0131] Optionally, assume that the preset leakage constant is β. By calculating the sum of the product of the leakage coefficient and the reset membrane potential and the leakage constant to determine the current cell membrane potential, it can be expressed as:

[0132]

[0133] S208. When the current word vector is the last word vector in the text to be classified, determine the classification result of the text to be classified according to the output result of the neuron model.

[0134] In a possible implementation, when the current word vector is the last word vector in the text to be classified, it can be considered that the text to be classified has been completely input into the spiking neural network for processing at the current moment, and the result after the current word vector is processed by the neuron model is related to the information of each word vector in the entire text to be classified. The output result obtained after its processing by the neuron model can be input into the fully connected layer for processing, and the classification result of the text to be classified is determined according to the processed result.

[0135] Among them, the fully connected layer can be understood as a "classifier" in a neural network where each node is connected to all nodes in the previous layer and is used to comprehensively process all the previously extracted features to obtain a result.

[0136] Exemplarily, Figure 4 The information processing schematic diagram of a neuron model provided by an exemplary embodiment of the present disclosure. The word vector of the pulse signal type or analog signal type at the i-th moment is combined with the membrane potential information obtained at the (i - 1)-th moment and the pre-trained first weight matrix to obtain the temporary membrane potential at the i-th moment; the firing result at the i-th moment and the reset membrane potential at the i-th moment are determined by combining the temporary membrane potential and the preset threshold; the leakage coefficient at the i-th moment is determined by combining the word vector obtained at the i-th moment, the output result at the (i - 1)-th moment, and the target weight matrix, where the output result at the (i - 1)-th moment may include the membrane potential information or the firing result at the (i - 1)-th moment, and the target weight matrix may include the second weight matrix and the third weight matrix. Further, the first result is determined according to the word vector at the i-th moment and the second weight matrix, the second result is determined according to the membrane potential information or the firing result at the (i - 1)-th moment and the third weight matrix, and the leakage coefficient is determined according to the first result and the second result; the membrane potential information at the i-th moment is determined by combining the reset membrane potential and the leakage coefficient.

[0137] Further, Table 1 below shows the performance comparison when different neuron models provided by an exemplary embodiment of the present disclosure process the same IMDB (Internet Movie Database) text classification task.

[0138] Table 1

[0139]

[0140]

[0141] Among them, for the neuron model provided by the present disclosure, since there is a connection relationship between the output result and the previous output result, the model can converge and has good performance.

[0142] Furthermore, Table 2 below shows the comparison of the number of multiply-accumulate operations and the number of parameters per single time step for different neuron models provided by an exemplary embodiment of the present disclosure under the same input (100) and output (256) dimensions.

[0143] Table 2

[0144] The neuron model of the present disclosure LSTM Number of parameters 77568 366592 Number of multiplications 77056 365312 Number of additions 78212 365568

[0145] It shows that the neuron model of the present disclosure reduces the computational complexity and the number of parameters by approximately 78.8% compared to the LSTM model. When applied to actual situations, it can greatly save computational resources and storage resources. Further, combining the contents of Table 1 and Table 2 above, it can be seen that although the performance of the neuron model of the present disclosure is slightly worse than that of the LSTM model, due to the fact that the computational resources required by it are much smaller than those of the LSTM model, its comprehensive performance is far superior to that of the LSTM model. The neuron model provided by the present disclosure can achieve good convergence and power consumption reduction effects.

[0146] The technical solution of the embodiment of the present disclosure determines the leakage coefficient through the output result of the neuron model at the (i - 1)-th moment and the target weight matrix, and determines the membrane potential information at the i-th moment according to the leakage coefficient and the reset membrane potential at the i-th moment, or according to the leakage coefficient, the reset membrane potential at the i-th moment, and the membrane potential information at the (i - 1)-th moment. When the i-th moment is the moment corresponding to the last word vector in the text to be classified, the classification result of the text to be classified is determined according to its output result, making the connection relationship between the output result corresponding to any word vector in the determined text to be classified and the output results of its previous word vectors closer. Furthermore, the classification result determined according to the output result of the neuron model is more accurate, and the spiking neural network has characteristics such as low power consumption, high computing speed, and low storage requirements. By classifying the text to be classified through the spiking neural network, the computational resources required in the classification process can be reduced, and the classification efficiency can be improved.

[0147] Figure 5 It is a schematic structural diagram of a text classification device provided by an exemplary embodiment of the present disclosure. The text classification device includes: a word vector determination module 31, a word vector processing module 32, and a classification result determination module 33.

[0148] Among them, the word vector determination module 31 is used to determine the word vectors corresponding to each word in the text to be classified; the word vector processing module 32 is used to sequentially input the word vectors corresponding to each word into the neuron model of the spiking neural network for processing; the classification result determination module 33 is used to determine the classification result of the text to be classified according to the output result of the neuron model when the current word vector is the last word vector in the text to be classified; wherein, the neuron model determines the leakage coefficient according to the output result at the (i - 1)-th moment and the target weight matrix, and the leakage coefficient is used to determine the output result of the neuron model at the i-th moment.

[0149] The technical solution of this embodiment solves the problems of a large number of model weights and high computational resource requirements during text classification, realizes the classification of long-sequence texts with fewer computational resources, increases the connection between the obtained classification result and the input content, improves the speed of long-sequence text classification, and reduces the power consumption and storage requirements of long-sequence text classification.

[0150] Furthermore, the signal types of the word vectors include spiking signal types and analog signal types.

[0151] Optionally, the word vector processing module 32 includes:

[0152] The information acquisition unit is used to acquire the pre-trained first weight matrix and the membrane potential information at the (i - 1)-th moment for the word vector at the i-th moment input into the neuron model.

[0153] The temporary membrane potential determination unit is used to determine the temporary membrane potential according to the word vector at the i-th moment, the first weight matrix, and the membrane potential information at the (i - 1)-th moment.

[0154] The leakage coefficient determination unit is used to determine the leakage coefficient according to the output result at the (i - 1)-th moment and the target weight matrix.

[0155] The result and potential determination unit is used to determine the firing result at the i-th moment and reset the membrane potential according to the temporary membrane potential and a preset threshold.

[0156] The membrane potential information determination unit is used to determine the membrane potential information at the i-th moment according to the reset membrane potential and the leakage coefficient.

[0157] Furthermore, the output result at the (i - 1)-th moment includes the membrane potential information at the (i - 1)-th moment and the firing result at the (i - 1)-th moment. Determining the leakage coefficient according to the output result at the (i - 1)-th moment and the target weight matrix includes one of the following methods:

[0158] Determine the leakage coefficient according to the membrane potential information at the (i - 1)-th moment and the target weight matrix;

[0159] The leakage coefficient is determined based on the release result at the i-1th moment and the target weight matrix.

[0160] Furthermore, the target weight matrix includes a second weight matrix and a third weight matrix, and the leakage coefficient determination unit is used to: determine the first result based on the word vector at the i-th moment and the second weight matrix; determine the second result based on the membrane potential information at the i-1-th moment and the third weight matrix; determine the leakage coefficient based on the first result, the second result and the activation function.

[0161] Furthermore, the membrane potential information determination unit is used to: determine the first membrane potential information based on the reset membrane potential and the leakage coefficient; determine the second membrane potential information based on the membrane potential information at the i-1th moment and the leakage coefficient; determine the membrane potential information at the i-th moment based on the first membrane potential information and the second membrane potential information.

[0162] Furthermore, the temporary membrane potential determination unit is used to: perform an integration operation based on the word vector at the i-th moment and the first weight matrix to obtain an intermediate vector; and sum the intermediate vector with the membrane potential information at the i-1-th moment to obtain a temporary membrane potential.

[0163] Furthermore, the result and potential determination unit is used to: determine the discharge result at the i-th moment according to the temporary membrane potential, the preset threshold and the simulated activation function, wherein the discharge result is a simulated value; if the temporary membrane potential is greater than or equal to the preset threshold, the discharge result is determined to be 1, and if the temporary membrane potential is less than the preset threshold, the discharge result is determined to be 0, wherein the discharge result is a pulse value; if the temporary membrane potential is greater than or equal to the preset threshold, the preset resting potential is determined as the reset membrane potential; if the temporary membrane potential is less than the preset threshold, the temporary membrane potential is determined to be the reset membrane potential.

[0164] The text classification device provided in the embodiment of the present disclosure can execute the text classification method provided in any embodiment of the present disclosure, and has the corresponding functional modules and beneficial effects of the execution method.

[0165] Figure 6 A schematic diagram of a device structure provided by an exemplary embodiment of the present disclosure is shown in FIG. Figure 6 As shown, the device includes a processor 41, a storage device 42, an input device 43 and an output device 44; the number of processors 41 in the device can be one or more. Figure 6 A processor 41 is taken as an example; the processor 41, storage device 42, input device 43 and output device 44 in the device can be connected by a bus or other means. Figure 6 The example of connecting through bus is taken in the following.

[0166] The storage device 42, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as the program instructions / modules corresponding to the text classification method in the embodiments of the present disclosure (for example, the word vector determination module 31, the word vector processing module 32, and the classification result determination module 33). The processor 41 executes various functional applications and data processing of the device by running the software programs, instructions, and modules stored in the storage device 42, that is, implements the above-mentioned text classification method.

[0167] The storage device 42 may mainly include a program storage area and a data storage area. Among them, the program storage area can store an operating system and application programs required for at least one function; the data storage area can store data created according to the use of the terminal, etc. In addition, the storage device 42 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage devices. In some instances, the storage device 42 may further include a memory remotely set relative to the processor 41, and these remote memories can be connected to the device through a network. Examples of the above network include but are not limited to the Internet, enterprise intranet, local area network, mobile communication network, and their combinations.

[0168] The input device 43 can be used to receive input digital or character information, and generate key signal inputs related to the user settings and function controls of the device, and can include a touch screen, a keyboard, a mouse, etc. The output device 44 may include a display device such as a display screen.

[0169] An exemplary embodiment of the present disclosure also provides a storage medium containing computer-executable instructions, and the computer-executable instructions are used to execute a text classification method when executed by a computer processor. The method includes:

[0170] Determine the word vectors corresponding to each word in the text to be classified;

[0171] Input the word vectors corresponding to each word into the neuron model of the pulsed neural network for processing in sequence according to the text order;

[0172] When the current word vector is the last word vector in the text to be classified, determine the classification result of the text to be classified according to the output result of the neuron model;

[0173] Among them, the neuron model determines the leakage coefficient according to the output result at the (i - 1)-th moment and the target weight matrix, and the leakage coefficient is used to determine the output result of the neuron model at the i-th moment.

[0174] Of course, for a storage medium containing computer-executable instructions provided by the embodiments of the present disclosure, the computer-executable instructions are not limited to the method operations described above, and can also execute related operations in the text classification method provided by any embodiment of the present disclosure.

[0175] From the above description of the embodiments, those skilled in the art can clearly understand that the present disclosure can be implemented by means of software and necessary general-purpose hardware. Of course, it can also be implemented by hardware, but in many cases the former is a better implementation. Based on such an understanding, the technical solution of the present disclosure, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as a floppy disk, a read-only memory (ROM), a random access memory (RAM), a flash memory (FLASH), a hard disk, or an optical disc of a computer, etc., including several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments of the present disclosure.

[0176] It should be noted that in the embodiments of the above search device, the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific names of the functional units are only for the convenience of mutual distinction and do not limit the protection scope of the present disclosure.

[0177] Note that the above is only the preferred embodiment of the present disclosure and the applied technical principle. Those skilled in the art will understand that the present disclosure is not limited to the specific embodiments described herein, and various obvious changes, re-adjustments, and substitutions can be made by those skilled in the art without departing from the protection scope of the present disclosure. Therefore, although the present disclosure has been described in detail through the above embodiments, the present disclosure is not limited to the above embodiments. Without departing from the concept of the present disclosure, more other equivalent embodiments can be included, and the scope of the present disclosure is determined by the scope of the appended claims.

Claims

1. A text classification method, characterized in that, Including: Determine the word vectors corresponding to each word in the text to be classified; Input the word vectors corresponding to the respective words into the neuron model of the spiking neural network for processing in sequence according to the text order; When the current word vector is the last word vector in the text to be classified, determine the classification result of the text to be classified according to the output result of the neuron model; Wherein, the neuron model determines a leakage coefficient according to the output result at the (i - 1)-th moment and the target weight matrix, and the leakage coefficient is used to determine the output result of the neuron model at the i-th moment; the target weight matrix is a preset weight matrix used for calculating the leakage coefficient in the text classification task; The output result at the (i - 1)-th moment includes the membrane potential information at the (i - 1)-th moment and the firing result at the (i - 1)-th moment, Wherein, determining the leakage coefficient according to the output result at the (i - 1)-th moment and the target weight matrix includes one of the following methods: Determine the leakage coefficient according to the membrane potential information at the (i - 1)-th moment and the target weight matrix; Determine the leakage coefficient according to the firing result at the (i - 1)-th moment and the target weight matrix; The target weight matrix includes a second weight matrix and a third weight matrix. Wherein, determining the leakage coefficient according to the membrane potential information at the (i - 1)-th moment and the target weight matrix includes: Determine a first result according to the word vector at the i-th moment and the second weight matrix; Determine a second result according to the membrane potential information at the (i - 1)-th moment and the third weight matrix; Determine the leakage coefficient according to the first result, the second result and the activation function.

2. The method according to claim 1, wherein the signal types of the word vectors include pulse signal types and analog signal types.

3. The method according to claim 1 or 2, characterized in that, The inputting the word vectors corresponding to the respective words into the neuron model of the spiking neural network for processing in sequence according to the text order includes: For the word vector at the i-th moment input into the neuron model, obtain the pre-trained first weight matrix and the membrane potential information at the (i - 1)-th moment; Determine the temporary membrane potential according to the word vector at the i-th moment, the first weight matrix and the membrane potential information at the (i - 1)-th moment; Determine the leakage coefficient according to the output result at the (i - 1)-th moment and the target weight matrix; Determine the firing result at the i-th moment and reset the membrane potential according to the temporary membrane potential and the preset threshold; Determine the membrane potential information at the i-th moment according to the reset membrane potential and the leakage coefficient.

4. The method according to claim 3, characterized in that, The determining the membrane potential information at the i-th moment according to the reset membrane potential and the leakage coefficient includes: Determine the first membrane potential information according to the reset membrane potential and the leakage coefficient; Determine the second membrane potential information according to the membrane potential information at the (i - 1)-th moment and the leakage coefficient; Determine the membrane potential information at the i-th moment according to the first membrane potential information and the second membrane potential information.

5. The method according to claim 3, characterized in that, The determining the firing result at the i-th moment according to the temporary membrane potential and the preset threshold includes one of the following methods: Determine the firing result at the i-th moment according to the temporary membrane potential, the preset threshold and the analog activation function, where the firing result is an analog value; If the temporary membrane potential is greater than or equal to the preset threshold, determine that the firing result is 1; if the temporary membrane potential is less than the preset threshold, determine that the firing result is 0, where the firing result is a pulse value.

6. The method according to claim 3, characterized in that, Determining the reset membrane potential at the i-th moment according to the temporary membrane potential and the preset threshold includes: If the temporary membrane potential is greater than or equal to the preset threshold, determine the preset resting potential as the reset membrane potential; If the temporary membrane potential is less than the preset threshold, determine the temporary membrane potential as the reset membrane potential.

7. The method according to claim 3, characterized in that, Determining the temporary membrane potential according to the word vector at the i-th moment, the first weight matrix, and the membrane potential information at the (i - 1)-th moment includes: Performing an integration operation according to the word vector at the i-th moment and the first weight matrix to obtain an intermediate vector; Adding the intermediate vector and the membrane potential information at the (i - 1)-th moment to obtain the temporary membrane potential.

8. A text classification device, characterized in that, Includes: A word vector determination module for determining word vectors corresponding to each word in the text to be classified; A word vector processing module for sequentially inputting the word vectors corresponding to the respective words into a neuron model of a pulsed neural network for processing; A classification result determination module for determining the classification result of the text to be classified according to the output result of the neuron model when the current word vector is the last word vector in the text to be classified; Wherein, the neuron model determines a leakage coefficient according to the output result at the (i - 1)-th moment and the target weight matrix, and the leakage coefficient is used to determine the output result of the neuron model at the i-th moment; the target weight matrix is a preset weight matrix used for calculating the leakage coefficient in a text classification task; The output result at the (i - 1)-th moment includes the membrane potential information at the (i - 1)-th moment and the firing result at the (i - 1)-th moment, Wherein, determining the leakage coefficient according to the output result at the (i - 1)-th moment and the target weight matrix includes one of the following methods: Determine the leakage coefficient according to the membrane potential information at the (i - 1)-th moment and the target weight matrix; Determine the leakage coefficient according to the firing result at the (i - 1)-th moment and the target weight matrix; The target weight matrix includes a second weight matrix and a third weight matrix, wherein determining the leakage coefficient according to the membrane potential information at the (i - 1)-th moment and the target weight matrix includes: Determine a first result according to the word vector at the i-th moment and the second weight matrix; Determine a second result according to the membrane potential information at the (i - 1)-th moment and the third weight matrix; Determine the leakage coefficient according to the first result, the second result, and an activation function.

9. A device, characterized in that, The device includes: One or more processors; A storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the text classification method according to any one of claims 1 - 7.

10. A storage medium containing computer-executable instructions, characterized in that, The computer-executable instructions are used to execute the text classification method according to any one of claims 1 - 7 when executed by a computer processor.

Images