Text classification method and storage medium

Abstract

The invention discloses a text classification method and a storage medium, and the method comprises the steps: obtaining a data set constructed by a to-be-classified text, constructing a graph structure based on the data set, and obtaining an adjacent matrix generated by the graph structure; pre-training nodes in the graph structure to obtain a node vector adapted to the graph structure; inputting the feature matrix of the node vector and the adjacent matrix into a graph convolutional network, and classifying the text; wherein the feature matrix of the node vector and the adjacent matrix are input into the graph convolutional network, and the text is classified, which comprises the following steps: taking the feature matrix as input, and in each layer of the graph convolutional network, using the input feature matrix of the previous layer to obtain the hidden layer representation of the next layer; and mapping the output of the last layer of the graph convolutional network to a result space through a feedforward network to obtain the final output of each node. According to the method, the prediction capability of the unseen nodes is improved, and the reasoning efficiency on a large-scale graph data set is improved.

Description

Technical Field

[0001] The present application relates to the technical field of natural language processing, and more specifically to a text classification method and storage medium. Background Art

[0002] Diabetes is a chronic disease with a long treatment cycle and is prone to cause various complications such as kidney disease and eye disease. Doctors need to be able to quickly and accurately determine the type of disease, analyze the trend of disease changes, and provide personalized treatment plans and preventive measures for each patient. In the study of diabetes and its complications, single-label text classification algorithms are mainly used for the classification and prediction of patient data to help doctors make diagnoses and formulate treatment plans.

[0003] These methods can meet the needs of text classification tasks to a certain extent, but they have certain limitations when dealing with complex semantic relationships and long text sequences. Traditional graph-based methods cannot predict text that did not appear during training, and require a large number of matrix operations and adjacency matrix updates during each inference. There are many repeated calculations in the inference stage, which is time-consuming and inefficient, and is not practical in actual applications. Summary of the invention

[0004] In order to solve the above problems, the present application is proposed. According to one aspect of the present application, a text classification method is provided, characterized in that the method comprises:

[0005] Obtain a data set constructed by the text to be classified and construct a graph structure based on the data set, and obtain an adjacency matrix generated by the graph structure;

[0006] Pre-training the nodes in the graph structure to obtain node vectors adapted to the graph structure;

[0007] Inputting the feature matrix of the node vector and the adjacency matrix into a graph convolutional network to classify the text;

[0008] The step of inputting the feature matrix of the node vector and the adjacency matrix into a graph convolutional network to classify the text includes:

[0009] Taking the feature matrix as input, in each layer of the graph convolutional network, the input feature matrix of the previous layer is used to obtain the hidden layer representation of the next layer;

[0010] The output of the last layer of the graph convolutional network is mapped to the result space through a feedforward network to obtain the final output of each node.

[0011] In one embodiment of the present application, the nodes include text nodes, word nodes, and label nodes; and the pre-training of the nodes in the graph structure to obtain node vectors adapted to the graph structure includes:

[0012] Inputting the nodes in the graph structure into a long short-term memory network for pre-training to obtain a set of initialized text vectors, word vectors, and label vectors that are adapted to the graph structure;

[0013] The input of the LSTM network is a text node X = {x1, x2, ..., x n}, the output is the hidden layer output H = {h1,h2,...,h n};

[0014] The obtained hidden layer output sequence is transformed into a vector to obtain the text vector H (0) ∈R n×k , the k-dimensional feature matrix of n nodes;

[0015] Use attention mechanisms to obtain a high-level representation of the information from each hidden layer;

[0016] The word nodes and label nodes are initialized using trainable parameters to obtain the word vectors and label vectors.

[0017] In one embodiment of the present application, the step of using the input feature matrix of the previous layer to obtain the hidden layer representation of the next layer includes:

[0018] Regularizing the adjacency matrix;

[0019] In each layer of the graph convolutional network, the input feature matrix of the previous layer and the regularized adjacency matrix are used to obtain the hidden layer representation of the next layer;

[0020] The calculation formula is:

[0021] H (j+1) =ReLU(AH (j) W j +b j )

[0022] Among them, ReLU is the activation function, A is the regularized adjacency matrix A, H (j) is the input feature of the previous layer of the graph convolutional network, j is the number of graph convolutional network layers, and W j is the weight matrix of this layer, b j is the bias vector of the corresponding layer.

[0023] In one embodiment of the present application, the output of the last layer of the graph convolutional network is mapped to the result space through a feedforward network to obtain the final output of each node, including:

[0024] The output of the last layer of the graph convolutional network is passed through an artificial neural network, where the result is mapped to the range of 0-1, and the calculation formula is:

[0025] Z = sigmoid(f out (H last ))

[0026] Among them, the sigmoid function is the activation function of the output layer, f out is an artificial neural network, H last is the output of the last layer of the graph convolutional network, and Z is the final output of each node.

[0027] In one embodiment of the present application, the feature matrix of the node vector and the adjacency matrix are input into a graph convolutional network to classify the text, further comprising:

[0028] Use a two-layer graph convolutional network, each layer of which is followed by an activation function ReLU;

[0029] The input of the first layer of the graph convolutional network is the hidden layer output of the long short-term memory network.

[0030] In one embodiment of the present application, the method further includes training the graph convolutional network, using the training text nodes and label nodes to calculate the loss, and the total loss is calculated as:

[0031] Loss=Loss positive +λ*Loss negative

[0032] Among them, Loss positive is the average prediction error of samples with label 1, Loss negative is the average prediction error of samples with label 0, and λ is a hyperparameter.

[0033]

[0034] Among them, N is the total number of samples, i is the sample, and Z i is the label prediction value, y i is the actual value of the label, is the indicator vector, is the mean square error of the ith sample.

[0035]

[0036] in, yes The complement of .

[0037]

[0038] Among them, y ik is the i-th sample in the k-dimensional hidden layer.

[0039] In one embodiment of the present application, a data set constructed by texts to be classified is obtained, a graph structure is constructed based on the data set, and an adjacency matrix generated by the graph structure is obtained, including:

[0040] Based on the data set, construct a graph containing texts, words and labels, where each node in the graph represents a unique text, word or label;

[0041] The adjacency matrix generated by the graph structure is A. Since each node is connected to itself, the diagonal elements of the adjacency matrix are set to 1.

[0042] In one embodiment of the present application, for the label node, the label of each training data is regarded as a word in the text and is calculated in the same way as the word node.

[0043] In one embodiment of the present application, the method further includes:

[0044] During the test, the initial features of the test text are the same as those of the LSTM network during training. After obtaining the regularized adjacency matrix and the initial features, the weights of each graph convolution layer and the word node weight parameters retained during the training process are taken out, and the word vector and the text vector encoded by the LSTM network are concatenated as the input of the graph convolution network.

[0045] The test text first passes through multiple layers of graph convolution and then passes through the output layer to get the result.

[0046] According to another aspect of the present application, a storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the processor executes the above-mentioned text classification method.

[0047] The text classification method of the present invention uses the input feature matrix of the previous layer to obtain the hidden layer representation of the next layer in each layer of the graph convolutional network, which helps to save the intermediate calculation results. Combined with the method of pre-training the nodes in the graph structure, it not only improves the prediction ability of unseen nodes, but also reduces repeated calculations in the reasoning stage, and achieves significant advantages in reasoning efficiency on large-scale graph data sets, making this method very practical in practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0049] Figure 1 A schematic flow chart of a text classification method according to an embodiment of the present application is shown;

[0050] Figure 2 A schematic flowchart of sub-steps of a text classification method according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solutions and advantages of the present application more obvious, the example embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the example embodiments described herein. Based on the embodiments of the present application described in the present application, all other embodiments obtained by those skilled in the art without creative work should fall within the scope of protection of the present application.

[0052] First, refer to Figure 1 and Figure 2 To describe the text classification method used to implement the embodiment of the present invention. Figure 1 FIG. 4 shows a schematic flow chart of a text classification method according to an embodiment of the present application. Figure 1 As shown, the text classification method according to the embodiment of the present application may include the following steps:

[0053] In step S100, a data set constructed by the text to be classified is obtained and a graph structure is constructed based on the data set, and an adjacency matrix generated by the graph structure is obtained.

[0054] In step S200, pre-training is performed on the nodes in the graph structure to obtain node vectors adapted to the graph structure;

[0055] In step S300, the feature matrix of the node vector and the adjacency matrix are input into a graph convolutional network to classify the text.

[0056] Figure 2 FIG. 1 shows a schematic flow chart of sub-steps of a text classification method according to an embodiment of the present application. Figure 2 As shown, the sub-steps of the text classification method according to the embodiment of the present application may include the following steps:

[0057] In step S310, the feature matrix is ​​used as input, and in each layer of the graph convolutional network, the input feature matrix of the previous layer is used to obtain the hidden layer representation of the next layer;

[0058] In step S320, the output of the last layer of the graph convolutional network is mapped to the result space through a feedforward network to obtain the final output of each node.

[0059] The text classification method of the present invention uses the input feature matrix of the previous layer to obtain the hidden layer representation of the next layer in each layer of the graph convolutional network, which helps to save the intermediate calculation results. Combined with the method of pre-training the nodes in the graph structure, it not only improves the prediction ability of unseen nodes, but also reduces repeated calculations in the reasoning stage, and achieves significant advantages in reasoning efficiency on large-scale graph data sets, making this method very practical in practical applications.

[0060] In the embodiment of the present application, in step S100, a data set constructed by the text to be classified is obtained and a graph structure is constructed based on the data set to obtain an adjacency matrix generated by the graph structure. By constructing a global graph including texts, words and labels, the associations between similar texts can be fully mined.

[0061] First, the text to be classified is obtained and preprocessed, including: extracting information from the text to be classified to obtain key data; merging the key data by patient and by time; processing the key data for null values ​​and arranging the key data in time series.

[0062] Then, a data set is constructed based on the preprocessed text to be classified, including: dividing the data set into a training data set, a verification data set and a test data set in proportion; adding a label column to the data set to mark the data as belonging to the training data set, the verification data set or the test data set.

[0063] In the information extraction step, key feature columns such as patient ID, age, doctor's initial diagnosis and admission diagnosis are identified and selected from the original data through manual screening combined with the functions of the pandas library, and necessary date calculations and format standardization are performed. The information merging step involves aggregating multiple medical records of the same patient and arranging them in chronological order to form a complete medical record. In the process of processing null values ​​in the data, a time series sorting method is used to fill and correct missing values ​​in the data to ensure the integrity and consistency of the data. Finally, in the data set partitioning step, low-frequency labels are removed to reduce the interference of noise on model training, and the data set is divided into training set, validation set and test set according to the established ratio. This series of operations provides an accurate, orderly and practical data set for subsequent data analysis.

[0064] In one embodiment, a data set constructed by the text to be classified is obtained and a graph structure is constructed based on the data set, and an adjacency matrix generated by the graph structure is obtained, including: based on the data set, a graph containing text, words and labels is constructed, in which each node represents a unique text, word or label. The adjacency matrix generated by the graph structure is A. Since each node is connected to itself, the diagonal elements of the adjacency matrix are set to 1. Consider a graph G = {V, E}, where V and E are node sets and edge sets, respectively. There are three types of edges in the constructed graph, including doc-word edges, word-word edges, and word-label edges. Compared with other global graph methods, additional label nodes are introduced when constructing the global graph. Experiments have found that after deleting the label nodes, various indicators on the data set have decreased to varying degrees, indicating that the co-occurrence relationship between labels plays an auxiliary role in the node information propagation of the global graph network.

[0065] In one embodiment, for a label node, the label of each training data is regarded as a word in the text and is calculated in the same way as a word node.

[0066] In the embodiment of the present application, the nodes in the graph structure are pre-trained in step S200 to obtain node vectors adapted to the graph structure to initialize the nodes in the graph and extract the semantic features of the text.

[0067] In one embodiment, the nodes include text nodes, word nodes, and label nodes; pre-training the nodes in the graph structure to obtain node vectors adapted to the graph structure includes: inputting the nodes in the graph structure into a long short-term memory network (LSTM) for pre-training to obtain a set of initialized text vectors, word vectors, and label vectors adapted to the graph structure. The input of the LSTM network is a text node X = {x1, x2, ..., x n}, the output is the hidden layer output H = {h1,h2,...,h n Because the sequence obtained by the hidden layer is not suitable for being initialized as a matrix of nodes, the hidden layer output sequence is transformed into a vector to obtain the text vector H (0) ∈R n×k, a k-dimensional feature matrix of n nodes. The attention mechanism is used to obtain a high-level representation of the information from each hidden layer. The word nodes and label nodes are initialized using trainable parameters, and their representations are continuously adjusted during the training process to obtain the word vectors and label vectors. As in the case of graph composition, label nodes are also regarded as words here, so the processing method is the same as that of ordinary words. In this way, a set of initialized text, word and label vectors that are adapted to the graph structure are obtained, and are sent to the graph convolutional network (Graph Convolutional Networks, GCN) for classification together with the adjacency matrix generated by the previous graph construction. Pre-trained node vectors can effectively reduce gradient calculations and enhance the contextual representation of text nodes, thereby facilitating node initialization.

[0068] In one embodiment, using One-Hot Encoding to convert the category labels into a numerical form acceptable to the model is a critical step. One-Hot Encoding is a widely used method that digitizes category information by creating a binary vector to represent each category. Following this rule, replacing the actual category labels in the dataset with the corresponding One-Hot Encoded vector, we can generate the following One-Hot Encoded vector:

[0069] ['Sitagliptin Phosphate Tablets','(Guocai) Metformin Hydrochloride Sustained Release Tablets']=[0,1,0,0,1,...]

[0070] ['Epalrestat Tablets','Ginkgo Biloba Ester Dropping Pills','(Guo Tan) Proline Henggliflozin']=[1,0,1,1,0,...]

[0071] The node vector and one-hot label encoding after long short-term memory network encoding not only improve the model's ability to process and understand diabetic patients' text data in the diabetes text classification task, but also provide a solid foundation for model training and prediction.

[0072] In an embodiment of the present application, in step S300, the feature matrix and adjacency matrix of the node vector are input into the graph convolution network to classify the text. Using the graph convolution network to classify text nodes can well fuse node information. Compared with other graph neural networks, the graph convolution network can better consider the adjacency weights and fuse global information. Therefore, a graph convolution network is used for information fusion. The graph convolution network is a multi-layer neural network that fuses different node embeddings based on the weight information of the edge attributes in the graph structure. The adjacency matrix and initialization vector of the obtained graph structure are input into the graph convolution network for classification.

[0073] In an embodiment of the present application, the feature matrix is ​​used as input in step S310, and in each layer of the graph convolutional network, the input feature matrix of the previous layer is used to obtain the hidden layer representation of the next layer. Regularizing the adjacency matrix and storing the hidden layer representation of the word node solves the problem that the traditional graph-based method cannot predict text that did not appear during training.

[0074] In one embodiment, the input feature matrix of the previous layer is used to obtain the hidden layer representation of the next layer, including: regularizing the adjacency matrix. In each layer of the graph convolutional network, the input feature matrix of the previous layer and the regularized adjacency matrix are used to obtain the hidden layer representation of the next layer, and the calculation formula is:

[0075] H (j+1) =ReLU(AH (j) W j +b j )

[0076] Among them, ReLU is the activation function, A is the regularized adjacency matrix A, H (j) is the input feature of the previous layer of the graph convolutional network, j is the number of graph convolutional network layers, and W j is the weight matrix of this layer, b j is the bias vector of the corresponding layer.

[0077] In an embodiment of the present application, in step S320, the output of the last layer of the graph convolutional network is mapped to the result space through a feedforward network to obtain the final output of each node.

[0078] In the embodiment of the present application, the output of the last layer of the graph convolutional network is mapped to the result space through a feedforward network to obtain the final output of each node, including:

[0079] The output of the last layer of the graph convolutional network is passed through an artificial neural network, where the result is mapped to the range of 0-1, and the calculation formula is:

[0080] Z = sigmoid(f out (H last ))

[0081] Among them, the sigmoid function is the activation function of the output layer, f out is an artificial neural network, H last is the output of the last layer of the graph convolutional network, and Z is the final output of each node.

[0082] In one embodiment, the feature matrix and adjacency matrix of the node vector are input into a graph convolution network to classify the text, and the method further includes: using a two-layer graph convolution network, each layer of which is followed by an activation function ReLU; the first layer of the graph convolution network input is the hidden layer output of the long short-term memory network. In a single-layer graph convolution network, the representation update of each node is only affected by its neighboring nodes and itself, while when a multi-layer graph convolution network is used, the receptive field can be wider. The hidden layer output size is 300, and the hidden layer output size of the second layer GCN is 150.

[0083] In an embodiment of the present application, the method further includes training the graph convolutional network, using the training text nodes and label nodes to calculate the loss, and the total loss is calculated as:

[0084] Loss=Loss positive +λ*Loss negative

[0085] Among them, Loss positive is the average prediction error of samples with label 1, Loss negative is the average prediction error of samples with label 0, and λ is a hyperparameter.

[0086]

[0087] Among them, N is the total number of samples, i is the sample, and Z i is the label prediction value, y i is the actual value of the label, is the indicator vector, is the mean square error of the ith sample.

[0088]

[0089] in, yes The complement of .

[0090]

[0091] Among them, y ik is the i-th sample in the k-dimensional hidden layer.

[0092] In one embodiment, after preprocessing the dataset, the network is implemented in the Pytorch framework and trained using a single Nvidia GPU RTX 3090Ti (24G physical memory).

[0093] The encoder based on long short-term memory network is used to process text data: a bidirectional LSTM layer is used, each word is mapped to a word embedding vector of size 300, and the hidden layer size of LSTM is set to 150. The weights of the word embedding layer are initialized to uniform distribution, the weights and biases of the LSTM layer are initialized using Xavier, the training learning rate of the model is set to 5e-4, and the optimizer uses the AdamW optimizer, whose parameters include weight_decay=0.1. Fine-tuning is performed on the LSTM layer, and the hidden layer output is added to the attention mechanism to obtain a better deep representation of the information from each hidden layer. The binary cross entropy loss function (BCELoss) is used during training, and the early stopping strategy is used. If the validation set loss does not decrease in 10 consecutive epochs, the training is stopped. Each epoch performs a complete forward and backward propagation on the entire training data set. Gradient clipping is used to prevent gradient explosion, and the clipping threshold is set to 5.0. Training is performed according to the predetermined parameters, and the optimal parameters of the model are saved according to the optimal model strategy for subsequent network use.

[0094] In the graph convolutional network, two layers of GCN are used, each layer is followed by a ReLU activation function. The input of the first layer of GCN is the hidden layer output of the LSTM encoder, and the hidden layer output size is 300. The hidden layer output size of the second layer of GCN is 150. The weight of the GCN layer is initialized using Xavier, the learning rate is set to 8e-3, and the hyperparameter λ is set to 0.5 to weigh the weight relationship between the positive sample loss and the negative sample loss of the GCN part. The training is performed for 1500 epochs and optimized using the Adam optimizer. The loss function is found to be better than other functions through experiments. The rest of the training parameter settings remain the same as the LSTM settings.

[0095] In an embodiment of the present application, during the test process, the initial features of the test text are the same as the long short-term memory network during training. After obtaining the regularized adjacency matrix and the initial features, the weights of each graph convolution layer and the word node weight parameters retained during the training process are taken out, and the word vector is concatenated with the text vector encoded by the long short-term memory network as the input of the graph convolution network. The test text first passes through multiple layers of graph convolution, and then the result is obtained through the output layer.

[0096] The text classification method of the present invention can fully explore the association between similar texts by constructing a global graph containing texts, words and labels. By using the input feature matrix of the previous layer to obtain the hidden layer representation of the next layer in each layer of the graph convolutional network, it is helpful to save the intermediate calculation results. Combined with the method of pre-training the nodes in the graph structure, the nodes in the graph are initialized, the semantic features of the text are extracted, and the graph convolutional network is used for information fusion, which solves the problem that traditional methods cannot handle text that does not appear during training. Not only does it improve the prediction ability of unseen nodes, it also reduces repeated calculations in the reasoning stage, and has achieved significant advantages in reasoning efficiency on large-scale graph data sets, making this method very practical in practical applications.

[0097] In addition, the present application also provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the processor executes the text classification method according to the embodiment of the present application described above. The storage medium may include, for example, a memory card of a smart phone, a storage component of a tablet computer, a hard disk of a personal computer, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disk read-only memory (CD-ROM), a USB memory, or any combination of the above storage media. The computer-readable storage medium may be any combination of one or more computer-readable storage media.

[0098] Although example embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above example embodiments are merely exemplary and are not intended to limit the scope of the present application to this. Those of ordinary skill in the art may make various changes and modifications therein without departing from the scope and spirit of the present application. All these changes and modifications are intended to be included within the scope of the present application as required by the appended claims.

[0099] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0100] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed.

[0101] In the description provided herein, a large number of specific details are described. However, it is understood that the embodiments of the present application can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0102] Similarly, it should be understood that in order to streamline the present application and help understand one or more of the various inventive aspects, in the description of the exemplary embodiments of the present application, the various features of the present application are sometimes grouped together into a single embodiment, figure, or description thereof. However, the method of the present application should not be interpreted as reflecting the following intention: the claimed application requires more features than the features clearly stated in each claim. More specifically, as reflected in the corresponding claims, the inventive point is that the corresponding technical problem can be solved with features less than all the features of a single disclosed embodiment. Therefore, the claims following the specific embodiment are hereby explicitly incorporated into the specific embodiment, wherein each claim itself serves as a separate embodiment of the present application.

[0103] It will be understood by those skilled in the art that, except for mutually exclusive features, all features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed in this specification may be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature that provides the same, equivalent or similar purpose.

[0104] In addition, those skilled in the art will appreciate that, although some embodiments described herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of the present application and form different embodiments. For example, in the claims, any one of the claimed embodiments may be used in any combination.

[0105] The various component embodiments of the present application can be implemented in hardware, or implemented in software modules running on one or more processors, or implemented in a combination thereof. It should be understood by those skilled in the art that a microprocessor or digital signal processor (DSP) can be used in practice to implement some or all functions of some modules according to the embodiments of the present application. The application can also be implemented as a part or all of a program (e.g., a computer program and a computer program product) for performing the method described herein. Such a program implementing the present application can be stored on a computer-readable medium, or can have the form of one or more signals. Such a signal can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0106] It should be noted that the above embodiments illustrate the present application rather than limit the present application, and that those skilled in the art may design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets shall not be construed as a limitation on the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present application may be implemented with the aid of hardware comprising a number of different elements and with the aid of a suitably programmed computer. In a unit claim listing several abnormality detection devices for train traction systems, several of these abnormality detection devices for train traction systems may be embodied by the same hardware item. The use of the words first, second, and third, etc., does not indicate any order. These words may be interpreted as names.

[0107] The above is only a specific implementation or description of a specific implementation of the present application, and the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. The protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A text classification method, characterized in that: The method comprises: Obtain a data set constructed by the text to be classified and construct a graph structure based on the data set, and obtain an adjacency matrix generated by the graph structure; Pre-training the nodes in the graph structure to obtain node vectors adapted to the graph structure; Inputting the feature matrix of the node vector and the adjacency matrix into a graph convolutional network to classify the text; The step of inputting the feature matrix of the node vector and the adjacency matrix into a graph convolutional network to classify the text includes: Taking the feature matrix as input, in each layer of the graph convolutional network, the input feature matrix of the previous layer is used to obtain the hidden layer representation of the next layer; The output of the last layer of the graph convolutional network is mapped to the result space through a feedforward network to obtain the final output of each node.

2. The method according to claim 1, characterized in that The nodes include text nodes, word nodes, and label nodes; the pre-training of the nodes in the graph structure to obtain node vectors adapted to the graph structure includes: Inputting the nodes in the graph structure into a long short-term memory network for pre-training to obtain a set of initialized text vectors, word vectors, and label vectors that are adapted to the graph structure; The input of the LSTM network is a text node X = {x1, x2, ..., x n }, the output is the hidden layer output H = {h1,h2,...,h n }; The obtained hidden layer output sequence is transformed into a vector to obtain the text vector H (0) ∈R n×k , the k-dimensional feature matrix of n nodes; Use attention mechanisms to obtain a high-level representation of the information from each hidden layer; The word nodes and label nodes are initialized using trainable parameters to obtain the word vectors and label vectors.

3. The method according to claim 1, characterized in that The method of using the input feature matrix of the previous layer to obtain the hidden layer representation of the next layer includes: Regularizing the adjacency matrix; In each layer of the graph convolutional network, the input feature matrix of the previous layer and the regularized adjacency matrix are used to obtain the hidden layer representation of the next layer; The calculation formula is: A (j+1) =ReLU(AH (j) W j +b j ) Among them, ReLU is the activation function, A is the regularized adjacency matrix A, H (j) is the input feature of the previous layer of the graph convolutional network, j is the number of graph convolutional network layers, and W j is the weight matrix of this layer, b j is the bias vector of the corresponding layer.

4. The method according to claim 1, characterized in that The output of the last layer of the graph convolutional network is mapped to the result space through the feedforward network to obtain the final output of each node, including: The output of the last layer of the graph convolutional network is passed through an artificial neural network, where the result is mapped to the range of 0-1, and the calculation formula is: Z=sigmoid(f out (H last )) Among them, the sigmoid function is the activation function of the output layer, f out is an artificial neural network, H last is the output of the last layer of the graph convolutional network, and Z is the final output of each node.

5. The method according to claim 1, characterized in that Inputting the feature matrix of the node vector and the adjacency matrix into a graph convolutional network to classify the text, further comprising: Use a two-layer graph convolutional network, each layer of which is followed by an activation function ReLU; The input of the first layer of the graph convolutional network is the hidden layer output of the long short-term memory network.

6. The method according to claim 1, characterized in that The method further includes training the graph convolutional network, using the training text nodes and the label nodes to calculate the loss, and the total loss is calculated as: Loss=Loss positive +λ*Loss negative Among them, Loss positive is the average prediction error of samples with label 1, Loss negative is the average prediction error of samples with label 0, and λ is a hyperparameter. Among them, N is the total number of samples, i is the sample, and Z i is the label prediction value, y i is the actual value of the label, is the indicator vector, is the mean square error of the ith sample. in, yes The complement of . Among them, y ik is the i-th sample in the k-dimensional hidden layer.

7. The method according to claim 1, characterized in that Obtaining a data set constructed by the text to be classified and constructing a graph structure based on the data set, and obtaining an adjacency matrix generated by the graph structure, including: Based on the data set, construct a graph containing texts, words and labels, where each node in the graph represents a unique text, word or label; The adjacency matrix generated by the graph structure is A. Since each node is connected to itself, the diagonal elements of the adjacency matrix are set to 1.

8. The method according to claim 7, characterized in that For the label node, the label of each training data is regarded as a word in the text and is calculated in the same way as the word node.

9. The method according to claim 1, characterized in that The method further comprises: During the test, the initial features of the test text are the same as those of the LSTM network during training. After obtaining the regularized adjacency matrix and the initial features, the weights of each graph convolution layer and the word node weight parameters retained during the training process are taken out, and the word vector and the text vector encoded by the LSTM network are concatenated as the input of the graph convolution network. The test text first passes through multiple layers of graph convolution and then passes through the output layer to get the result.

10. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is run, the text classification method according to any one of claims 1 to 9 is executed.

Images