Text classification method, device, equipment and medium

Abstract

The present invention relates to artificial intelligence technology in the medical field and discloses a text classification method, comprising: performing a vectorization operation on an acquired training text set; performing a first feature extraction on the obtained training text vector set using multiple conversion layers of a text classification model; performing a second feature extraction on the obtained first training text feature set using multiple side layers of the model; performing text classification on the obtained second training text feature set using multiple classification layers of the model to obtain a predicted text classification result; calculating a loss value for the predicted text classification result, and adjusting parameters in the text classification model according to the loss value until the loss value meets a preset threshold, thereby obtaining a trained text classification model; and finally using the trained text classification model to classify the text set to be classified to obtain a classification result. The present invention also provides a text classification device, equipment, and medium. The present invention can improve the accuracy of text classification.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to a text classification method, apparatus, device and storage medium. Background Art

[0002] Text classification refers to the automatic classification and labeling of text collections according to a specific classification system or standard. For example, in the medical industry, medical disease information can be classified using text classification models. Existing text classification methods primarily classify text collections using text classification models. Within the internal structure of these text classification models, a classification layer is added to each transformation layer to determine whether the model's prediction performance meets the model training exit criteria. If the model meets the training exit criteria, training is terminated, resulting in a fully trained text classification model.

[0003] However, the existing methods require joint training of each classification layer of the text classification model. Therefore, when there is an accuracy conflict between the current classification layer and the next classification layer of the text classification model, the accuracy of the text classification model will be reduced, resulting in a low accuracy of text classification. Summary of the Invention

[0004] The present invention provides a text classification method, apparatus, device and storage medium, the main purpose of which is to improve the accuracy of text classification.

[0005] To achieve the above object, the present invention provides a text classification method, comprising:

[0006] Obtaining a training text set and a real text classification result set corresponding to the training text set, and performing a vectorization operation on the training text set to obtain a training text vector set;

[0007] Obtaining a preset text classification model, wherein the preset text classification model includes multiple conversion layers, multiple side branches, and multiple classification layers;

[0008] Performing first feature extraction on the training text vector set using the plurality of conversion layers to obtain a first training text feature set;

[0009] Performing second feature extraction on the first training text feature set using the plurality of side-branch layers to obtain a second training text feature set;

[0010] Performing text classification on the second training text feature set using the plurality of classification layers to obtain a predicted text classification result of the second training text feature set;

[0011] Calculating the loss value of the predicted text classification result according to the real text classification result set and the preset loss function, and adjusting the parameters in the preset text classification model according to the loss value until the loss value meets the preset threshold, thereby obtaining a trained text classification model;

[0012] A text set to be classified is obtained, and the text set to be classified is classified using the trained text classification model to obtain a target text classification result.

[0013] Optionally, the extracting a second feature from the first training text feature set using the plurality of side-branch layers to obtain a second training text feature set includes:

[0014] Using the lower projection layers in the plurality of side-branch layers to perform a dimensionality reduction operation on the first training text feature set to obtain a first training text feature matrix;

[0015] Using the upper projection layer in the plurality of the side-branch layers to restore the dimension of the first training text feature matrix to obtain a second training text feature matrix;

[0016] The first training text feature matrix and the second training text feature matrix are activated by using the nonlinear layers in the plurality of side-branch layers to obtain the second training text feature set.

[0017] Optionally, performing text classification on the second training text feature set using the plurality of classification layers to obtain a predicted text classification result of the second training text feature set includes:

[0018] Using the fully connected layers in the plurality of classification layers to respectively perform decoding operations on the second training text feature set to obtain a decoded training text feature set corresponding to each of the fully connected layers;

[0019] Outputting a plurality of initial predicted text classification results of the plurality of decoded training text feature sets at each of the classification layers using activation functions in the plurality of classification layers;

[0020] Calculating the similarity between the initial predicted text classification results corresponding to any two of the classification layers respectively;

[0021] When the similarity meets the preset similarity, the classification of the second training text feature set is stopped, and the initial predicted text classification result obtained by the classification layer when the classification is stopped is used as the predicted text classification result of the second training text feature set.

[0022] Optionally, respectively calculating the similarity between the initial predicted text classification results corresponding to any two classification layers includes:

[0023] Generating a predicted text category histogram for the initial predicted text classification results corresponding to any two of the classification layers;

[0024] Performing normalization calculation on the predicted text category histogram to obtain a normalized histogram;

[0025] The normalized histogram is calculated to obtain the similarity between the initial predicted text classification results corresponding to any two of the classification layers.

[0026] Optionally, the extracting a first feature from the training text vector set using the plurality of conversion layers to obtain a first training text feature set includes:

[0027] Performing position encoding on the training text vector set using a multi-head attention mechanism in a plurality of the conversion layers to obtain a text position vector set of the training text set;

[0028] Performing linear transformation on the text position vector set using a plurality of preset transformation parameters to obtain a plurality of text transformation vectors;

[0029] A decoding operation is performed on the multiple text conversion vectors to obtain the first training text feature set of each of the conversion layers.

[0030] Optionally, performing a vectorization operation on the training text set to obtain a training text vector set includes:

[0031] Performing a vectorization operation on the training text set using a pre-built vector conversion model to generate multiple word vectors;

[0032] Pre-training the plurality of word vectors to obtain a plurality of pre-trained word vectors;

[0033] The words in the training text set are compared with the plurality of pre-trained word vectors to obtain the training text vector set.

[0034] In order to solve the above problems, the present invention further provides a text classification device, comprising:

[0035] A first acquisition module is used to obtain a training text set and a real text classification result set corresponding to the training text set, and perform a vectorization operation on the training text set to obtain a training text vector set;

[0036] A second acquisition module is used to acquire a preset text classification model, wherein the preset text classification model includes multiple conversion layers, multiple side-branch layers, and multiple classification layers;

[0037] A first feature extraction module, configured to perform first feature extraction on the training text vector set using the plurality of conversion layers to obtain a first training text feature set;

[0038] A second feature extraction module is used to perform second feature extraction on the first training text feature set using the plurality of side-branch layers to obtain a second training text feature set;

[0039] a pre-classification module, configured to perform text classification on the second training text feature set using the plurality of classification layers to obtain a predicted text classification result of the second training text feature set;

[0040] A model training module is used to calculate the loss value of the predicted text classification result based on the real text classification result set and a preset loss function, and adjust the parameters in the preset text classification model according to the loss value until the loss value meets a preset threshold, thereby obtaining a trained text classification model;

[0041] The text classification module is used to obtain a text set to be classified, classify the text set to be classified using the trained text classification model, and obtain a target text classification result.

[0042] In order to solve the above problem, the present invention further provides an electronic device, comprising:

[0043] a memory storing at least one computer program; and

[0044] The processor executes the computer program stored in the memory to implement the above-mentioned text classification method.

[0045] In order to solve the above problem, the present invention also provides a computer-readable storage medium, in which at least one computer program is stored. The at least one computer program is executed by a processor in an electronic device to implement the above text classification method.

[0046] In an embodiment of the present invention, after obtaining a training text set and a real text classification result set corresponding to the training text set, the training text set is first vectorized to facilitate subsequent feature extraction of the training text set, and a plurality of conversion layers in a preset text classification model are used to perform first feature extraction on the training text vector set, so that the features of the training text vector set can be accurately extracted, which is convenient for subsequent improvement of the accuracy of text classification; secondly, a plurality of side branches in the text classification model are used to perform second feature extraction on the first training text feature set to obtain a second training text feature set, and the dimensionality reduction parameters and restoration parameters of the side branches can be used to perform dimensionality reduction and restoration operations on the first training text feature set, so as to achieve the fixing of the model parameters upstream of the side branches during model training, and the fine-tuning of the downstream model parameters through the side branches, that is, the side branches can pass the parameters to the next conversion layer. The data and the data passed to the subsequent classification layer are trained separately, which is convenient for improving the accuracy of the current classification layer and the subsequent classification layer of the model. Without introducing a large number of parameters, the accuracy of the model prediction can be improved, thereby improving the accuracy of subsequent text classification. Finally, by using multiple classification layers in the text classification model to perform text classification on the second training text feature set, a predicted text classification result of the second training text feature set is obtained, which can ensure the accuracy of each classification layer in the text classification model, thereby improving the accuracy of text classification, and adjusting the parameters in the preset text classification model according to the loss value until the loss value meets the preset threshold, thereby obtaining a trained text classification model to improve the classification accuracy of the text classification model, and using the trained text classification model to classify the text set to be classified to obtain the target text classification result, which can improve the accuracy of text classification. Therefore, the text classification method, device, equipment and storage medium proposed in the embodiment of the present invention can improve the accuracy of text classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 A flowchart of a text classification method provided by one embodiment of the present invention;

[0048] Figure 2 A detailed flowchart of a step in a text classification method provided by one embodiment of the present invention;

[0049] Figure 3 A detailed flowchart of a step in a text classification method provided by one embodiment of the present invention;

[0050] Figure 4 A schematic diagram of modules of a text classification device provided by an embodiment of the present invention;

[0051] Figure 5 A schematic diagram of the internal structure of an electronic device for implementing a text classification method provided by an embodiment of the present invention.

[0052] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0053] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0054] Embodiments of the present invention provide a text classification method. The text classification method may be executed by at least one electronic device, such as a server or a terminal, that can be configured to execute the method provided by the embodiments of this application. In other words, the text classification method may be executed by software or hardware installed on a terminal or server device, where the software may be a blockchain platform. The server may include, but is not limited to, a single server, a server cluster, a cloud server, or a cloud server cluster.

[0055] Reference Figure 1 FIG. 1 is a flow chart of a text classification method provided by an embodiment of the present invention. In the embodiment of the present invention, the text classification method includes the following steps S1-S7:

[0056] S1. Obtain a training text set and a real text classification result set corresponding to the training text set, and perform a vectorization operation on the training text set to obtain a training text vector set.

[0057] In an embodiment of the present invention, the training text set may include different types of texts. For example, the training text set may include treatment information for various medical diseases and description texts for various medical drugs.

[0058] In the embodiment of the present invention, the real text classification result set refers to the real and accurate classification results obtained by training the training text set in the text classification model, and the real classification results corresponding to the training text set can be annotated by an annotation tool.

[0059] As an embodiment of the present invention, performing a vectorization operation on the training text set to obtain a training text vector set includes:

[0060] A pre-built vector conversion model is used to perform a vectorization operation on the training text set to generate multiple word vectors; the multiple word vectors are pre-trained to obtain multiple pre-trained word vectors; and the words in the training text set are compared with the multiple pre-trained word vectors to obtain the training text vector set.

[0061] The pre-built vector conversion model may be word2vec, through which the training text set can be represented as a low-dimensional dense vector space. Word2vec takes into account the correlation between words and can better represent the words in the training text set.

[0062] In one embodiment of the present invention, since the pre-trained word vectors can be reduced to the output parameters of the quantization process, by pre-training the word vectors, words with similar semantics in the training text can be clustered in the word vector space, which provides convenience for subsequent text classification operations.

[0063] Furthermore, the embodiment of the present invention can remove the words in the training text set that are different from the multiple pre-trained word vectors by comparing the words in the training text set with the multiple pre-trained word vectors, and only retain the words that are the same as the multiple pre-trained word vectors to obtain the final training text vector set.

[0064] S2. Obtain a preset text classification model, where the preset text classification model includes multiple conversion layers, multiple side layers, and multiple classification layers.

[0065] In an embodiment of the present invention, the preset text classification model may be Bert, and the text classification model includes multiple conversion layers, multiple classification layers, and multiple side branches. The function of the conversion layer is to encode and decode the input training text vector set, and the model includes multiple conversion layers, each conversion layer is connected to the corresponding side branch layer, wherein the conversion layer may be a Transformer network. The side branch layer is an Adapter layer connected to each conversion layer, and the number of side branches is consistent with the number of conversion layers. The function of the side branch layer is to fix the model parameters upstream of the side branch layer during model training, and to fine-tune the downstream model parameters through the side branch layer, so as to separate the data passed to the next conversion layer and the data passed to the subsequent classification layer for training, thereby improving the classification accuracy of the text classification model.

[0066] S3. Utilize the plurality of conversion layers to perform first feature extraction on the training text vector set to obtain a first training text feature set.

[0067] In the embodiment of the present invention, the first training text feature set refers to the training text feature set output by each conversion layer according to the conversion layer sequence, and the training text feature set can be expressed as H = {H1, H2, ..., H m}.

[0068] The embodiment of the present invention obtains a first training text feature set by performing first feature extraction on the training text vector set using multiple conversion layers, which can accurately extract the features of the training text vector set and facilitate subsequent improvement of text classification accuracy.

[0069] As an embodiment of the present invention, the step of extracting a first feature from the training text vector set using the plurality of conversion layers to obtain a first training text feature set includes:

[0070] The training text vector set is position-encoded using the multi-head attention mechanism in the multiple conversion layers to obtain the text position vector set of the training text set; the text position vector set is linearly converted using multiple preset conversion parameters to obtain multiple text conversion vectors; and the multiple text conversion vectors are decoded to obtain the first training text feature set of each conversion layer.

[0071] Among them, by performing an Embedding operation on the position of each text character in the training text vector set, a text position vector set containing the position information of each text character can be obtained.

[0072] In one embodiment of the present invention, multiple preset conversion parameters include Q (query parameter), K (key-value parameter) and V (numerical parameter). By performing dot multiplication operations on each text character position vector in the text position vector set through these three parameters, the query vector, key-value vector and numerical vector of each text character can be obtained. After combining the query vector, key-value vector and numerical vector of each text character, the combined data are input into the feedforward layer (such as the Feed Forward layer) for linear conversion to obtain multiple text conversion vectors.

[0073] Furthermore, by performing a decoding operation on a plurality of text conversion vectors, feature extraction can be performed on the training text set, thereby improving the accuracy of extracting the first training text feature set from the training text vector set.

[0074] S4. Using the plurality of the side-branch layers, extract a second feature from the first training text feature set to obtain a second training text feature set.

[0075] In an embodiment of the present invention, the side layer is an adapter layer connected to each conversion layer, and the number of side layers is consistent with the number of conversion layers. The side layer includes: a lower projection layer, a nonlinear layer, an encoder, a nonlinear layer, and an upper projection layer.

[0076] The embodiment of the present invention obtains a second training text feature set by utilizing multiple side-branch layers to perform second feature extraction on the first training text feature set. The dimensionality reduction parameters and restoration parameters of the side-branch layer can be used to perform dimensionality reduction and restoration operations on the first training text feature set, so as to fix the model parameters upstream of the side-branch layer during model training and fine-tune the downstream model parameters through the side-branch layer. That is, the side-branch layer can separate the data passed to the next conversion layer and the data passed to the subsequent classification layer for training, which is convenient for improving the accuracy of the current classification layer and the subsequent classification layer of the model. Without introducing more parameters, the accuracy of the model prediction can be improved, thereby improving the accuracy of subsequent text classification.

[0077] As an embodiment of the present invention, refer to Figure 2As shown, the method of using the plurality of side-branch layers to extract the second feature of the first training text feature set to obtain the second training text feature set includes the following steps S41-S43:

[0078] S41, using the lower projection layers in the plurality of side-branch layers to perform a dimensionality reduction operation on the first training text feature set to obtain a first training text feature matrix;

[0079] S42, using the upper projection layer in the plurality of side-branch layers to restore the dimension of the first training text feature matrix to obtain a second training text feature matrix;

[0080] S43: Utilize the nonlinear layers in the plurality of side-branch layers to activate the first training text feature matrix and the second training text feature matrix to obtain the second training text feature set.

[0081] The second feature extraction of the first training text feature set using multiple side-branch layers can be achieved by the following formula:

[0082]

[0083] in, It refers to the second training text feature set corresponding to the first training text feature set obtained by the m-th conversion layer output by the p-th side branch layer, where p = m; H m-1 It refers to the first training text feature set output by the m-1th conversion layer; Wdown refers to the dimensionality reduction matrix; Wup refers to the restoration matrix; Enc represents the encoding layer; g1 and g2 represent the activation function Gelu of the nonlinear layer, where the g1 activation function is to activate the first training text feature matrix, and the g2 activation function is to activate the second training text feature matrix.

[0084] In one embodiment of the present invention, the lower projection layer is used to m-1 A dimensionality reduction matrix Wdown is applied on the upper layer to achieve dimensionality reduction operation on the first training text feature set, further extract important features of the first training text feature set, and facilitate improving the accuracy of subsequent model classification training; the role of the upper projection layer is to apply the reduction matrix Wup on the first training text feature matrix to ensure the consistency of parameter dimensions in the model. By introducing a small number of parameters of the dimensionality reduction matrix and the reduction matrix, it is possible to fix the model parameters upstream of the side branch layer during model training, and fine-tune the downstream model parameters through the side branch layer, that is, the side branch layer can separate the data passed to the next conversion layer and the data passed to the subsequent classification layer for training, so as to improve the accuracy of the current classification layer and the subsequent classification layer of the model, and improve the accuracy of the model prediction without introducing more parameters.

[0085] S5. Perform text classification on the second training text feature set using the multiple classification layers to obtain a predicted text classification result of the second training text feature set.

[0086] In this embodiment of the present invention, the classification layer is a network that classifies the training text set into text categories. The predicted text classification result refers to the classification result of each classification layer obtained for the training text set in the trained layer. For example, if the training text set is about purchasing financial product A, the predicted classification result can be the investment status of financial product A.

[0087] The embodiment of the present invention utilizes multiple classification layers to perform text classification on the second training text feature set to obtain a predicted text classification result of the second training text feature set, thereby realizing the prediction of the text category to which the training text set belongs and ensuring the accuracy of each classification layer in the text classification model, thereby improving the accuracy of text classification.

[0088] As an embodiment of the present invention, refer to Figure 3 As shown, the method of performing text classification on the second training text feature set using the multiple classification layers to obtain a predicted text classification result of the second training text feature set includes the following steps S51-S54:

[0089] S51, using the fully connected layers in the plurality of classification layers to respectively perform decoding operations on the second training text feature set to obtain a decoded training text feature set corresponding to each of the fully connected layers;

[0090] S52, using the activation functions in the multiple classification layers to output multiple initial predicted text classification results of the multiple decoding training text feature sets at each classification layer;

[0091] S53, respectively calculating the similarity between the initial predicted text classification results corresponding to any two of the classification layers;

[0092] S54: When the similarity meets a preset similarity, stop classifying the second training text feature set, and use the initial predicted text classification result obtained by the classification layer when the classification is stopped as the predicted text classification result of the second training text feature set.

[0093] Among them, since each classification layer contains a fully connected layer and an activation function, each fully connected layer corresponds to a decoding training text feature set, that is, each classification layer corresponds to a decoding training text feature set.

[0094] In one embodiment of the present invention, the activation function may be a ReLU function.

[0095] Furthermore, in an embodiment of the present invention, each decoded training text feature set obtained by each fully connected layer and the initial predicted text classification result output by the activation function connected to the fully connected layer are in a one-to-one correspondence, that is, a decoded training text feature set is input into a classification layer to obtain a corresponding initial predicted text classification result.

[0096] In one embodiment of the present invention, the similarity can be calculated by a cosine similarity algorithm to calculate the similarity between the initial predicted text classification results corresponding to any two classification layers. By calculating the similarity between the initial predicted text classification results corresponding to any two classification layers, when the similarity meets a preset similarity, the initial predicted text classification result obtained by the classification layer where the second training text feature set stops classification can be used as the predicted text classification result of the second training text feature set, thereby stopping the calculation of the subsequent text classification model, reducing the consumption of computing resources of the text classification model, and improving the training efficiency of the text classification model.

[0097] For example, similarity calculation refers to calculating the similarity between the initial predicted text classification result a of the sequential classification layer A and the initial predicted text classification result b of the classification layer B, and then calculating the similarity between the initial predicted text classification result b of the classification layer B and the initial predicted text classification result c of the classification layer C, until the initial predicted text classification results of the adjacent two classification layers meet the preset similarity, and then the classification of the second training text feature set is stopped.

[0098] Furthermore, respectively calculating the similarity between the initial predicted text classification results corresponding to any two of the classification layers includes:

[0099] Generating a predicted text category histogram for the initial predicted text classification results corresponding to any two of the classification layers;

[0100] Performing normalization calculation on the predicted text category histogram to obtain a normalized histogram;

[0101] The normalized histogram is calculated to obtain the similarity between the initial predicted text classification results corresponding to any two of the classification layers.

[0102] Among them, in order to simplify the difficulty of calculating similarity, the initial predicted text classification result can be converted into a histogram model first, which improves the calculation speed of similarity and thus improves the training efficiency of the preset text classification model.

[0103] S6. Calculate the loss value of the predicted text classification result based on the real text classification result set and the preset loss function, and adjust the parameters in the preset text classification model according to the loss value until the loss value meets the preset threshold, thereby obtaining a trained text classification model.

[0104] In an embodiment of the present invention, the loss value of the predicted text classification result can be calculated using the following preset loss function:

[0105] LC=m g logm p +(1-m g )log(1-m p )

[0106] Among them, LC represents the loss value, m g Represents the predicted text classification result, m p Represents the real text classification results.

[0107] Furthermore, in the embodiment of the present invention, the parameters of the preset text classification model include weight parameters and bias parameters, and the parameters of the preset text classification model can be adjusted using a gradient descent algorithm, such as a stochastic gradient descent algorithm.

[0108] In one embodiment of the present invention, the preset threshold may be 0.85.

[0109] S7. Obtain a text set to be classified, and classify the text set to be classified using the trained text classification model to obtain a target text classification result.

[0110] In the embodiment of the present invention, the to-be-classified text set refers to different types of text to be classified. For example, the to-be-classified text set may include texts on various types of medical products and treatments for various diseases. The target text classification result refers to the classification result of the to-be-classified text set.

[0111] Specifically, the trained text classification model can be used to classify the text set to be classified to obtain classification results for the text set. For example, the text set to be classified may be a collection of texts describing the uses of various types of medical drugs. The classification results for each type of medical drug may be classified as Category D for treating chronic diseases or Category E for treating lung cancer.

[0112] In an embodiment of the present invention, after obtaining a training text set and a real text classification result set corresponding to the training text set, the training text set is first vectorized to facilitate subsequent feature extraction of the training text set, and a plurality of conversion layers in a preset text classification model are used to perform first feature extraction on the training text vector set, so that the features of the training text vector set can be accurately extracted, which is convenient for subsequent improvement of the accuracy of text classification; secondly, a plurality of side branches in the text classification model are used to perform second feature extraction on the first training text feature set to obtain a second training text feature set, and the dimensionality reduction parameters and restoration parameters of the side branches can be used to perform dimensionality reduction and restoration operations on the first training text feature set, so as to achieve the fixing of the model parameters upstream of the side branches during model training, and the fine-tuning of the downstream model parameters through the side branches, that is, the side branches can pass the parameters to the next conversion layer. The data and the data passed to the subsequent classification layer are trained separately, which is convenient for improving the accuracy of the current classification layer and the subsequent classification layer of the model. Without introducing a large number of parameters, the accuracy of the model prediction can be improved, thereby improving the accuracy of subsequent text classification. Finally, by using multiple classification layers in the text classification model to perform text classification on the second training text feature set, a predicted text classification result of the second training text feature set is obtained, which can ensure the accuracy of each classification layer in the text classification model, thereby improving the accuracy of text classification. The parameters in the preset text classification model are adjusted according to the loss value until the loss value meets the preset threshold, thereby obtaining a trained text classification model to improve the classification accuracy of the text classification model. The trained text classification model is used to classify the text set to be classified to obtain the target text classification result, which can improve the accuracy of text classification. Therefore, the text classification method proposed in the embodiment of the present invention can improve the accuracy of text classification.

[0113] like Figure 4 As shown, the text classification device 100 of the present invention can be installed in an electronic device. Depending on the functions to be implemented, the text classification device can include a first acquisition module 101, a second acquisition module 102, a first feature extraction module 103, a second feature extraction module 104, a pre-classification module 105, a model training module 106, and a text classification module 107. The modules of the present invention can also be referred to as units, which refer to a series of computer program segments that can be executed by a processor of an electronic device and can perform fixed functions. They are stored in the memory of the electronic device.

[0114] In this embodiment, the functions of each module / unit are as follows:

[0115] In detail, each module in the text classification device 100 according to the embodiment of the present invention adopts the same Figures 1 to 3 The text classification method described in the text classification method is the same technical means and can produce the same technical effects, so it will not be repeated here.

[0116] In an embodiment of the present invention, after obtaining a training text set and a real text classification result set corresponding to the training text set, the training text set is first vectorized to facilitate subsequent feature extraction of the training text set, and a plurality of conversion layers in a preset text classification model are used to perform first feature extraction on the training text vector set, so that the features of the training text vector set can be accurately extracted, which is convenient for subsequent improvement of the accuracy of text classification; secondly, a plurality of side branches in the text classification model are used to perform second feature extraction on the first training text feature set to obtain a second training text feature set, and the dimensionality reduction parameters and restoration parameters of the side branches can be used to perform dimensionality reduction and restoration operations on the first training text feature set, so as to achieve the fixing of the model parameters upstream of the side branches during model training, and the fine-tuning of the downstream model parameters through the side branches, that is, the side branches can pass the parameters to the next conversion layer. The data and the data passed to the subsequent classification layer are trained separately, which facilitates improving the accuracy of the current classification layer and the subsequent classification layer of the model. Without introducing a large number of parameters, the accuracy of the model prediction can be improved, thereby improving the accuracy of subsequent text classification. Finally, by using multiple classification layers in the text classification model to perform text classification on the second training text feature set, a predicted text classification result of the second training text feature set is obtained, which can ensure the accuracy of each classification layer in the text classification model, thereby improving the accuracy of text classification. The parameters in the preset text classification model are adjusted according to the loss value until the loss value meets the preset threshold, thereby obtaining a trained text classification model to improve the classification accuracy of the text classification model. The trained text classification model is used to classify the text set to be classified to obtain the target text classification result, which can improve the accuracy of text classification. Therefore, the text classification device proposed in the embodiment of the present invention can improve the accuracy of text classification.

[0117] like Figure 5 FIG. 1 is a schematic diagram of the structure of an electronic device for implementing the text classification method of the present invention.

[0118] The electronic device may include a processor 10 , a memory 11 , a communication bus 12 , and a communication interface 13 . It may also include a computer program stored in the memory 11 and executable on the processor 10 , such as a text classification program.

[0119] The memory 11 includes at least one type of medium, including a flash memory, a mobile hard disk, a multimedia card, a card-type memory (for example, an SD or DX memory, etc.), a magnetic memory, a local disk, an optical disk, etc. In some embodiments, the memory 11 may be an internal storage unit of an electronic device, such as a mobile hard disk of the electronic device. In other embodiments, the memory 11 may also be an external storage device of an electronic device, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card, etc. equipped on the electronic device. Furthermore, the memory 11 may also include both an internal storage unit of the electronic device and an external storage device. The memory 11 can be used not only to store application software and various types of data installed in the electronic device, such as the code of a text classification program, but also to temporarily store data that has been output or is to be output.

[0120] In some embodiments, the processor 10 may be comprised of an integrated circuit, such as a single packaged integrated circuit or a plurality of packaged integrated circuits with the same or different functions, including a combination of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The processor 10 is the control core (control unit) of the electronic device, connecting the various components of the entire electronic device using various interfaces and circuits. It executes or runs programs or modules (such as text classification programs) stored in the memory 11 and calls data stored in the memory 11 to perform various functions of the electronic device and process data.

[0121] The communication bus 12 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. Such a bus may be classified as an address bus, a data bus, a control bus, etc. The communication bus 12 is configured to enable communication between the memory 11 and at least one processor 10. For ease of illustration, only one thick line is used in the figure, but this does not mean that there is only one bus or only one type of bus.

[0122] Figure 5 Only the electronic device with components is shown, and it can be understood by those skilled in the art that Figure 5The structure shown does not limit the electronic device, and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0123] For example, although not shown, the electronic device may further include a power source (such as a battery) for powering various components. Preferably, the power source may be logically connected to the at least one processor 10 via a power management device, thereby implementing functions such as charge management, discharge management, and power consumption management through the power management device. The power source may further include any components such as one or more DC or AC power sources, a recharging device, a power failure detection circuit, a power converter or inverter, a power status indicator, etc. The electronic device may further include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.

[0124] Optionally, the communication interface 13 may include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is generally used to establish a communication connection between the electronic device and other electronic devices.

[0125] Optionally, the communication interface 13 may further include a user interface, which may be a display or an input unit (such as a keyboard). Optionally, the user interface may also be a standard wired interface or a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. The display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device and to display a visual user interface.

[0126] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited to this structure.

[0127] The text classification program stored in the memory 11 of the electronic device is a combination of multiple computer programs. When running in the processor 10, it can achieve the following:

[0128] Obtaining a training text set and a real text classification result set corresponding to the training text set, and performing a vectorization operation on the training text set to obtain a training text vector set;

[0129] Obtaining a preset text classification model, wherein the preset text classification model includes multiple conversion layers, multiple side branches, and multiple classification layers;

[0130] Performing first feature extraction on the training text vector set using the plurality of conversion layers to obtain a first training text feature set;

[0131] Performing second feature extraction on the first training text feature set using the plurality of side-branch layers to obtain a second training text feature set;

[0132] Performing text classification on the second training text feature set using the plurality of classification layers to obtain a predicted text classification result of the second training text feature set;

[0133] Calculating the loss value of the predicted text classification result according to the real text classification result set and the preset loss function, and adjusting the parameters in the preset text classification model according to the loss value until the loss value meets the preset threshold, thereby obtaining a trained text classification model;

[0134] A text set to be classified is obtained, and the text set to be classified is classified using the trained text classification model to obtain a target text classification result.

[0135] Specifically, the specific implementation method of the processor 10 for the above computer program can refer to Figure 1 The description of the relevant steps in the corresponding embodiments will not be repeated here.

[0136] Furthermore, if the module / unit integrated into the electronic device is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable medium. The computer-readable medium can be non-volatile or volatile. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM).

[0137] An embodiment of the present invention may further provide a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program. When the computer program is executed by a processor of an electronic device, the computer program may implement:

[0138] Obtaining a training text set and a real text classification result set corresponding to the training text set, and performing a vectorization operation on the training text set to obtain a training text vector set;

[0139] Obtaining a preset text classification model, wherein the preset text classification model includes multiple conversion layers, multiple side branches, and multiple classification layers;

[0140] Performing first feature extraction on the training text vector set using the plurality of conversion layers to obtain a first training text feature set;

[0141] Performing second feature extraction on the first training text feature set using the plurality of side-branch layers to obtain a second training text feature set;

[0142] Performing text classification on the second training text feature set using the plurality of classification layers to obtain a predicted text classification result of the second training text feature set;

[0143] Calculating the loss value of the predicted text classification result according to the real text classification result set and the preset loss function, and adjusting the parameters in the preset text classification model according to the loss value until the loss value meets the preset threshold, thereby obtaining a trained text classification model;

[0144] A text set to be classified is obtained, and the text set to be classified is classified using the trained text classification model to obtain a target text classification result.

[0145] Furthermore, the computer-readable storage medium may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function, etc.; the data storage area may store data created according to the use of the blockchain node, etc.

[0146] In the several embodiments provided herein, it should be understood that the disclosed media, devices, apparatuses, and methods may be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For example, the module division is merely a logical functional division, and actual implementation may employ other division methods.

[0147] The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed across multiple network elements. Some or all of the modules may be selected to achieve the purpose of the solution of this embodiment according to actual needs.

[0148] In addition, the functional modules in various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or hardware plus software functional modules.

[0149] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0150] Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the invention is defined by the appended claims rather than the foregoing description, and all changes that come within the meaning and range of equivalents of the claims are intended to be embraced therein. Any reference to a figure in a claim should not be construed as limiting the claim to which it relates.

[0151] Blockchain, as used in this article, refers to a novel application model for computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Blockchain is essentially a decentralized database, a series of data blocks generated using cryptographic methods. Each block contains information about a batch of online transactions, used to verify the validity of this information (to prevent counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, the platform product service layer, and the application service layer.

[0152] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in a system claim may also be implemented by a single unit or device through software or hardware. Second-order terms are used to indicate names and do not imply any particular order.

[0153] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A text classification method, characterized in that: The method comprises: Obtaining a training text set and a real text classification result set corresponding to the training text set, and performing a vectorization operation on the training text set to obtain a training text vector set; Obtaining a preset text classification model, wherein the preset text classification model includes multiple conversion layers, multiple side branches, and multiple classification layers; Performing first feature extraction on the training text vector set using the plurality of conversion layers to obtain a first training text feature set; Performing second feature extraction on the first training text feature set using the plurality of side-branch layers to obtain a second training text feature set; Performing text classification on the second training text feature set using the plurality of classification layers to obtain a predicted text classification result of the second training text feature set; Calculating the loss value of the predicted text classification result according to the real text classification result set and the preset loss function, and adjusting the parameters in the preset text classification model according to the loss value until the loss value meets the preset threshold, thereby obtaining a trained text classification model; Obtaining a text set to be classified, and classifying the text set to be classified using the trained text classification model to obtain a target text classification result; Among them, the use of multiple side-branch layers to extract the second feature of the first training text feature set to obtain the second training text feature set includes: using the lower projection layer in the multiple side-branch layers to perform a dimensionality reduction operation on the first training text feature set to obtain a first training text feature matrix; using the upper projection layer in the multiple side-branch layers to restore the dimension of the first training text feature matrix to obtain a second training text feature matrix; using the nonlinear layer in the multiple side-branch layers to activate the first training text feature matrix and the second training text feature matrix to obtain the second training text feature set.

2. The text classification method according to claim 1, wherein: The step of performing text classification on the second training text feature set using the plurality of classification layers to obtain a predicted text classification result of the second training text feature set includes: Using the fully connected layers in the plurality of classification layers to respectively perform decoding operations on the second training text feature set to obtain a decoded training text feature set corresponding to each of the fully connected layers; Outputting a plurality of initial predicted text classification results of the plurality of decoded training text feature sets at each of the classification layers using activation functions in the plurality of classification layers; Calculating the similarity between the initial predicted text classification results corresponding to any two of the classification layers respectively; When the similarity meets the preset similarity, the classification of the second training text feature set is stopped, and the initial predicted text classification result obtained by the classification layer when the classification is stopped is used as the predicted text classification result of the second training text feature set.

3. The text classification method according to claim 2, wherein: The respectively calculating the similarity between the initial predicted text classification results corresponding to any two of the classification layers includes: Generating a predicted text category histogram for the initial predicted text classification results corresponding to any two of the classification layers; Performing normalization calculation on the predicted text category histogram to obtain a normalized histogram; The normalized histogram is calculated to obtain the similarity between the initial predicted text classification results corresponding to any two of the classification layers.

4. The text classification method according to any one of claims 1 to 3, characterized in that: The step of extracting a first feature from the training text vector set using the plurality of conversion layers to obtain a first training text feature set includes: Performing position encoding on the training text vector set using a multi-head attention mechanism in a plurality of the conversion layers to obtain a text position vector set of the training text set; Performing linear transformation on the text position vector set using a plurality of preset transformation parameters to obtain a plurality of text transformation vectors; A decoding operation is performed on the multiple text conversion vectors to obtain the first training text feature set of each of the conversion layers.

5. The text classification method according to any one of claims 1 to 3, characterized in that: The vectorization operation is performed on the training text set to obtain a training text vector set, including: Performing a vectorization operation on the training text set using a pre-built vector conversion model to generate multiple word vectors; Pre-training the plurality of word vectors to obtain a plurality of pre-trained word vectors; The words in the training text set are compared with the plurality of pre-trained word vectors to obtain the training text vector set.

6. A text classification device, characterized in that: The device comprises: A first acquisition module is used to obtain a training text set and a real text classification result set corresponding to the training text set, and perform a vectorization operation on the training text set to obtain a training text vector set; A second acquisition module is used to acquire a preset text classification model, wherein the preset text classification model includes multiple conversion layers, multiple side-branch layers, and multiple classification layers; A first feature extraction module, configured to perform first feature extraction on the training text vector set using the plurality of conversion layers to obtain a first training text feature set; A second feature extraction module is used to perform second feature extraction on the first training text feature set using the plurality of side-branch layers to obtain a second training text feature set; a pre-classification module, configured to perform text classification on the second training text feature set using the plurality of classification layers to obtain a predicted text classification result of the second training text feature set; A model training module is used to calculate the loss value of the predicted text classification result based on the real text classification result set and a preset loss function, and adjust the parameters in the preset text classification model according to the loss value until the loss value meets a preset threshold, thereby obtaining a trained text classification model; A text classification module is used to obtain a text set to be classified, classify the text set to be classified using the trained text classification model, and obtain a target text classification result; Among them, the second feature extraction module is specifically used to: use the lower projection layers in the multiple side-branch layers to perform a dimensionality reduction operation on the first training text feature set to obtain a first training text feature matrix; use the upper projection layers in the multiple side-branch layers to restore the dimension of the first training text feature matrix to obtain a second training text feature matrix; use the nonlinear layers in the multiple side-branch layers to activate the first training text feature matrix and the second training text feature matrix to obtain the second training text feature set.

7. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor. The computer program is executed by the at least one processor to enable the at least one processor to perform the text classification method according to any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the text classification method according to any one of claims 1 to 5 is implemented.

Images