Text hierarchical classification method and device, terminal and storage medium

Abstract

The application provides a text hierarchical classification method and device, a terminal and a storage medium. The method comprises the following steps: preprocessing a target text, and converting the preprocessed target text into a text ID; inputting the text ID into a pre-trained language model, and outputting a semantic dense vector corresponding to the text ID; inputting the semantic dense vector and a preset hierarchical label dictionary into a deep learning time sequence model based on an attention mechanism, and outputting classification labels of each level corresponding to the target text. The above method can not only reduce error propagation caused by constructing multiple classification modules, but also avoid the problem of poor model learning effect caused by the inability to utilize fine-grained hierarchical information, thereby optimizing the hierarchical classification effect.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of natural language processing, and particularly relates to a text hierarchical classification method and device, a terminal and a storage medium. BACKGROUND

[0002] In a common text classification task, categories are usually orthogonal, i.e., there is no inclusion relationship. Hierarchical classification is a special text classification task, i.e., there is a hierarchical relationship between categories, which can be represented as a tree or an undirected graph. In this kind of task, the label of a sample includes both the parent category and the child category in the hierarchical structure. In a more complex case, a sample contains multiple paths in the hierarchical structure, and this kind of task is called hierarchical multi-classification (HMC).

[0003] The common methods for hierarchical multi-classification mainly include two categories, Local (local method) and global (global method), which differ in the use of hierarchical structure information from different perspectives.

[0004] The Local method learns the relationship between different layers of categories and texts, and aggregates the prediction results of different layers to obtain the final prediction result. This kind of method usually consists of multiple classification modules, such as top-down hierarchical classification, in which there is a local classifier at each non-leaf node, and the final child category is predicted according to the prediction result of the parent category. The Local-based method can utilize more fine-grained hierarchical information, but it often needs to construct multiple classification modules, and is also susceptible to error propagation.

[0005] The Global method usually consists of one classification module, which directly utilizes the hierarchical structure information for modeling. For example, the hierarchical structure is used to construct a recursive regularization loss term to constrain the classification parameters. The Global-based method is simpler, but it often cannot utilize fine-grained hierarchical information in learning the semantic representation of texts, resulting in poor model learning effect and easy underfitting. SUMMARY

[0006] The present application provides a text hierarchical classification method, device, terminal and storage medium to solve the problem of poor text hierarchical classification effect in the prior art.

[0007] In a first aspect, the present application provides a text hierarchical classification method, comprising:

[0008] preprocessing a target text, and converting the preprocessed target text into a text ID;

[0009] input the text ID into a pre-trained language model to output a semantic dense vector corresponding to the text ID;

[0010] input the semantic dense vector and a preset hierarchical label dictionary into a deep learning time sequence model based on an attention mechanism to output classification labels of each level corresponding to the target text;

[0011] The preset hierarchical label dictionary includes all classification labels and the levels corresponding to each classification label.

[0012] In a possible implementation, the inputting of the text ID into the pre-trained language model to output the semantic dense vector corresponding to the text ID comprises:

[0013] The text ID is input into a Bert pre-trained language model to output a semantic dense vector corresponding to the text ID, and the dimension of the semantic dense vector is batch_size*sequence_length*hidden_size, wherein batch_size represents the number of sentences of the target text, sequence_length represents the maximum length of the sentence, and hidden_size represents the number of hidden units in the Bert pre-trained language model.

[0014] In a possible implementation, the inputting of the semantic dense vector and the preset hierarchical label dictionary into the deep learning time sequence model based on the attention mechanism to output the classification labels of each level corresponding to the target text comprises:

[0015] The semantic dense vector and the preset hierarchical label dictionary are input into a deep learning time sequence model based on a Bahdanau Attention computer mechanism and a Seq2Seq network framework to output the classification labels of each level corresponding to the target text.

[0016] In a possible implementation, the neurons of the deep learning time sequence model are long short-term memory networks (LSTM).

[0017] In a possible implementation, the preprocessing of the target text comprises:

[0018] The target text is cleaned to remove special symbols, abnormal texts and junk information in the target text.

[0019] The cleaned target text is analyzed in terms of sentence length distribution, and a standard length is determined according to the sentence length distribution, and the target text is truncated or padded based on the standard length.

[0020] In a second aspect, the present application provides a text hierarchical classification device, which comprises:

[0021] a text processing module, configured to pre-process a target text and convert the pre-processed target text into a text ID;

[0022] a semantic vector output module, configured to input the text ID into a pre-trained language model and output a semantic dense vector corresponding to the text ID;

[0023] a hierarchical label generation module, configured to input the semantic dense vector and a preset hierarchical label dictionary into a deep learning time sequence model based on an attention mechanism and output classification labels of each hierarchy corresponding to the target text;

[0024] The preset hierarchical label dictionary includes all classification labels and hierarchies corresponding to each classification label.

[0025] In a possible implementation, the semantic vector output module includes:

[0026] inputting the text ID into a Bert pre-trained language model and outputting a semantic dense vector corresponding to the text ID, wherein a dimension of the semantic dense vector is batch_size*sequence_length*hidden_size, wherein batch_size represents a number of sentences of the target text, sequence_length represents a maximum length of the sentences, and hidden_size represents a number of hidden units in the Bert pre-trained language model.

[0027] In a possible implementation, the hierarchical label generation module includes:

[0028] inputting the semantic dense vector and the preset hierarchical label dictionary into a deep learning time sequence model based on a Bahdanau Attention computer mechanism and a Seq2Seq network framework and outputting classification labels of each hierarchy corresponding to the target text.

[0029] In a third aspect, a terminal is provided, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the method in the possible implementation manners of the first aspect.

[0030] In a fourth aspect, a computer readable storage medium is provided, which stores a computer program, and the computer program is executable on a processor to implement the steps of the method in any of the possible implementation manners of the first aspect.

[0031] The embodiment of the present application provides a text hierarchical classification method, device, terminal and storage medium, the method first pre-processes a target text, and converts the pre-processed target text into a text ID; then inputs the text ID into a pre-trained language model, and outputs a semantic dense vector corresponding to the text ID; finally, inputs the semantic dense vector and a preset hierarchical label dictionary into a deep learning time sequence model based on an attention mechanism, and outputs classification labels of each level corresponding to the target text, the above method does not need to construct multiple classification modules, so that error propagation caused by constructing multiple classification modules can be reduced, the model structure is simplified, and the problem that the model learning effect is poor due to the inability to utilize fine-grained hierarchical information can be avoided, and the hierarchical classification effect is optimized. BRIEF DESCRIPTION OF DRAWINGS

[0032] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or prior art description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0033] Figure 1 is a flowchart of a text hierarchical classification method provided by the embodiment of the present application;

[0034] Figure 2 is a structural diagram of a deep learning time sequence model of a Seq2Seq network framework based on a Bahdanau Attention computer mechanism provided by the embodiment of the present application;

[0035] Figure 3 is a structural diagram of a text hierarchical classification device provided by the embodiment of the present application;

[0036] Figure 4 is a schematic diagram of a terminal provided by the embodiment of the present application. DETAILED DESCRIPTION

[0037] In the following description, specific details such as specific system structures, techniques, etc. are presented in order to thoroughly understand the embodiments of the present application. However, it should be clear to those skilled in the art that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits and methods are omitted to avoid unnecessary details that hinder the description of the present application.

[0038] In order to make the purpose, technical solutions and advantages of the present application clearer, the following will be described by specific embodiments in conjunction with the drawings.

[0039] Referring to Figure 1It shows an implementation flowchart of the text hierarchical classification method provided by the embodiment of the application, and is described in detail as follows:

[0040] S101: Preprocessing the target text, and converting the preprocessed target text into a text ID (Identity document).

[0041] In the embodiment, first, the target text to be classified is acquired, the target text is cleaned to eliminate special characters and abnormal texts, and then the cleaned text is formatted to obtain a standard text.

[0042] In the embodiment, after preprocessing the target text, the preprocessed target text is converted into a text ID through a preset dictionary, and the preset dictionary stores an ID corresponding to each character.

[0043] In a possible implementation manner, the specific implementation process of S101 includes:

[0044] The target text is cleaned to remove special symbols, abnormal texts and junk information in the target text.

[0045] The cleaned target text is analyzed in terms of sentence length distribution, and a standard length is determined according to the sentence length distribution, and the target text is truncated or padded based on the standard length.

[0046] In the embodiment, the text cleaning process includes removing junk information such as crawlers and web page labels, and removing illegal characters: \r\b\f\t...

[0047] S102: Inputting the text ID into a pre-trained language model, and outputting a semantic dense vector corresponding to the text ID.

[0048] In the embodiment, the pre-trained language model is used to capture linguistic features and reflect language phenomena, wherein the linguistic features include but are not limited to semantic features and syntactic features, the semantic features are used to represent the semantics of words and sentences, and the syntactic features are used to represent the structural organization of sentences and the dependency relationship between words in sentences. The language phenomena include polysemy, anaphora, pragmatics, etc.

[0049] Specifically, the text ID is input into the pre-trained language model, and a semantic dense vector corresponding to the text ID is output; wherein the dense vector represents a vector stored after modeling the vector using an array data structure.

[0050] S103: input the semantic dense vector and the preset hierarchical label dictionary into the deep learning time sequence model based on the attention mechanism, and output classification labels of each level corresponding to the target text.

[0051] The preset hierarchical label dictionary includes all classification labels and the levels corresponding to each classification label.

[0052] Specifically, attention is a concept proposed in deep learning, which is a kind of bionic method simulating human behavior. When humans obtain external information through eyes, information collection is processed through retina and brain. Human retina nerves selectively focus on some target scenes, which is the earliest source of attention. Attention mechanisms include additive, multiplicative and other attention mechanisms.

[0053] In the preset hierarchical label dictionary, all classification labels and the levels corresponding to each label are stored, and each storage location stores a classification label. The deep learning time sequence model based on the attention mechanism can predict the probability of each position in the semantic dense vector belonging to each classification label in each level. A sentence may include multiple hierarchical labels. The deep learning time sequence model takes the classification label with the maximum probability in each level as the classification label corresponding to the sentence in the level. For example, the sentence "XX model car has strong power, and refueling is fast!" The corresponding hierarchical label is: power (first level), engine (second level), acceleration (third level), and positive (fourth level).

[0054] In one possible implementation, the specific implementation process of S102 includes:

[0055] input the text ID into the Bert pre-training language model, and output a semantic dense vector corresponding to the text ID, the dimension of the semantic dense vector being batch_size*sequence_length*hidden_size, wherein batch_size represents the number of sentences of the target text, sequence_length represents the maximum length of the sentence, and hidden_size represents the number of hidden units in the Bert pre-training language model.

[0056] In the embodiment, BERT (Bidirectional Encoder Representations from Transformers) is a pre-trained language model (PLM), and BERT is implemented based on a Transformer and contains many Transformer modules. A key factor for the success of BERT is the powerful role of the Transformer. The Transformer can be understood as a neural network module, and the module has a complex network structure inside. The module implements fast parallelism through a self-attention mechanism, improves the most criticized slow training of RNN, and can be increased to a very deep depth, fully explores the characteristics of the DNN model, and improves the accuracy of the model.

[0057] As a pre-trained model, the BERT pre-trained language model does not need to be trained with a large amount of corpus when used in a specific scene, is time-saving and efficient, has strong generalization ability, is a bidirectional model, and has better performance. In addition, BERT is an end-to-end model, and the network structure does not need to be adjusted. Only an output layer specific to a downstream task needs to be added at the end.

[0058] For example, batch_size = 64, sequence_length = 256, and hidden_size = 768.

[0059] In a possible implementation, the specific implementation process of S103 includes the following steps.

[0060] The semantic dense vector and the preset hierarchical label dictionary are input into a deep learning time sequence model of a Seq2Seq network framework based on a Bahdanau Attention mechanism, and classification labels of each level corresponding to the target text are output.

[0061] In the embodiment, the Seq2Seq network framework is usually composed of two parts: an encoder and a decoder. The encoder is responsible for encoding and processing text / picture information into hidden information in deep learning. The decoder is usually a time sequence network, which decodes the hidden information, that is, gradually generates target information. The combination of Seq2seq and attention can better integrate semantic, position information and other information encoding.

[0062] Specifically, the Bahdanau Attention mechanism used in the embodiment is an additive attention mechanism, which aligns the hidden state of the decoder and the output of all positions of the encoder through linear combination to obtain a context vector, which is used to improve the sequence-to-sequence translation model. The essence is a two-layer fully connected network, and the hidden layer activation function is tanh, and the output layer dimension is 1.

[0063] Specifically, the preset hierarchical label dictionary is {0: <start>,1:tag0,...n+1:tag n ,...N+1:tag N }, where tag n This represents the category label at the nth position in the predefined hierarchical label dictionary, where N represents the number of category labels in the predefined hierarchical label dictionary. Therefore, the vector dimension corresponding to the predefined hierarchical label dictionary input to the deep learning time series model is batch_size*(N+1).

[0064] Figure 2 This embodiment illustrates the structure of a deep learning temporal model based on the Bahdanau Attention mechanism-based Seq2Seq network framework. Figure 2 As shown, the deep learning temporal model provided in this embodiment includes sequence_length encoders. The input of each encoder is a semantically dense vector, and the dimension of the input vector of each encoder is batch_size * hidden_size, that is, an encoder inputs a semantic vector corresponding to a position in a sentence at a time. The encoder generates a hidden state vector h for each input vector; the hidden state vector of the encoder at the final time step can be used as the hidden state vector of the decoder at the initial time step.

[0065] A deep learning temporal model consists of K decoders, where K represents the total number of classification label levels. The output of each time step in the decoder is determined by several variables, including the hidden state vector (h1, h2, ..., hn) at each position in the encoder. sequence_length ), and the output y of the decoder at the previous time step. t-1 And the hidden state vector s in the decoder at the current time. t Then there is p(y) t )=g(y t-1 ,s t ,c t ), where c t This represents the context vector at time t. a tj h represents time t j The probability distribution vector of , and a t,j =a(s t-1 ,h j ) = softmax(tanh(s t-1 W decoder +h j W encoder W alignment ), s t =f(s) t-1 c t y t-1 ). Wherein, W decoder , W encoder , W alignment respectively represent weight parameters.

[0066] In summary, the semantic dense vector corresponding to the target text is input into the encoder of the deep learning time sequence model, and the hidden state vector is output. The decoder determines the context vector based on the hidden state vector output by the encoder and the preset hierarchical label dictionary, and outputs the probability of all classification labels at each level based on the context vector, the output of the decoder at the previous moment and the hidden state vector, and takes the classification label with the maximum probability at the same level as the classification label corresponding to the target text at the level.

[0067] In a possible implementation, the neuron of the deep learning time sequence model is a long short-term memory network (LSTM).

[0068] The above method considers the interaction between the upper and lower hierarchical classification labels, and the model is simple, easy to train, and easy to achieve the ideal effect of application.

[0069] It should be understood that the size of the serial number of each step in the above embodiment does not mean the order of execution, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0070] The following is a device embodiment of the present application. For details not described in detail, reference can be made to the corresponding method embodiments described above.

[0071] Figure 3 The structure schematic diagram of the text hierarchical classification device provided by the embodiments of the present application is shown. For ease of illustration, only the parts related to the embodiments of the present application are shown, and the details are as follows:

[0072] As shown in Figure 3 , the text hierarchical classification device 100 comprises:

[0073] The text processing module 110 is configured to pre-process the target text and convert the pre-processed target text into a text ID.

[0074] The semantic vector output module 120 is configured to input the text ID into a pre-trained language model and output a semantic dense vector corresponding to the text ID.

[0075] The hierarchical label generation module 130 is configured to input the semantic dense vector and a preset hierarchical label dictionary into a deep learning time sequence model based on an attention mechanism, and output the classification labels of each level corresponding to the target text.

[0076] The preset hierarchical label dictionary includes all classification labels and a corresponding hierarchical level of each classification label.

[0077] In a possible implementation, the semantic vector output module 120 includes:

[0078] The text ID is input into a Bert pre-training language model, and a semantic dense vector corresponding to the text ID is output, wherein the semantic dense vector has a dimension of batch_size*sequence_length*hidden_size, batch_size represents a number of sentences of the target text, sequence_length represents a maximum length of the sentences, and hidden_size represents a number of hidden units in the Bert pre-training language model.

[0079] In a possible implementation, the hierarchical label generation module 130 includes:

[0080] The semantic dense vector and the preset hierarchical label dictionary are input into a deep learning time sequence model based on a Seq2Seq network framework of a Bahdanau Attention computer mechanism, and a classification label of each hierarchical level corresponding to the target text is output.

[0081] In a possible implementation, the neuron of the deep learning time sequence model is a long short-term memory network (LSTM).

[0082] In a possible implementation, the text processing module 110 includes:

[0083] The target text is cleaned to remove special symbols, abnormal texts, and junk information in the target text.

[0084] The cleaned target text is subjected to sentence length distribution analysis, and a standard length is determined according to the sentence length distribution, and the target text is truncated or padded based on the standard length.

[0085] The above device does not need to construct multiple classification modules, thereby reducing error propagation caused by constructing multiple classification modules, simplifying a model structure, avoiding a problem of poor model learning effect caused by being unable to use fine-grained hierarchical information, and optimizing hierarchical classification effect.

[0086] The embodiments of the present application also provide a computer program product having program codes, which perform the steps in any one of the above text hierarchical classification method embodiments when running in a corresponding processor, controller, computing device or terminal, for example Figure 1 The steps S101-S103 are shown. Those skilled in the art should understand that the method and the device provided by the embodiments of the present application can be implemented in various forms of hardware, software, firmware, special-purpose processor or combination thereof. The special-purpose processor can include application-specific integrated circuit (ASIC), reduced instruction set computer (RISC) and / or field programmable gate array (FPGA). The method and the device provided by the embodiments are preferably implemented as a combination of hardware and software. The software is preferably installed as an application program on a program storage device. It is typically a machine based on a computer platform with hardware, such as one or more central processing units (CPUs), random access memories (RAMs) and one or more input / output (I / O) interfaces. An operating system is also typically installed on the computer platform. The various processes and functions described herein can be part of the application program or part thereof can be executed by the operating system.

[0087] Figure 4 is a schematic diagram of a terminal provided by the embodiments of the present application. As shown in Figure 4 , the terminal 4 of the embodiments includes a processor 40, a memory 41 and a computer program 42 stored in the memory 41 and executable on the processor 40. The processor 40 implements the steps in each of the text hierarchical classification method embodiments described above when executing the computer program 42, for example Figure 2 steps 201-203 shown. Alternatively, the processor 40 implements the functions of each module / unit in each of the device embodiments described above when executing the computer program 42, for example Figure 3 the functions of the modules 110-130 shown.

[0088] For example, the computer program 42 can be divided into one or more modules / units, which are stored in the memory 41 and executed by the processor 40 to complete / implement the schemes provided by the present application. The one or more modules / units can be a series of computer program instruction segments capable of completing a specific function, which are used to describe the execution process of the computer program 42 in the terminal 4. For example, the computer program 42 can be divided into Figure 3 the modules 110-130 shown.

[0089] The terminal 4 can be a desktop computer, a notebook, a palm computer and a cloud server, etc. The terminal 4 can include, but is not limited to, the processor 40, the memory 41. Those skilled in the art can understand that Figure 4 The terminal 4 is merely an example and does not limit the terminal 4, which can include more or fewer components than shown, or have components in different locations or arrangements, and can also include different components not shown, such as input / output devices, network access devices, buses, etc.

[0090] The processor 40 can be a central processing unit (CPU), a general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or the like. The general-purpose processor can be a microprocessor, or the processor can be any conventional processor.

[0091] The memory 41 can be an internal storage unit of the terminal 4, such as a hard disk or a memory of the terminal 4. The memory 41 can also be an external storage device of the terminal 4, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, or the like. Further, the memory 41 can include both an internal storage unit and an external storage device of the terminal 4. The memory 41 is used to store the computer program and other programs and data required by the terminal. The memory 41 can also be used to temporarily store data that has been output or is to be output.

[0092] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above functional units and modules is exemplified, and in actual application, the above functions can be completed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules to complete all or part of the above-described functions. Each functional unit and module in the embodiment can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit, and the integrated unit can be realized in the form of hardware or in the form of a software functional unit. In addition, the specific names of each functional unit and module are only for easy distinction, and do not limit the protection scope of the present application. The specific working process of the units and modules in the system can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0093] In the above-described embodiments, the description of each embodiment focuses on different aspects, and parts not described in detail or recorded in a certain embodiment can be referred to the relevant description of other embodiments.

[0094] Those skilled in the art can appreciate that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether the functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0095] In the embodiments provided in the present application, it should be understood that the disclosed apparatus / terminal and method can be implemented in other ways. For example, the apparatus / terminal embodiments described above are only schematic, for example, the division of the modules or units is only a logical function division, and there can be another division in actual implementation, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interface, device or unit, and can be electrical, mechanical or other forms.

[0096] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiments.

[0097] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0098] The integrated module / unit, if implemented in the form of a software functional unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, all or part of the processes in the above-mentioned embodiments can also be implemented by a computer program instructing related hardware, and the computer program can be stored in a computer readable storage medium. The computer program can implement the steps of each text level classification method embodiment when executed by a processor. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or some intermediate forms, etc. The computer readable medium can include any entity or device, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium, etc. that can carry the computer program code. It should be noted that the contents included in the computer readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer readable medium does not include electrical carrier signals and telecommunication signals.

[0099] In addition, the features of the embodiments shown in the drawings or mentioned in the specification of the present application are not necessarily understood as independent embodiments from each other. Rather, each feature described in one example of an embodiment can be combined with one or more other desired features from other embodiments, thereby generating other embodiments not described in words or with reference to the drawings.

[0100] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.< / start>

Claims

1. A method for text hierarchical classification, the method comprising: The method comprises the following steps: preprocessing a target text and converting the preprocessed target text into a text identity number; inputting the text identity number into a pre-trained language model to output a semantic dense vector corresponding to the text identity number; inputting the semantic dense vector and a preset hierarchical label dictionary into a deep learning time sequence model based on an attention mechanism to output classification labels of each level corresponding to the target text; the preset hierarchical label dictionary comprises all classification labels and the levels corresponding to each classification label; the step of inputting the text identity number into the pre-trained language model to output the semantic dense vector corresponding to the text identity number comprises: inputting the text identity number into a bidirectional encoder representation method pre-training language model, and outputting a semantic dense vector corresponding to the text identity number, wherein the semantic dense vector has a dimension of a number of sentences of the target text * a maximum length of the sentence * a number of hidden units in the bidirectional encoder representation method pre-training language model the step of inputting the semantic dense vector and the preset hierarchical label dictionary into the deep learning time sequence model based on the attention mechanism to output the classification labels of each level corresponding to the target text comprises: inputting the semantic dense vector and the preset hierarchical label dictionary into a deep learning time sequence model based on a Bahdanau attention computer mechanism and a sequence-to-sequence network framework to output the classification labels of each level corresponding to the target text.

2. The text hierarchical classification method of claim 1, wherein, The neurons of the deep learning time sequence model are long short-term memory networks.

3. The text hierarchical classification method of claim 1, wherein, The preprocessing of the target text comprises the following steps: cleaning the target text to remove special symbols, abnormal texts and junk information in the target text; performing sentence length distribution analysis on the cleaned target text, determining a standard length according to the sentence length distribution, and truncating or padding the target text based on the standard length.

4. A text hierarchical classification apparatus characterized by comprising: The method comprises the following steps: a text processing module for preprocessing a target text and converting the preprocessed target text into a text identity number; a semantic vector output module for inputting the text identity number into a pre-trained language model to output a semantic dense vector corresponding to the text identity number; a hierarchical label generation module for inputting the semantic dense vector and a preset hierarchical label dictionary into a deep learning time sequence model based on an attention mechanism to output classification labels of each level corresponding to the target text; the preset hierarchical label dictionary comprises all classification labels and the levels corresponding to each classification label; the semantic vector output module comprises: inputting the text identity number into a bidirectional encoder representation method pre-training language model, and outputting a semantic dense vector corresponding to the text identity number, wherein a dimension of the semantic dense vector is a number of sentences of the target text * a maximum length of the sentence * a number of hidden units in the bidirectional encoder representation method pre-training language model the hierarchical label generation module comprises: inputting the semantic dense vector and the preset hierarchical label dictionary into a deep learning time sequence model based on a Bahdanau attention computer mechanism and a sequence-to-sequence network framework to output the classification labels of each level corresponding to the target text.

5. A terminal comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to realize the steps of the text hierarchical classification method according to any one of claims 1 to 3.

6. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 5. The computer program is executed by the processor to realize the steps of the text hierarchical classification method according to any one of claims 1 to 3.

Images