A text classification model construction method, text classification method and device

Abstract

The invention discloses a text classification model construction method, a text classification method and a device, wherein the text classification model construction method comprises: obtaining training text data and a plurality of preset label attribute information corresponding to the field to which the training text data belongs; inputting the training text data and the plurality of preset label attribute information into a reading comprehension model to obtain a text interaction representation between each training text data and each preset label attribute information; inputting each text interaction representation into a feature extraction model to obtain a text key feature; inputting the text key feature into an initial classification model for classification and recognition to obtain a probability value that each training text data belongs to each preset label attribute information, and obtaining a classification result according to the probability value; and training the model parameters of the initial classification model according to the classification result until the loss function value of the initial classification model meets the target condition, thereby obtaining a target text classification model.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing and machine learning, and in particular to a text classification model construction method, a text classification method and a device. Background Art

[0002] Text classification refers to the use of computers to automatically classify and label text sets according to a certain classification system or standard. In related technologies, when classifying text, the semantic features of the text are usually extracted according to the text classification model for classification. However, this method requires a large number of training samples, and the text data corresponding to different text label categories are unbalanced, resulting in poor generalization ability of the model. Therefore, it is urgent to propose a method for building a text classification model to reduce training samples and improve the generalization ability of the model. Summary of the invention

[0003] Therefore, the technical problem to be solved by the present invention is to overcome the defect of poor generalization ability of the prior art in classifying texts by extracting semantic features of texts according to text classification models, thereby providing a text classification model construction method, a text classification method and a device.

[0004] According to the first aspect, an embodiment of the present invention discloses a method for constructing a text classification model, comprising the following steps: obtaining training text data and multiple preset label attribute information corresponding to the field to which the training text data belongs; inputting the training text data and multiple preset label attribute information into a reading comprehension model to obtain a text interaction representation between each training text data and each preset label attribute information; inputting each text interaction representation into a feature extraction model to obtain text key features; inputting the text key features into an initial classification model for classification and recognition to obtain a probability value that each training text data belongs to each preset label attribute information, and obtaining a classification result based on the probability value; training the model parameters of the initial classification model based on the classification result until the loss function value of the initial classification model meets the target condition, thereby obtaining a target text classification model.

[0005] Optionally, before inputting the training text data and multiple preset label attribute information into the reading comprehension model, the method also includes: obtaining pre-training text data, which has the same field and classification task as the training text data; inputting the pre-training text data into a machine learning model for training to obtain the reading comprehension model.

[0006] Optionally, the training text data and multiple preset label attribute information are input into a reading comprehension model, including: performing word segmentation processing on the training text data and the multiple preset label attribute information respectively to obtain a training text data sequence and a preset label attribute information sequence; splicing the training text data sequence and the preset label attribute information sequence, and inputting them into the reading comprehension model.

[0007] Optionally, the initial classification model is a multi-layer perception mechanism model, and the probability value of each training text data belonging to each preset label attribute information is obtained by the following formula:

[0008]

[0009] Among them, p(label j |W) represents the probability that the training text data belongs to the jth label; hcnn represents the key features of the text; h CLS represents the first characterization vector; Represents hcnn and h CLS Splicing; MLP represents the function of the multi-layer perception mechanism model.

[0010] Optionally, the loss function value is obtained by the following formula:

[0011] Loss = y × p (label j |W)+(1-y)×(1-p(label j |W))

[0012] Where loss represents the loss function value; y represents the true probability that the label of the training text data belongs to the jth label; p(label j |W) represents the model output probability that the label of the training text data belongs to the jth label.

[0013] According to the second aspect, an embodiment of the present invention further discloses a text classification method, comprising the following steps: obtaining a text to be classified and a plurality of preset attribute label information; inputting the text to be classified and the plurality of preset attribute label information into a text classification model trained by the text classification model training method described in the first aspect or any optional implementation manner of the first aspect for classification and recognition, and obtaining a text classification result.

[0014] According to the third aspect, an embodiment of the present invention also discloses a text classification model construction device, including: a first acquisition module, used to acquire training text data and multiple preset label attribute information corresponding to the field to which the training text data belongs; a first input module, used to input the training text data and multiple preset label attribute information into a reading comprehension model, and obtain a text interaction representation between each training text data and each preset label attribute information; a second input module, used to input each text interaction representation into a feature extraction model, and obtain text key features; an identification module, used to input the text key features into an initial classification model for classification and identification, and obtain a probability value that each training text data belongs to each preset label attribute information, and obtain a classification result based on the probability value; a model training module, used to train the model parameters of the initial classification model according to the classification result until the loss function value of the initial classification model meets the target condition, and obtains a target text classification model.

[0015] According to the fourth aspect, an embodiment of the present invention further discloses a text classification device, comprising: a second acquisition module, used to obtain the text to be classified and multiple preset attribute label information; a classification module, used to input the text to be classified and multiple preset attribute label information into the text classification model trained by the text classification model training method described in the first aspect or any optional implementation method of the first aspect for classification and recognition, to obtain a text classification result.

[0016] According to the fifth aspect, an embodiment of the present invention further discloses a computer device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the steps of the text classification model construction method described in the first aspect or any optional implementation manner of the first aspect or the steps of the text classification method described in the second aspect.

[0017] According to the sixth aspect, an embodiment of the present invention further discloses a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the text classification model construction method described in the first aspect or any optional implementation manner of the first aspect or the steps of the text classification method described in the second aspect are implemented.

[0018] The technical solution of the present invention has the following advantages:

[0019] 1. The text classification model construction method and device provided by the present invention obtains training text data and multiple preset label attribute information corresponding to the field to which the training text data belongs, inputs the training text data and multiple preset label attribute information into the reading comprehension model, obtains the text interaction representation between each training text data and each preset label attribute information, inputs each text interaction representation into the feature extraction model, obtains the text key feature, inputs the text key feature into the initial classification model for classification and recognition, obtains the probability value that each training text data belongs to each preset label attribute information, obtains the classification result according to the probability value, trains the model parameters of the initial classification model according to the classification result, until the loss function value of the initial classification model meets the target condition, and obtains the target text classification model. The present invention uses the label as attribute information to classify text, makes full use of the external information attached to the label itself, overcomes the problem of unbalanced text data, and uses the reading comprehension framework to fully interact the training text data and the label attribute information, effectively play the role of the label attribute information, improve the generalization ability of the text classification model, and improve the classification effect of the text classification model.

[0020] 2. The text classification method and device provided by the present invention obtains the text to be classified and multiple preset attribute label information, and inputs the text to be classified and the multiple preset attribute label information into a text classification model for classification and recognition to obtain a text classification result. The present invention simultaneously inputs the text to be classified and the multiple preset attribute label information into a pre-trained text classification model for classification and recognition, and the obtained text classification result is more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0022] Figure 1 A flowchart of a specific example of a method for building a text classification model in an embodiment of the present invention;

[0023] Figure 2 A flowchart of a specific example of a text classification method in an embodiment of the present invention;

[0024] Figure 3 A principle block diagram of a specific example of a device for building a text classification model in an embodiment of the present invention;

[0025] Figure 4is a principle block diagram of a specific example of a text classification device in an embodiment of the present invention;

[0026] Figure 5 FIG. 4 is a specific example diagram of a computer device in an embodiment of the present invention. DETAILED DESCRIPTION

[0027] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.

[0029] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, it can also be the internal connection of two components, it can be a wireless connection, or it can be a wired connection. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0030] In addition, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0031] The embodiment of the present invention discloses a method for constructing a text classification model. Figure 1 As shown, the following steps are included:

[0032] S11: Acquire training text data and a plurality of preset label attribute information corresponding to the field to which the training text data belongs.

[0033] Exemplarily, the training text data can be text data in any field, for example, comprehensive management, entertainment, etc. The training text data can be obtained from a search engine through a crawler, or purchased from a third-party technology manufacturer, or obtained from other companies through business cooperation with other companies. The embodiment of the present invention does not make specific settings for the training text data and the method for obtaining the training text data, and those skilled in the art can set them according to actual conditions.

[0034] The multiple preset tag attribute information can be the category description text of the preset tag, or it can be the additional information of the preset tag in the domain knowledge graph. The method for obtaining the multiple preset tag attribute information can be formulated in advance by experts according to the classification task requirements, or it can be generated through a reinforcement learning framework. The embodiment of the present invention is described by taking the multiple preset tag attribute information as the category description text of the preset tag as an example. The embodiment of the present invention does not specifically limit the multiple preset tag attribute information and the method for obtaining the multiple preset tag attribute information, and those skilled in the art can set it according to actual conditions.

[0035] S12: Input the training text data and a plurality of preset label attribute information into the reading comprehension model to obtain a text interaction representation between each training text data and each preset label attribute information.

[0036] Exemplarily, the reading comprehension model may be an existing BERT model, or may be obtained by training according to a classification task. The embodiment of the present invention does not specifically limit the reading comprehension model, and those skilled in the art may select one according to actual conditions.

[0037] The BERT model converts each word of the training text data and multiple preset label attribute information into a one-dimensional vector by querying the word vector table as the model input; the model output is the vector representation of the input corresponding to each word after integrating the semantic information of the full text. In addition, in addition to the word vector, the model input also includes two other parts: 1. Text vector: The value of this vector is automatically learned during the model training process, which is used to characterize the global semantic information of the text and integrate with the semantic information of a single word / word. 2. Position vector: Since the semantic information carried by words / words appearing at different positions in the text is different (for example: "I love you" and "You love me"), the BERT model attaches a different vector to the words / words at different positions to distinguish them. Finally, the BERT model uses the sum of the word vector, text vector and position vector as the model input.

[0038] S13: Input each text interaction representation into the feature extraction model to obtain the key features of the text.

[0039] Exemplarily, the key features of the text may include emotional words, weather information, modal particles, directional information, etc. The embodiment of the present invention does not specifically limit the key features of the text, and those skilled in the art may set them according to actual conditions. The feature extraction model may be a textcnn model, and each text interaction representation is input into the textcnn model for feature extraction. Multiple filters of different sizes in the model are used to extract key information from the text interaction representation to obtain the key features of the text for final classification.

[0040] For example, the text "Resident XXX reported that the neighbor's TV volume was very loud" is input into the feature extraction model, and the key features extracted may include: "XXX", "next door", "TV", and "loud sound".

[0041] S14: Inputting the key features of the text into the initial classification model for classification and recognition, obtaining the probability value of each training text data belonging to each preset label attribute information, and obtaining the classification result according to the probability value.

[0042] Exemplarily, the initial classification model can be a multi-layer perception mechanism (MLP) model. The key features of the text are input into the initial classification model, and the probability value of each training text data belonging to each preset label attribute information is obtained through the sigmoid function. Specifically, the probability value of each training text data belonging to each preset label attribute information can be obtained by the following formula:

[0043]

[0044] Among them, p(label j |W) represents the probability that the training text data belongs to the jth label; hcnn represents the key features of the text; h CLS represents the first characterization vector; Represents hcnn and h CLS Splicing; MLP represents the function of the multi-layer perception mechanism model.

[0045] According to the probability value, the classification result may be obtained by taking the preset label corresponding to the highest probability value as the classification result of the training text data.

[0046] S15: Train the model parameters of the initial classification model according to the classification results until the loss function value of the initial classification model meets the target conditions, thereby obtaining the target text classification model.

[0047] Exemplarily, the loss function value is obtained by the following formula:

[0048] Loss = y × p (label j |W)+(1-y)×(1-p(labelj |W))

[0049] Among them, loss represents the loss function value; y represents the true probability that the label of the training text data belongs to the jth label; p(label j |W) represents the model output probability that the label of the training text data belongs to the jth label. The text is fully learned using deep learning technology until the loss function value is reduced to a certain level.

[0050] The text classification model construction method provided by the present invention obtains training text data and a plurality of preset label attribute information of the fields to which the training text data belongs, inputs the training text data and the plurality of preset label attribute information into a reading comprehension model, obtains a text interaction representation between each training text data and each preset label attribute information, inputs each text interaction representation into a feature extraction model, obtains a text key feature, inputs the text key feature into an initial classification model for classification and recognition, obtains a probability value that each training text data belongs to each preset label attribute information, obtains a classification result according to the probability value, trains the model parameters of the initial classification model according to the classification result, until the loss function value of the initial classification model meets the target condition, and obtains a target text classification model. The present invention uses the label as attribute information to classify text, makes full use of the external information attached to the label itself, overcomes the problem of unbalanced text data, and uses a reading comprehension framework to fully perform feature interaction between the training text data and the label attribute information, effectively exerts the role of the label attribute information, improves the generalization ability of the text classification model, and improves the classification effect of the text classification model.

[0051] As an optional implementation of the embodiment of the present invention, before step S12, the text classification model construction method further includes:

[0052] First, obtain pre-training text data, which has the same domain and classification task as the training text data.

[0053] Exemplarily, the pre-trained text data is unlabeled data. In order to improve the generalization ability of the reading comprehension model, the pre-trained text data has the same domain and classification task as the training text data, and is different from the training text data. The acquisition method of the pre-trained text data can be the same as the acquisition method of the training text data, which will not be repeated here.

[0054] Secondly, the pre-trained text data is input into the machine learning model for training to obtain a reading comprehension model.

[0055] Exemplarily, the machine learning model is based on the BERT model, and the unlabeled pre-trained text data is input into the basic BERT model for strong supervision or weak supervision training. The embodiment of the present invention does not specifically limit the training method, and those skilled in the art can set it according to actual conditions.

[0056] The training can be based on a variety of task models, such as Mask Language Model (MLM), Next Sentence Prediction (NSP), etc. Based on different task models, there are different inputs and outputs. For example, for MLM, it is described as given a sentence, one or several words in the sentence are randomly erased, and it is required to predict what the erased words are based on the remaining vocabulary. The input is to randomly mask a part of each pre-trained text data, and the output is the masked words; for NSP, it is described as given two sentences in an article, judging whether the second sentence follows the first sentence in the text, the input is two sentences A and B, and the output is whether B is the following of A. The embodiment of the present invention does not specifically limit the task model, and those skilled in the art can select it according to the actual situation of the classification task.

[0057] As an optional implementation of the embodiment of the present invention, step S12 includes:

[0058] First, word segmentation is performed on the training text data and a plurality of preset label attribute information to obtain a training text data sequence and a preset label attribute information sequence.

[0059] Exemplarily, the training text data and a plurality of preset label attribute information are respectively segmented to obtain the training text data sequence and the preset label attribute information sequence. Specifically, the training text data and each character or word in the plurality of preset label attribute information are segmented by a maximum matching greedy algorithm (tokenize) with the help of a vocabulary to obtain the training text data sequence and the preset label attribute information sequence. For example, for the word manipulate, the word manipulate can be segmented into: "man", "##ip", "##ula", "##te" according to the vocabulary. If there are subwords that do not exist in the vocabulary, they are represented by a special mark [unk]. The vocabulary can be an existing vocabulary, or a vocabulary set in advance according to classification needs. The embodiment of the present invention does not specifically limit the vocabulary, and those skilled in the art can select according to actual conditions.

[0060] In the embodiment of the present invention, each training text data W = {w1, w2, ..., w n}Use the tokenize method to perform word segmentation to obtain the corresponding training text data sequence T = {t1, t2, ..., tm}, where w1 represents the i-th word in the training text data, n is the total number of words, and t i Represents the i-th token in the converted training text data sequence, and m is the total number of tokens. Similarly, each preset tag attribute information Use the tokenize method to perform word segmentation to obtain the corresponding preset tag attribute information sequence in represents the i-th word in the j-th preset tag attribute information, n j is the total number of words, Represents the i-th token in the converted preset tag attribute information sequence, m i Is the total number of tokens.

[0061] Secondly, the training text data sequence and the preset label attribute information sequence are concatenated and input into the reading comprehension model.

[0062] Exemplarily, the concatenation process of the training text data sequence and the preset label attribute information sequence may be as follows: the training text data sequence T and each preset label attribute information sequence T j Splice and get To facilitate direct input into the BERT model, [CLS] and [SEP] are essential special characters in the general BERT model. The [CLS] mark is placed at the first position of the first sentence in the BERT model, and the first representation vector obtained by the BERT model is used for subsequent classification; the [SEP] mark is used to separate the two input sentences.

[0063] The embodiment of the present invention performs word segmentation processing on the training text data and multiple preset tag attribute information respectively, and then inputs them into the reading comprehension model after splicing, so as to facilitate the processing of the reading comprehension model.

[0064] The embodiment of the present invention also discloses a text classification method, such as Figure 2 As shown, the following steps are included:

[0065] S21: Obtain the text to be classified and multiple preset attribute label information.

[0066] Exemplarily, the text to be classified can be uploaded to the server by the user, or stored in advance in the memory and directly called when used. The embodiment of the present invention does not specifically limit the method for obtaining the text to be classified, and those skilled in the art can set it according to actual conditions. The multiple preset attribute label information and the method for obtaining the same are the same as those in the above embodiment, and will not be repeated here.

[0067] S22: Input the text to be classified and a plurality of preset attribute label information into the text classification model trained by the text classification model training method of the above embodiment for classification and recognition to obtain a text classification result.

[0068] The text classification method provided by the present invention obtains the text to be classified and multiple preset attribute label information, and inputs the text to be classified and the multiple preset attribute label information into a text classification model for classification and recognition to obtain a text classification result. The present invention simultaneously inputs the text to be classified and the multiple preset attribute label information into a pre-trained text classification model for classification and recognition, and the obtained text classification result is more accurate.

[0069] The embodiment of the present invention also discloses a text classification model construction device, such as Figure 3 As shown, including:

[0070] The first acquisition module 31 is used to acquire training text data and a plurality of preset label attribute information corresponding to the field to which the training text data belongs; the specific implementation method is shown in the relevant description of step S11 in the embodiment, which will not be repeated here.

[0071] The first input module 32 is used to input the training text data and multiple preset label attribute information into the reading comprehension model to obtain the text interaction representation between each training text data and each preset label attribute information; the specific implementation method is shown in the relevant description of step S12 in the embodiment, which will not be repeated here.

[0072] The second input module 33 is used to input each text interaction representation into the feature extraction model to obtain the key features of the text; the specific implementation method can be found in the relevant description of step S13 in the embodiment, which will not be repeated here.

[0073] The recognition module 34 is used to input the key features of the text into the initial classification model for classification and recognition, obtain the probability value of each training text data belonging to each preset label attribute information, and obtain the classification result according to the probability value; the specific implementation method is shown in the relevant description of step S14 in the embodiment, which will not be repeated here.

[0074] The model training module 35 is used to train the model parameters of the initial classification model according to the classification results until the loss function value of the initial classification model meets the target condition, thereby obtaining the target text classification model. The specific implementation method is described in the relevant description of step S15 in the embodiment, which will not be repeated here.

[0075] The text classification model construction device provided by the present invention obtains training text data and multiple preset label attribute information corresponding to the field to which the training text data belongs, inputs the training text data and the multiple preset label attribute information into a reading comprehension model, obtains a text interaction representation between each training text data and each preset label attribute information, inputs each text interaction representation into a feature extraction model, obtains a text key feature, inputs the text key feature into an initial classification model for classification and recognition, obtains a probability value that each training text data belongs to each preset label attribute information, obtains a classification result according to the probability value, trains the model parameters of the initial classification model according to the classification result, until the loss function value of the initial classification model meets the target condition, and obtains a target text classification model. The present invention uses the label as attribute information to perform text classification, fully utilizes the external information attached to the label itself, overcomes the problem of unbalanced text data, and uses a reading comprehension framework to fully perform feature interaction between the training text data and the label attribute information, effectively exerts the role of the label attribute information, improves the generalization ability of the text classification model, and improves the classification effect of the text classification model.

[0076] As an optional implementation of the embodiment of the present invention, the text classification model construction method also includes:

[0077] The pre-trained text data acquisition module is used to obtain pre-trained text data. The pre-trained text data has the same domain and classification task as the training text data. The specific implementation method can be found in the relevant description of the corresponding steps in the embodiment, which will not be repeated here.

[0078] The training module is used to input the pre-trained text data into the machine learning model for training to obtain a reading comprehension model. The specific implementation method is described in the corresponding steps in the embodiment, which will not be repeated here.

[0079] As an optional implementation of the embodiment of the present invention, the first input module 32 includes:

[0080] The word segmentation module is used to perform word segmentation processing on the training text data and multiple preset label attribute information respectively to obtain a training text data sequence and a preset label attribute information sequence; the specific implementation method can be found in the relevant description of the corresponding steps in the embodiment, which will not be repeated here.

[0081] The splicing module is used to splice the training text data sequence and the preset tag attribute information sequence and input them into the reading comprehension model. The specific implementation method can be found in the relevant description of the corresponding steps in the embodiment, which will not be repeated here.

[0082] As an optional implementation of the embodiment of the present invention, the initial classification model is a multi-layer perception mechanism model, and the probability value of each training text data belonging to each preset label attribute information is obtained by the following formula:

[0083]

[0084] Among them, p(label j |W) represents the probability that the training text data belongs to the jth label; hcnn represents the key features of the text; h CLS represents the first characterization vector; Represents hcnn and h CLS Splicing; MLP represents a function of a multi-layer perception mechanism model. The specific implementation method is described in the corresponding steps in the embodiment, which will not be repeated here.

[0085] As an optional implementation of the embodiment of the present invention, the loss function value is obtained by the following formula:

[0086] Loss = y × p (label j |W)+(1-y)×(1-p(label j |W))

[0087] Among them, loss represents the loss function value; y represents the true probability that the label of the training text data belongs to the jth label; p(label j |W) represents the model output probability that the label of the training text data belongs to the jth label. The specific implementation method is shown in the relevant description of the corresponding steps in the embodiment, which will not be repeated here.

[0088] The embodiment of the present invention also discloses a text classification device, such as Figure 4 As shown, including:

[0089] The second acquisition module 41 is used to acquire the text to be classified and multiple preset attribute label information; the specific implementation method can be found in the relevant description of step S21 in the embodiment, which will not be repeated here.

[0090] The classification module 42 is used to input the text to be classified and multiple preset attribute label information into the text classification model trained by the text classification model training method of the above embodiment for classification and recognition to obtain the text classification result. The specific implementation method is shown in the relevant description of step S22 in the embodiment, which will not be repeated here.

[0091] The text classification device provided by the present invention obtains the text to be classified and multiple preset attribute label information, and inputs the text to be classified and the multiple preset attribute label information into a text classification model for classification and recognition to obtain a text classification result. The present invention simultaneously inputs the text to be classified and the multiple preset attribute label information into a pre-trained text classification model for classification and recognition, and the obtained text classification result is more accurate.

[0092] The embodiment of the present invention also provides a computer device, such as Figure 5 As shown, the computer device may include a processor 51 and a memory 52, wherein the processor 51 and the memory 52 may be connected via a bus or other means. Figure 5 The example of connecting through bus is taken in the following.

[0093] The processor 51 may be a central processing unit (CPU). The processor 51 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or a combination of the above chips.

[0094] The memory 52 is a non-transitory computer-readable storage medium that can be used to store non-transitory software programs, non-transitory computer executable programs and modules, such as the program instructions / modules corresponding to the text classification model construction method or the text classification method in the embodiment of the present invention (for example, Figure 3 The first acquisition module 31, the first input module 32, the second input module 33, the recognition module 34 and the model training module 35 shown in the figure or Figure 4 The processor 51 executes various functional applications and data processing of the processor by running the non-transient software programs, instructions and modules stored in the memory 52, that is, the text classification model construction method or the text classification method in the above method embodiment is implemented.

[0095] The memory 52 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and applications required for at least one function; the data storage area may store data created by the processor 51, etc. In addition, the memory 52 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 52 may optionally include a memory remotely arranged relative to the processor 51, and these remote memories may be connected to the processor 51 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0096] The one or more modules are stored in the memory 52 and when executed by the processor 51, perform the following steps: Figure 1 The text classification model construction method in the embodiment shown or Figure 2 The text classification method in the illustrated embodiment.

[0097] For details of the above computer equipment, please refer to Figure 1 to Figure 2 The corresponding related descriptions and effects in the illustrated embodiments can be understood and will not be repeated here.

[0098] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD), etc.; the storage medium can also include a combination of the above-mentioned types of memory.

[0099] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A method for constructing a text classification model, characterized in that: The steps include: Acquire training text data and a plurality of preset label attribute information corresponding to the field to which the training text data belongs; The training text data and the plurality of preset label attribute information are respectively subjected to word segmentation processing to obtain a training text data sequence and a preset label attribute information sequence, the training text data sequence and the preset label attribute information sequence are spliced ​​and input into a reading comprehension model to obtain a text interaction representation between each training text data and each preset label attribute information, wherein the training of the reading comprehension model comprises: obtaining pre-training text data, the pre-training text data having the same domain and classification task as the training text data; inputting the pre-training text data into a machine learning model for training to obtain the reading comprehension model, the machine learning model being a BERT model, [CLS] being a necessary special character in the BERT model, the mark being placed at the first position of the first sentence in the BERT model, the first position representation vector being obtained through the BERT model, the first position representation vector being used for classification recognition of an initial classification model; Input each text interaction representation into a feature extraction model to obtain key features of the text, wherein the key features of the text include emotional words, weather information, modal particles, and direction information; The key features of the text are input into the initial classification model for classification and identification, and the probability value of each training text data belonging to each preset label attribute information is obtained. According to the probability value, the classification result is obtained, wherein the initial classification model is a multi-layer perception mechanism model, and the probability value of each training text data belonging to each preset label attribute information is obtained by the following formula: Among them, p(label j |W) represents the probability that the training text data belongs to the jth label; hcnn represents the key features of the text; h CLS represents the first characterization vector; Represents hcnn and h CLS Splicing; MLP represents the function of the multi-layer perception mechanism model; The model parameters of the initial classification model are trained according to the classification results until the loss function value of the initial classification model meets the target conditions, thereby obtaining a target text classification model.

2. The method according to claim 1, characterized in that: The loss function value is obtained by the following formula: Loss=y×p(label j |W)+(1-y)×(1-p(label j |W)) Where loss represents the loss function value; y represents the true probability that the label of the training text data belongs to the jth label; p(label j |W) represents the model output probability that the label of the training text data belongs to the jth label.

3. A text classification method, characterized in that: The steps include: Get the text to be classified and multiple preset attribute label information; The text to be classified and multiple preset attribute label information are input into the text classification model trained by the text classification model training method described in any one of claims 1-2 for classification and recognition to obtain a text classification result.

4. A text classification model construction device, characterized in that: include: A first acquisition module, used to acquire training text data and a plurality of preset label attribute information corresponding to the field to which the training text data belongs; A first input module is used to perform word segmentation processing on the training text data and the plurality of preset tag attribute information respectively to obtain a training text data sequence and a preset tag attribute information sequence, concatenate the training text data sequence and the preset tag attribute information sequence, and input them into a reading comprehension model to obtain a text interaction representation between each training text data and each preset tag attribute information, wherein the training of the reading comprehension model comprises: obtaining pre-training text data, the pre-training text data and the training text data have the same domain and classification task; inputting the pre-training text data into a machine learning model for training to obtain the reading comprehension model, the machine learning model is a BERT model, [CLS] is a special character required in the BERT model, the mark is placed at the first position of the first sentence in the BERT model, and a first position representation vector is obtained through the BERT model, and the first position representation vector is used for classification recognition of an initial classification model; A second input module is used to input each text interaction representation into a feature extraction model to obtain key features of the text, wherein the key features of the text include emotional words, weather information, modal particles, and direction information; The recognition module is used to input the text key features into the initial classification model for classification and recognition, obtain the probability value of each training text data belonging to each preset label attribute information, and obtain the classification result according to the probability value, wherein the initial classification model is a multi-layer perception mechanism model, and the probability value of each training text data belonging to each preset label attribute information is obtained by the following formula: Among them, p(label j |W) represents the probability that the training text data belongs to the jth label; hcnn represents the key features of the text; h CLS represents the first characterization vector; Represents hcnn and h CLS Splicing; MLP represents the function of the multi-layer perception mechanism model; The model training module is used to train the model parameters of the initial classification model according to the classification result until the loss function value of the initial classification model meets the target condition, thereby obtaining a target text classification model.

5. A text classification device, characterized in that: include: The second acquisition module is used to acquire the text to be classified and multiple preset attribute label information; A classification module is used to input the text to be classified and multiple preset attribute label information into the text classification model trained by the text classification model training method described in any one of claims 1-2 for classification and recognition to obtain a text classification result.

6. A computer device, characterized in that: include: at least one processor; And a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor executes the steps of the text classification model construction method as described in any one of claims 1-2 or the steps of the text classification method as described in claim 3.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the text classification model construction method according to any one of claims 1 to 2 or the steps of the text classification method according to claim 3 are implemented.

Images