Text classification method and device, computer device and medium

Abstract

This invention relates to the field of natural language processing (NLP) technology, and more particularly to a text classification method, apparatus, computer device, and medium. The method acquires the text to be classified; performs primary classification on the text using a multi-classification module of a text classification model to obtain primary category names; determines secondary candidate category names based on the primary category names and a category relationship table; performs multi-level classification on the text and all secondary candidate category names using a relationship prediction module to obtain multi-level category names; predicts the primary category names and all multi-level category names to obtain multi-level classification prediction values; and determines the text classification result based on all multi-level classification prediction values. By redesigning the mask matrix of the attention mechanism in the text classification model, input conflicts between different tasks are avoided, enabling joint training and independent inference between different tasks, thereby achieving mutual learning between tasks and unifying multi-classification and NSP tasks.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, and in particular to a text classification method, apparatus, computer device, and medium. Background Technology

[0002] With the development of technology, classification techniques will be applied in more fields and play an increasingly important role. Text classification models are one of the important methods of classification techniques, capable of identifying the category to which text belongs.

[0003] In existing technologies, multi-level classification solutions typically decompose multi-level classification into a single-level multi-class classification model and a high-level NSP matching model. However, this approach has the following drawbacks: First, it requires two independent models (the multi-class classification task model and the NSP task model), which are more time-consuming and consume more GPU memory resources compared to a single model, both in training and inference. Second, predicting the single-level classification and the multi-level NSP matching classification separately limits the information learned by the model, resulting in lower accuracy in text classification.

[0004] Therefore, improving the accuracy of multi-level text classification and unifying multi-classification and NSP tasks have become urgent problems to be solved. Summary of the Invention

[0005] In view of this, embodiments of the present invention provide a text classification method, apparatus, computer device, and medium to solve the problems of low accuracy in multi-level text classification and the inability to unify multi-classification and NSP tasks.

[0006] In a first aspect, embodiments of the present invention provide a text classification method, including:

[0007] Obtain at least one text to be classified;

[0008] Obtain a text classification model, and perform primary classification on the text to be classified using the multi-classification module of the text classification model to obtain the primary category name;

[0009] Based on the primary category names and the category relationship table, at least one secondary candidate category name is determined;

[0010] The relationship prediction module of the text classification model performs multi-level classification on the text to be classified and all the secondary candidate category names to obtain at least one multi-level category name.

[0011] Predict the first-level category name and all the multi-level category names to obtain at least one multi-level classification prediction value;

[0012] Based on all the multi-level classification prediction values, determine the text classification result corresponding to each of the texts to be classified.

[0013] Secondly, embodiments of the present invention provide a text classification device, comprising:

[0014] The text acquisition module is used to acquire at least one text to be classified.

[0015] The primary classification module is used to obtain a text classification model, and to perform primary classification on the text to be classified through the multi-classification module of the text classification model to obtain the primary category name;

[0016] The candidate category module is used to determine at least one second-level candidate category name based on the first-level category name and the category relationship table;

[0017] A multi-level classification module is used to perform multi-level classification on the text to be classified and all the secondary candidate category names through the relationship prediction module of the text classification model, so as to obtain at least one multi-level category name;

[0018] The classification prediction module is used to predict the first-level category name and all the multi-level category names to obtain at least one multi-level classification prediction value.

[0019] The classification result module is used to determine the text classification result corresponding to each of the texts to be classified based on all the multi-level classification prediction values.

[0020] Thirdly, embodiments of the present invention provide a computer device, the computer device including a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the text classification method as described in the first aspect.

[0021] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the text classification method as described in the first aspect.

[0022] The beneficial effects of the embodiments of the present invention compared with the prior art are as follows:

[0023] This invention obtains at least one text to be classified; acquires a text classification model, and performs primary classification of the text to be classified using the multi-classification module of the text classification model to obtain primary category names; determines at least one secondary candidate category name based on the primary category name and a category relationship table; performs multi-level classification of the text to be classified and all secondary candidate category names using the relationship prediction module of the text classification model to obtain multi-level category names; predicts the primary category names, secondary candidate category names, and multi-level category names to obtain at least one multi-level classification prediction value; and determines the text classification result corresponding to each text to be classified based on all multi-level classification prediction values. This invention achieves the determination of primary, secondary, and multi-level category names of the text to be classified by performing primary classification using a multi-classification module and multi-level classification using a relationship prediction module. All secondary category names corresponding to the primary category names are found in the category relationship table, and the relationship prediction module matches all secondary category names to determine the secondary category names, thereby determining the multi-level category names. The multi-level classification prediction values ​​are determined by calculating the combination probability of the primary category name and each multi-level category name. By comparing all multi-level classification predictions, the optimal combination was determined, thereby determining the text classification result. Furthermore, by redesigning the mask matrix of the attention mechanism in the text classification model, input conflicts between different tasks were avoided, enabling joint training and independent inference between different tasks, thus achieving mutual learning between tasks and unifying multi-classification and NSP tasks. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a flowchart illustrating a text classification method provided in Embodiment 1 of the present invention;

[0026] Figure 2 This is a flowchart illustrating a text classification model training method provided in Embodiment 2 of the present invention;

[0027] Figure 3 This is a schematic diagram of the structure of a text classification device provided in Embodiment 3 of the present invention;

[0028] Figure 4 This is a schematic diagram of the structure of a computer device provided in Embodiment 4 of the present invention. Detailed Implementation

[0029] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of the invention. However, those skilled in the art will understand that the invention can be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods are omitted so as not to obscure the description of the invention with unnecessary detail.

[0030] It should be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0031] It should also be understood that the term “and / or” as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0032] As used in this specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if [described condition or event] is detected" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once [described condition or event] is detected," or "in response to detection of [described condition or event]."

[0033] Furthermore, in the description of this invention and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0034] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of the invention include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0035] To illustrate the technical solution of the present invention, specific embodiments are described below.

[0036] See Figure 1 This is a flowchart illustrating a text classification method provided in Embodiment 1 of the present invention, as shown below. Figure 1 As shown, this text classification method may include the following steps:

[0037] Step S10: Obtain at least one text to be classified.

[0038] Understandably, text to be categorized refers to text whose content requires multi-level classification. For example, texts about complaints or reports on public platforms would be categorized sequentially according to the complainant's province, such as Guangdong Province - Shenzhen City - Nanshan District, or Guangdong Province - Shenzhen City - Bao'an District, and then sent to the corresponding management department. The format of text to be categorized is [CLSn][CLSc]Item Text[SEP]Category Name[SEP]. Text to be categorized can be collected from different databases, pre-prepared text sent from the client to the database, or user inquiries collected from different websites.

[0039] Step S20: Obtain a text classification model, and perform primary classification on the text to be classified using the multi-classification module of the text classification model to obtain the primary category name.

[0040] Step S30: Determine at least one secondary candidate category name based on the primary category name and the category relationship table.

[0041] Understandably, the text classification model is an improvement and construction based on the BERT model. The mask matrix of the BERT model's attention mechanism has been redesigned, allowing the output of multi-classification tasks to be used as input for NSP tasks. The first-level category name is the classification category obtained through the multi-classification module, such as "urban management" or "platform complaint management." The category relationship table is a table setting up relationships between each category, such as "urban management - street order management" or "urban management - public health," where each category corresponds to multiple sub-categories, such as "artificial intelligence" corresponding to multiple subcategories like detection technology or recognition technology. The second-level candidate category name is the sub-category name corresponding to the first-level category name in the category relationship table.

[0042] Specifically, the trained text classification model is retrieved, and all texts to be classified are input into the model. First, the texts are encoded, i.e., segmented into words, to obtain all words and markers within the text. These words and markers are then encoded to obtain word vectors. Next, entire sentences are encoded to obtain sentence vectors. Then, the positional encoding function learned by the text classification model is used to positionally encode all word and sentence vectors, resulting in positional vectors. Finally, the word vectors, sentence vectors, and positional vectors of the same text are concatenated to obtain the input vector.

[0043] Furthermore, the text content in the input vector is classified at the first level by the multi-classification module in the text classification model. This means the category names in the input vector are empty inputs. After being input into the text classification model, the multi-classification module identifies and classifies the item text portion of the input vector. Specifically, the multi-classification module selects the [CLSc] item text [SEP] portion from the input vector and performs first-level classification prediction on the [CLSc] item text [SEP] portion through a fully connected layer. This involves masking the category name portion using a mask matrix in the fully connected layer, allowing the input vector of the multi-classification module to interact with its own text for prediction, resulting in all predicted categories. The category with the highest predicted probability among all predicted categories is then determined as the first-level category name corresponding to the text to be classified. Further, a category relationship table is obtained. By searching the category relationship table, all second-level category names associated with the first-level category name are found, and all second-level category names are determined as second-level candidate category names.

[0044] Step S40: The text to be classified and all the secondary candidate category names are classified in a multi-level manner through the relationship prediction module of the text classification model to obtain at least one multi-level category name.

[0045] Understandably, multilevel classification involves multiple levels of classification, such as second-level, third-level, and fourth-level classifications, until there are no further subcategories, i.e., no more subclasses. The multilevel category names are those obtained by sequentially performing second-level and third-level classifications through the relationship prediction module.

[0046] Specifically, after obtaining the secondary candidate category names, the primary category names are filled into the category names in the text to be classified, i.e., an updated text is constructed based on the text to be classified and its corresponding primary category names. The updated text is then re-encoded, specifically word vectors, sentence vectors, and position vectors. The word vectors, sentence vectors, and position vectors corresponding to the same updated text are concatenated to obtain the updated input vector. Each secondary candidate category name is then encoded to obtain a secondary candidate vector. The updated input vector and all secondary candidate vectors are input into the relation prediction module. The relation prediction module performs multi-level classification on the updated input vector and all secondary candidate vectors, matching the updated input vector with each secondary candidate vector to obtain a confidence value corresponding to each secondary candidate vector. All confidence values ​​are sorted from largest to smallest, and the top K (e.g., 2 or 3) secondary candidate vectors with the highest confidence values ​​are selected. The secondary candidate category names corresponding to these secondary candidate vectors are then determined as the secondary category names.

[0047] Furthermore, by searching the category relationship table for all tertiary category names associated with each secondary category name, and identifying all tertiary category names corresponding to each secondary category name as tertiary candidate category names, the tertiary candidate category names corresponding to each secondary category name are determined. The tertiary candidate category names corresponding to each secondary category name are encoded to obtain tertiary candidate vectors. The update input vector and the secondary classification vector of each secondary category name are concatenated to obtain multiple secondary input vectors. Each secondary input vector is matched with all tertiary candidate vectors corresponding to that secondary category name to obtain a confidence value corresponding to each tertiary candidate vector. All confidence values ​​are sorted from largest to smallest, and the top K tertiary candidate vectors with the highest confidence values ​​in the sorting results are selected. The tertiary candidate category names corresponding to each of these tertiary candidate vectors are then determined as tertiary category names. This process continues until the category name has no subclasses (second-level categories), thus obtaining the multi-level category names corresponding to the text to be classified.

[0048] Step S50: Predict the first-level category name and all the multi-level category names to obtain at least one multi-level classification prediction value.

[0049] Understandably, multi-level classification prediction values ​​are probability values ​​for predicting the probability of each group of primary category names and multi-level category names. The formula for calculating classification path probability is as follows:

[0050] Among them, cls i Representing category i, p(cls) i ) represents the predicted probability of that category, L is the path length (to which level to classify, such as level three or level four), and the parameter α is used as a hyperparameter to adjust the weight of the length penalty.

[0051] Specifically, after obtaining the multi-level category names, the text to be classified, the first-level category name, each second-level category name, and each third-level category name are each determined as a set of classification paths. For example, these could be: Artificial Intelligence - Recognition and Detection - Image Recognition, Artificial Intelligence - Recognition and Detection - Speech Recognition, Artificial Intelligence - Classification and Detection - Image Classification, or Artificial Intelligence - Classification and Detection - Text Classification. Probability prediction is then performed on each set of classification paths. This involves first calculating the probability of the first-level category name for each set of paths, then calculating the probability of each second-level category name, and finally calculating the probability of each third-level category name corresponding to that second-level category name. This yields the multi-level classification prediction value corresponding to each set of classification paths. Thus, depending on the requirements, the classification prediction values ​​for the first-level, second-level, third-level, or fourth-level categories can be calculated sequentially. For example, consider the multi-level classification results for government affairs on different datasets. For the first dataset, two independent models show the following probabilities: Level 1 classification: 0.8123; Level 2 classification: 0.5635; Level 3 classification: 0.3726. The text classification model shows the following probabilities: Level 1 classification: 0.8231; Level 2 classification: 0.6876; Level 3 classification: 0.5189. For the second dataset, two independent models show the following probabilities: Level 1 classification: 0.8846; Level 2 classification: 0.5453; Level 3 classification: 0.4674; Level 4 classification: 0.3957. The text classification model shows the following probabilities: Level 1 classification: 0.8859; Level 2 classification: 0.6554; Level 3 classification: 0.4974; Level 4 classification: 0.4852. These probabilities are merely illustrative examples.

[0052] Step S60: Determine the text classification result corresponding to each of the texts to be classified based on all the multi-level classification prediction values.

[0053] Understandably, the text classification result is the path after the text to be classified has been classified into multiple levels, such as urban management - street order management - sidewalk management - illegal parking on the sidewalk.

[0054] Specifically, after obtaining the multi-level classification prediction values, all multi-level classification prediction values ​​are compared. This involves comparing the size of the classification categories at the same level for all classification paths composed of multi-level and first-level category names, as well as comparing the classification prediction values ​​for all first-level, second-level, third-level, and fourth-level categories. The multi-level classification prediction values ​​are then sorted based on these comparison results. When a comparison result indicates that the multi-level classification prediction value for a given classification path is the largest, the multi-level and first-level category names in the classification path corresponding to this largest multi-level classification prediction value are determined as the text classification result. In other words, the classification path composed of first-level, second-level, third-level, and fourth-level category names is determined as the text classification result corresponding to the text to be classified.

[0055] This invention achieves the determination of primary, secondary, and multi-level category names for the text to be classified through a multi-classification module and a relation prediction module. All secondary category names corresponding to the primary category names are found in the category relation table, and the relation prediction module matches all secondary category names to determine the secondary category names, thereby determining the multi-level category names. The multi-level classification prediction value is determined by calculating the combination probability of the primary category name and each multi-level category name. The optimal combination is determined by comparing all multi-level classification prediction values, thus determining the text classification result. By redesigning the mask matrix of the attention mechanism in the text classification model, input conflicts between different tasks are avoided, enabling joint training and independent inference between different tasks, thus achieving mutual learning between tasks and unifying multi-classification and NSP tasks.

[0056] In one embodiment, step S10, i.e., before obtaining at least one text to be classified, includes:

[0057] S101, Obtain at least one initial text.

[0058] S102, perform format conversion on all the initial texts to obtain the text to be classified corresponding to each initial text.

[0059] Understandably, the initial text is user-submitted text or a user-asked question, such as Mr. Zhang reporting numerous illegally parked vehicles on the sidewalk at the corner of xx Street in xx District every day. It can be collected from different databases, sent from the client to the database, or collected from different websites. Format conversion transforms the initial text into a format that conforms to the input requirements, such as the data format [CLSn][CLSc]initial text[SEP]category name[SEP].

[0060] Specifically, after obtaining the initial text, all initial texts undergo format conversion. This involves adding a flag to the initial text based on the flags set during the inference phase, resulting in [CLSc]initialText[SEP]. This flag is used for category classification in the multi-classification module, yielding the first format text. Then, a flag for the NSP matching task in the relationship prediction module is added to the first format text, i.e., [CLSc]initialText[SEP], resulting in [CLSn][CLSc]initialText[SEP]CategoryName[SEP]. The format conversion result is then used to determine the text to be classified corresponding to the initial text. Thus, the text to be classified corresponding to each initial text is determined through the above format conversion method.

[0061] In this embodiment, by acquiring the initial text and converting the format of all the initial texts into a format that meets the input requirements, the text to be classified is determined, and the multi-classification task is made compatible with the NSP task, thereby improving the classification accuracy of the subsequent text to be classified.

[0062] In one embodiment, step S40 involves performing multi-level classification on the text to be classified and all the secondary candidate category names using the relationship prediction module of the text classification model to obtain at least one multi-level category name, including:

[0063] S401, determine the updated text based on the first-level category name corresponding to the text to be classified and the text to be classified.

[0064] S402, Match the updated text with all the names of the secondary candidate categories to obtain the confidence value corresponding to each of the secondary candidate category names.

[0065] Understandably, the updated text is the text to be classified after the primary category name is entered. The confidence value is the confidence score for each secondary candidate category name.

[0066] Specifically, after obtaining the primary category names, these primary category names are filled into the category name positions in the text to be classified, resulting in the updated text corresponding to each primary category name. The format of the updated text is [CLSn][CLSc]Initial Text[SEP]Primary Category Name[SEP]. This process is repeated to fill all the primary category names corresponding to each text to be classified, resulting in the updated text corresponding to each primary category name. Further, the updated text and all corresponding secondary candidate category names are input into the relationship prediction module. The relationship prediction model matches the updated text with each secondary candidate category name, calculating the correlation score between the updated input vector of the updated text and the secondary candidate vector of each secondary candidate category name. This correlation score is then used as the confidence score, yielding at least one confidence value corresponding to each secondary candidate category name.

[0067] S403, based on the confidence value, filter all the secondary candidate category names to obtain the multi-level category name corresponding to the text to be classified.

[0068] Specifically, all secondary candidate category names are filtered based on confidence scores, that is, all confidence scores are sorted from largest to smallest, and the secondary candidate category names corresponding to the top n confidence scores are selected based on the sorting results. The selected secondary candidate category names are then determined as the secondary category names.

[0069] Further, the system searches the category relationship table for all tertiary candidate category names associated with each secondary category name, encodes each tertiary candidate category name to obtain a tertiary candidate vector. It then encodes each secondary category name to obtain a secondary classification vector, concatenates the update input vector of the updated text with the secondary classification vector of each secondary category name to obtain a secondary input vector. The secondary input vector is matched with the tertiary candidate vector corresponding to that secondary category name to obtain a correlation score between the two vectors, which is then used as the confidence score to obtain the confidence value corresponding to each tertiary candidate category name. All tertiary candidate category names are then filtered based on the confidence values; that is, all confidence values ​​are sorted from largest to smallest, and the top n confidence values ​​are selected as the tertiary candidate category names. This process continues until a category name has no lower-level category (subclass), thus obtaining the multi-level category names corresponding to the text to be classified.

[0070] In this embodiment, the updated text is determined by filling in the first-level category name into the text to be classified. The second-level category name is determined by calculating the confidence value of each second-level candidate category name and filtering all second-level candidate category names based on the confidence values, thus enabling the determination of multi-level category names. Furthermore, by redesigning the mask matrix of the text classification model, input conflicts between different tasks are avoided, thereby unifying multi-classification tasks and NSP tasks.

[0071] In one embodiment, step S50, namely predicting the first-level category name and all the multi-level category names to obtain at least one multi-level classification prediction value, includes:

[0072] S501, determine at least one multi-level candidate category name based on the multi-level category name and the category relationship table.

[0073] S502, determine the primary category label based on the primary category name and all the secondary candidate category names; and determine the multi-level category label based on the multi-level category name and the multi-level candidate category name.

[0074] S503, perform probability prediction on the primary classification label and the multi-level classification label to obtain the multi-level classification prediction value.

[0075] Understandably, multi-level candidate category names are the sub-categories corresponding to the multi-level category names. First-level category labels are the categories of the first-level classification. Multi-level category labels are the categories of multi-level classifications, such as second-level, third-level, etc. The formula for calculating the classification path probability is...

[0076] Among them, cls i Representing category i, p(cls) i ) represents the predicted probability of that category, L is the path length (to which level to classify, such as level three or level four), and the parameter α is used as a hyperparameter to adjust the weight of the length penalty.

[0077] Specifically, after obtaining the multi-level category names, the corresponding multi-level candidate category names are searched in the category relationship table. For example, the corresponding tertiary candidate category names are searched in the category relationship table. The first-level category names and each second-level candidate category name are combined to obtain multiple first-level classification labels. Similarly, the multi-level category names and multi-level candidate category names are combined, that is, the second-level category names are combined with each tertiary candidate category name, and the tertiary category names are combined with each quaternary candidate category name, to obtain multiple multi-level classification labels.

[0078] Furthermore, probability prediction is performed on the primary classification label and multi-level classification label. That is, the probability of the combination of the primary category name and each secondary candidate category name is calculated to obtain the secondary classification prediction value corresponding to each secondary candidate category name. The probability of the combination of the secondary category name and each tertiary candidate category name is calculated to obtain the tertiary classification prediction value corresponding to each tertiary candidate category name. The probability of the combination of the tertiary category name and each quaternary candidate category name is calculated to obtain the quaternary classification prediction value corresponding to each quaternary candidate category name. In this way, the multi-level classification prediction value can be obtained.

[0079] In this embodiment, the primary category label is determined by combining the primary category name with all secondary candidate category names. Multi-level category labels are determined by combining multi-level category names with all multi-level candidate category names. Probability calculations are performed on each level of classification to predict multi-level classification values, thereby achieving accurate prediction for each level of classification.

[0080] In one embodiment, step S60, namely determining the text classification result corresponding to the text to be classified based on all the multi-level classification prediction values, includes:

[0081] S601, compare all the multi-level classification prediction values ​​to obtain the comparison results.

[0082] S602, when the comparison result indicates that the multi-level classification prediction value is the largest, the multi-level category name and the first-level category name corresponding to the comparison result are determined as the text classification result.

[0083] Understandably, the comparison results are used to characterize the magnitude of the multi-level classification predictions. The text classification result is the result of the multi-level classification of the text to be classified.

[0084] Specifically, after obtaining the multi-level classification prediction values, the second-level classification prediction values ​​of all second-level category names corresponding to the same text to be classified are compared to obtain the second-level classification comparison results. The third-level classification prediction values ​​of all third-level classification paths are then compared to obtain the third-level classification comparison results. Finally, the fourth-level classification prediction values ​​of all fourth-level classification paths are compared to obtain the fourth-level classification comparison results. The comparison is performed up to the level to which each classification path is classified. For example, for "Urban Management - Street Order Management - Pedestrian Walkway Management," which is classified as a third-level path, all third-level classification prediction values ​​are compared. Further, the multi-level classification prediction values ​​are sorted according to all comparison results corresponding to the same text to be classified. The comparison result with the highest multi-level classification prediction value for that classification path, i.e., the path with the highest probability, is then determined as the text classification result corresponding to the text to be classified.

[0085] In this embodiment, the comparison results are determined by comparing the classification prediction values ​​of each classification path corresponding to the same text to be classified. The multi-level classification prediction values ​​are then sorted based on all comparison results, ensuring that the classification path with the highest probability is determined as the text classification result corresponding to the text to be classified, thus guaranteeing a high accuracy rate for the text classification results.

[0086] See Figure 2 This is a flowchart illustrating a text classification model training method provided in Embodiment 2 of the present invention. The steps of training a preset training model include:

[0087] Step 701: Obtain a sample training dataset, which includes at least one sample training data, and one sample training data corresponds to one sample classification label.

[0088] Understandably, the sample training data can be obtained from different clients or collected from different databases and prepared in advance. Each sample training data corresponds to a sample classification label, which is the text multi-level classification result of the sample training data. For example, the sample training data is [CLSn][CLSc]Item Text [SEP]Category Name [SEP] (such as [CLSn][CLSc]Complaints & Inquiries [SEP]Management Department [SEP]), and the sample classification label is Urban Management - Street Order Management - Pedestrian Walkway Management - Illegal Parking on Pedestrian Walkways. Then, a sample training dataset is constructed based on the obtained sample training data and sample classification labels. During the model training phase, a primary classification label corresponding to each sample training data is provided, along with multiple secondary categories associated with the primary classification label. A complete data input format is constructed using the secondary category names. The sample training dataset also includes a category relationship table.

[0089] Step S702: Obtain a preset training model, and perform primary classification on the sample training data through the multi-classification module of the preset training model to obtain the primary classification label of the sample.

[0090] Understandably, the preset training model is a pre-built training model based on an improved BERT, meaning the mask matrix has been redesigned for the BERT model's attention mechanism. The primary classification label of the sample is the classification result from the multi-classification module, such as artificial intelligence, cloud computing, or big data.

[0091] Specifically, after obtaining the sample training data, a preset training model is invoked, and all sample training data is input into the preset training model. First, the sample training data is encoded, that is, the preset training model segments and encodes the sample training data into words to obtain word vectors. Then, the sentences in the sample training data are encoded to obtain sentence vectors. Next, the positional encoding function learned by the preset training model is used to determine the positional vectors. Based on the word vectors, sentence vectors, and positional vectors corresponding to the same sample training data, the word vectors, sentence vectors, and positional vectors are concatenated to obtain the encoded vectors corresponding to the sample training data. Further, the multi-classification module of the preset training model performs primary classification on the encoded vectors of the sample training data. That is, the multi-classification module classifies the category to which the encoded vectors of the sample training data belong using a category relationship table. Specifically, the attention mechanism in the multi-classification module uses a mask matrix to perform text interaction on the vectors of the [CLSc] item text [SEP] part of the sample training data, and finds the primary classification label corresponding to the sample training data from the primary classification in the category relationship table based on the interaction results. Thus, the primary classification label corresponding to each sample training data is determined in the above manner. In order to enable the [CLSc] item text [SEP] part in the sample training data to perform its own text interaction, the weights of other additional text information (i.e. [CLSn], category name [SEP]) are reset to zero.

[0092] Step S703: The sample training data and the sample first-level classification label are classified into multiple levels by the relationship prediction module of the preset training model to obtain the sample multi-level classification label.

[0093] Understandably, multi-level classification labels for samples refer to classifying the training data of samples into multiple levels, with each level corresponding to a classification label, such as a second-level classification label or a third-level classification label, until there are no sub-class levels in the category relationship table.

[0094] Specifically, the primary classification label of the sample is filled into the category name part of the sample training data (i.e., [SEP] category name [SEP]), resulting in new sample training data. This process is repeated, filling the category name part of the sample training data with the primary classification label corresponding to each sample training data, resulting in updated training data corresponding to each sample training data. The updated training data is then input into the relation prediction module of the preset training model. This module performs multi-level classification on the sample training data and the primary classification labels, predicting all secondary category labels corresponding to each primary classification label in the category relation table. In other words, it matches the updated training data with each secondary category label, obtaining the confidence value between them, and selecting the top K secondary category labels as secondary classification labels. Further, the relation prediction module matches all tertiary labels of each secondary classification label for each primary classification label in the category relation table, obtaining the confidence value between them, and selecting the top K tertiary category labels as tertiary classification labels. This process continues until no subclasses remain, resulting in the multi-level classification labels for the samples. In this method, the top K labels with high confidence in each category are selected in the beam search method. The probability of each classification path is calculated and the optimal solution is obtained as the classification result.

[0095] Step S704: Determine the prediction loss value of the preset training model based on the sample first-level classification label, the sample multi-level classification label, and the sample classification label corresponding to the same sample training data.

[0096] Understandably, the predicted loss value is generated during the training of the pre-set training model.

[0097] Specifically, based on the primary classification label and the secondary classification label corresponding to the same training data sample, the difference between the primary classification label and the secondary classification label is calculated to determine the first data loss value of that training data sample. Then, based on the first data loss values ​​of all training data samples, the first loss value of the multi-classification module is determined. Based on the multi-level classification label and the secondary classification label corresponding to the same training data sample, the difference between the multi-level classification label and the secondary classification label is calculated to determine the second data loss value of that training data sample. Then, based on the second data loss values ​​of all training data samples, the second loss value of the relationship prediction module is determined. Finally, based on the loss weights corresponding to the relationship prediction module and the multi-classification module, the overall loss of the preset training model is calculated to obtain the predicted loss value of the preset training model.

[0098] Step S705: When the predicted loss value reaches the convergence condition, the preset training model after convergence is determined as a text classification model.

[0099] Understandably, the convergence condition can be either the predicted loss value being less than a set threshold, or the predicted loss value being very small after 500 calculations and no longer decreasing.

[0100] Specifically, after obtaining the predicted loss value, if the predicted loss value does not reach the preset convergence condition, the initial parameters of the preset training model are adjusted using the predicted loss value. All sample training data are then re-inputted into the preset training model with adjusted initial parameters, and the model is retrained using the sample training data to obtain the predicted loss value corresponding to the model with adjusted initial parameters. This ensures that the predicted loss value of the preset training model with adjusted initial parameters reaches the preset convergence condition. In this way, the output of the preset training model continuously approaches the accurate result, increasing the model's prediction accuracy until the predicted loss value of the preset training model reaches the preset convergence condition. At this point, the converged preset training model is recorded as a text classification model.

[0101] In this embodiment, a pre-defined training model is trained using a large amount of sample training data, and the loss value of each module is calculated using a loss function, thus realizing the calculation of the prediction loss value. The initial parameters of the pre-defined training model are then adjusted multiple times based on the prediction loss value until the model converges, thereby constructing a text classification model and ensuring that the text classification model has a high prediction accuracy.

[0102] In one embodiment, step S704, namely determining the prediction loss value of the preset training model based on the sample primary classification label, the sample multi-level classification label, and the sample classification label corresponding to the same sample training data, includes:

[0103] Step S7041: Determine the first loss value of the preset training model based on the sample primary classification label and the sample classification label corresponding to the same sample training data.

[0104] Understandably, the first loss value is generated during the first-level classification of the sample training data by the multi-classification module.

[0105] Specifically, the primary classification labels of the training data are arranged according to the order of the training data. Then, the classification label corresponding to the training data is compared with the primary classification label of the same sequence. That is, according to the training data, the classification label of the first training data is compared with the primary classification label of the first training data, and the loss value between the classification label and the primary classification label is calculated by the loss function. Then, the classification label of the second training data is compared with the primary classification label of the second training data, until the primary classification label is compared with the primary classification label. The first loss value of the multi-classification module of the preset training model can be determined.

[0106] Step S7042: Determine the second loss value of the preset training model based on the multi-level classification label and the classification label of the sample corresponding to the same sample training data.

[0107] Understandably, the second loss value is generated by the relation prediction module during the multi-level classification of the sample training data.

[0108] Specifically, the multi-level classification labels of the training data are arranged according to the order of the training data in the training dataset. Then, the classification label corresponding to the training data is compared with the multi-level classification labels of the same sequence. That is, according to the training data, the classification label of the first training data is compared with the multi-level classification label of the first training data, and the loss value between the classification label and the multi-level classification label is calculated using a loss function. Then, the classification label of the second training data is compared with the multi-level classification label of the second training data, until the comparison between the multi-level classification labels and the classification labels is completed, and the second loss value of the relationship prediction module of the preset training model can be determined.

[0109] Step S7043: Determine the prediction loss value of the preset training model based on the first loss value, the second loss value, and the preset weight value.

[0110] Understandably, the preset weight values ​​are the weight values ​​corresponding to the multi-classification module and the relationship prediction module. The overall loss expression is loss = α * loss multi-classifier +β*loss nsp Where α and β are the weight values ​​of their respective modules, used to adjust the classification direction emphasized by the preset training model, and loss multi-classifier The first loss value is called loss. nsp This is the second loss value.

[0111] Specifically, the overall loss value of the preset training model is calculated by obtaining the preset weight values ​​for each module. This involves multiplying the preset weight value of each module by its loss value; that is, multiplying the first loss value of the multi-classification module by its preset weight value, and multiplying the second loss value of the relationship prediction module by its prediction weight value. Finally, the loss calculation results of each module are summed to obtain the predicted loss value of the preset training model. The weight values ​​and loss functions for each module can be the same or different, and can be adjusted according to the actual situation.

[0112] In this embodiment, the loss value of each module is calculated using the real and predicted labels corresponding to the same sample training data, thus determining the loss value of each module. The overall loss value of the preset training model is calculated using different weight values ​​for each module within the preset training model, thereby determining the predicted loss value and improving the accuracy of the object detection network.

[0113] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0114] See Figure 3 A text classification device is provided, which corresponds one-to-one with the text classification method described in the above embodiments. The text classification device includes a text acquisition module 11, a primary classification module 12, a candidate category module 13, a multi-level classification module 14, a classification prediction module 15, and a classification result module 16. Detailed descriptions of each functional module are as follows:

[0115] Text acquisition module 11 is used to acquire at least one text to be classified;

[0116] The primary classification module 12 is used to obtain a text classification model, and to perform primary classification on the text to be classified through the multi-classification module of the text classification model to obtain the primary category name;

[0117] Candidate category module 13 is used to determine at least one second-level candidate category name based on the first-level category name and the category relationship table;

[0118] Multi-level classification module 14 is used to perform multi-level classification on the text to be classified and all the secondary candidate category names through the relationship prediction module of the text classification model to obtain at least one multi-level category name;

[0119] The classification prediction module 15 is used to predict the first-level category name and all the multi-level category names to obtain at least one multi-level classification prediction value.

[0120] The classification result module 16 is used to determine the text classification result corresponding to each of the texts to be classified based on all the multi-level classification prediction values.

[0121] Optionally, the text acquisition module 11 mentioned above includes:

[0122] An initial text unit is used to obtain at least one initial text.

[0123] The format conversion unit is used to convert the format of all the initial texts to obtain the text to be classified corresponding to each of the initial texts.

[0124] Optionally, the above-mentioned primary classification module 12 includes:

[0125] The sample acquisition module is used to acquire a sample training dataset, which includes at least one sample training data, and one sample training data corresponds to one sample classification label.

[0126] The primary classification module is used to obtain a preset training model, and to perform primary classification on the sample training data through the multi-classification module of the preset training model to obtain the primary classification label of the sample.

[0127] A multi-level classification module is used to perform multi-level classification on the sample training data and the sample first-level classification label through the relationship prediction module of the preset training model to obtain the sample multi-level classification label.

[0128] The loss prediction module is used to determine the predicted loss value of the preset training model based on the sample first-level classification label, the sample multi-level classification label and the sample classification label corresponding to the same sample training data.

[0129] The model convergence module is used to determine the converged preset training model as a text classification model when the predicted loss value reaches the convergence condition.

[0130] The aforementioned loss prediction module includes:

[0131] The first loss unit is used to determine the first loss value of the preset training model based on the sample primary classification label and the sample classification label corresponding to the same sample training data;

[0132] The second loss unit is used to determine the second loss value of the preset training model based on the multi-level classification label of the sample and the classification label of the sample corresponding to the same training data of the sample.

[0133] The overall loss unit is used to determine the predicted loss value of the preset training model based on the first loss value, the second loss value, and the preset weight value.

[0134] Optionally, the aforementioned multi-level classification module 14 includes:

[0135] The update text unit is used to determine the update text based on the first-level category name corresponding to the text to be classified and the text to be classified.

[0136] A category matching unit is used to match the updated text with all the secondary candidate category names to obtain a confidence value corresponding to each of the secondary candidate category names;

[0137] The category filtering unit is used to filter all the secondary candidate category names according to the confidence value to obtain the multi-level category name corresponding to the text to be classified.

[0138] Optionally, the classification prediction module 15 mentioned above includes:

[0139] A multi-level candidate unit is used to determine at least one multi-level candidate category name based on the multi-level category name and the category relationship table;

[0140] A multi-level classification label unit is used to determine a first-level classification label based on the first-level category name and all the second-level candidate category names; and to determine a multi-level classification label based on the multi-level category name and the multi-level candidate category name.

[0141] The probability prediction unit is used to perform probability prediction on the primary classification label and the multi-level classification label to obtain the multi-level classification prediction value.

[0142] Optionally, the classification result module 16 mentioned above includes:

[0143] The comparison result unit is used to compare all the multi-level classification prediction values ​​to obtain a comparison result.

[0144] The classification result unit is used to determine the multi-level category name and the first-level category name corresponding to the comparison result as the text classification result when the comparison result represents the maximum multi-level classification prediction value.

[0145] It should be noted that the information interaction and execution process between the above modules and units are based on the same concept as the method embodiments of the present invention. For details on their specific functions and technical effects, please refer to the method embodiments section, which will not be repeated here.

[0146] Figure 4 This is a schematic diagram of the structure of a computer device provided in Embodiment 4 of the present invention. Figure 4 As shown, the computer device of this embodiment includes: at least one processor ( Figure 4Only one is shown in the diagram), a memory, and a computer program stored in the memory and capable of running on at least one processor, which, when executing the computer program, implements the steps in any of the above-described text classification method embodiments.

[0147] This computer device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that... Figure 4 The examples of computer devices are merely examples and do not constitute a limitation on computer devices. Computer devices may include more or fewer components than shown in the illustration, or combinations of certain components, or different components, such as network interfaces, displays, and input devices.

[0148] The processor referred to can be a CPU, but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0149] Memory includes readable storage media, internal memory, etc., wherein internal memory can be the RAM of a computer device, providing an environment for the operation of the operating system and computer-readable instructions stored in the readable storage media. The readable storage media can be the hard drive of a computer device, or in other embodiments, it can be an external storage device of the computer device, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, memory can include both internal storage units and external storage devices of a computer device. Memory is used to store the operating system, applications, bootloader, data, and other programs, such as program code for computer programs. Memory can also be used to temporarily store data that has been output or will be output.

[0150] Those skilled in the art will understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the functions described above can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this invention. The specific working process of the units and modules in the above device can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here. If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention can implement all or part of the processes in the methods of the above embodiments by instructing related hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the above method embodiments. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium can include at least: any entity or device capable of carrying computer program code, a recording medium, a computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, such as a USB flash drive, portable hard drive, magnetic disk, or optical disk.

[0151] The present invention can implement all or part of the processes in the methods of the above embodiments, or it can be accomplished by a computer program product. When the computer program product is run on a computer device, the computer device executes the steps in the above method embodiments.

[0152] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0153] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0154] In the embodiments provided by this invention, it should be understood that the disclosed apparatus / computer devices and methods can be implemented in other ways. For example, the apparatus / computer device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0155] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0156] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A text classification method, characterized in that, include: Obtain at least one text to be classified; Obtain a text classification model, and perform primary classification on the text to be classified using the multi-classification module of the text classification model to obtain the primary category name; Based on the primary category names and the category relationship table, at least one secondary candidate category name is determined; The relationship prediction module of the text classification model performs multi-level classification on the text to be classified and all the secondary candidate category names to obtain at least one multi-level category name. Predict the first-level category name and all the multi-level category names to obtain at least one multi-level classification prediction value; Based on all the multi-level classification prediction values, determine the text classification result corresponding to each of the texts to be classified; The relationship prediction module of the text classification model performs multi-level classification on the text to be classified and all the secondary candidate category names to obtain at least one multi-level category name, including: The updated text is determined based on the first-level category name corresponding to the text to be classified and the text to be classified. The updated text is matched with all the names of the secondary candidate categories to obtain the confidence value corresponding to each of the secondary candidate category names; Based on the confidence value, all the secondary candidate category names are filtered to obtain the multi-level category names corresponding to the text to be classified; The step of predicting the first-level category name and all the multi-level category names to obtain at least one multi-level classification prediction value includes: Based on the multi-level category names and the category relationship table, at least one multi-level candidate category name is determined; A primary category label is determined based on the primary category name and all the secondary candidate category names; and a multi-level category label is determined based on the multi-level category name and the multi-level candidate category name. Probability prediction is performed on the primary classification label and the multi-level classification label to obtain the multi-level classification prediction value.

2. The text classification method according to claim 1, characterized in that, The step of determining the text classification result corresponding to the text to be classified based on all the multi-level classification prediction values ​​includes: Compare all the multi-level classification prediction values ​​to obtain the comparison results; When the comparison result represents the maximum multi-level classification prediction value, the multi-level category name and the first-level category name corresponding to the comparison result are determined as the text classification result.

3. The text classification method according to claim 1, characterized in that, Before obtaining at least one text to be classified, the process includes: Get at least one initial text; The format of all the initial texts is converted to obtain the text to be classified corresponding to each of the initial texts.

4. The text classification method according to claim 1, characterized in that, Before obtaining the text classification model, the following steps are included: Obtain a sample training dataset, which includes at least one sample training data, and one sample training data corresponds to one sample classification label; Obtain a preset training model, and perform primary classification on the sample training data through the multi-classification module of the preset training model to obtain the primary classification label of the sample. The relationship prediction module of the preset training model performs multi-level classification on the sample training data and the sample first-level classification label to obtain the sample multi-level classification label. The prediction loss value of the preset training model is determined based on the sample primary classification label, the sample multi-level classification label, and the sample classification label corresponding to the same sample training data. When the predicted loss value reaches the convergence condition, the preset training model after convergence is determined as a text classification model.

5. The text classification method according to claim 4, characterized in that, The step of determining the prediction loss value of the preset training model based on the sample's primary classification label, the sample's multi-level classification label, and the sample's classification label corresponding to the same sample training data includes: The first loss value of the preset training model is determined based on the primary classification label and the classification label of the sample corresponding to the same sample training data. The second loss value of the preset training model is determined based on the multi-level classification label and the classification label of the sample corresponding to the same sample training data; The predicted loss value of the preset training model is determined based on the first loss value, the second loss value, and the preset weight value.

6. A text classification device, characterized in that, include: The text acquisition module is used to acquire at least one text to be classified. The primary classification module is used to obtain a text classification model, and to perform primary classification on the text to be classified through the multi-classification module of the text classification model to obtain the primary category name; The candidate category module is used to determine at least one second-level candidate category name based on the first-level category name and the category relationship table; A multi-level classification module is used to perform multi-level classification on the text to be classified and all the secondary candidate category names through the relationship prediction module of the text classification model, so as to obtain at least one multi-level category name; The classification prediction module is used to predict the first-level category name and all the multi-level category names to obtain at least one multi-level classification prediction value. The classification result module is used to determine the text classification result corresponding to each of the texts to be classified based on all the multi-level classification prediction values. The multi-level classification module includes: The update text unit is used to determine the update text based on the first-level category name corresponding to the text to be classified and the text to be classified. A category matching unit is used to match the updated text with all the secondary candidate category names to obtain a confidence value corresponding to each of the secondary candidate category names; The category filtering unit is used to filter all the secondary candidate category names according to the confidence value to obtain the multi-level category names corresponding to the text to be classified; The classification prediction module includes: A multi-level candidate unit is used to determine at least one multi-level candidate category name based on the multi-level category name and the category relationship table; A multi-level classification label unit is used to determine a first-level classification label based on the first-level category name and all the second-level candidate category names; and to determine a multi-level classification label based on the multi-level category name and the multi-level candidate category name. The probability prediction unit is used to perform probability prediction on the primary classification label and the multi-level classification label to obtain the multi-level classification prediction value.

7. A computer device, characterized in that, The computer device includes a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the text classification method as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the text classification method as described in any one of claims 1 to 5.

Images