A text classification method, device, storage medium and equipment

Abstract

The present application discloses a text classification method, apparatus, storage medium, and device. The method comprises: first obtaining a target text to be classified, inputting the target text into a pre-built text classification model, and identifying a hyperplane feature vector corresponding to the target text; then respectively calculating the Euclidean distances between the hyperplane feature vector corresponding to the target text and the hyperplane feature vectors corresponding to the center points of N preset categories; wherein N is a positive integer greater than 0; then taking the preset category corresponding to the minimum Euclidean distance among the obtained N Euclidean distances as the target category, and determining the relationship between the minimum Euclidean distance and the radius of the hypersphere where the target category is located to obtain a judgment result, and then classifying the target text according to the judgment result to obtain a classification result of the target text. Thus, when performing text classification, human intervention is no longer required, thereby improving the accuracy of the text classification result.

Description

Technical Field

[0001] The present application relates to the field of natural language processing technology, and in particular to a text classification method, apparatus, storage medium and device. Background Art

[0002] With the explosive growth of computer computing power, the application of deep learning in artificial intelligence is also increasing. More and more engineering projects are replacing traditional manual features with neural network models, such as text intent classification.

[0003] At present, the existing text classification methods usually adopt a discriminant classification model, which outputs a confidence score for all categories of the text, and then takes the category to which the maximum confidence score belongs as the text category. This classification method cannot adaptively judge out-of-domain data of out-of-domain categories. If you want to identify out-of-domain data of out-of-domain categories, you need to manually set a confidence threshold so that when the maximum category confidence score of the failed category is lower than the threshold, the text can be classified as an out-of-domain category; but obviously this threshold is difficult to determine and needs to be adjusted continuously as the data changes dynamically. It relies heavily on manual experience and is disturbed by the uncertainty of the data. Therefore, the existing text classification method has a low recognition accuracy for out-of-domain data belonging to the out-of-domain category, which further makes the online learning accuracy of the classification model lower and lower or even not feasible. Moreover, the process of manually adjusting the threshold is highly subjective and difficult to quantify. Not only is the classification efficiency low, but it also requires a lot of human resources. Summary of the Invention

[0004] The main purpose of the embodiments of the present application is to provide a text classification method, apparatus, storage medium and device that can improve the accuracy of text classification results.

[0005] This embodiment of the present application provides a text classification method, including:

[0006] Get the target text to be classified;

[0007] Inputting the target text into a pre-built text classification model to identify and obtain a hyperplane feature vector corresponding to the target text;

[0008] Calculating the Euclidean distances between the hyperplane feature vector corresponding to the target text and the hyperplane feature vectors corresponding to N preset category center points, respectively; wherein N is a positive integer greater than 0;

[0009] The preset category corresponding to the minimum Euclidean distance among the obtained N Euclidean distances is used as the target category, and the relationship between the minimum Euclidean distance and the radius of the hypersphere where the target category is located is determined to obtain a determination result;

[0010] According to the judgment result, the target text is classified to obtain a classification result of the target text.

[0011] In one possible implementation, the text classification model is constructed as follows:

[0012] Get sample text;

[0013] The sample speech, the first optimization function, and the second optimization function are used to train an initial text classification model to obtain the text classification model.

[0014] In one possible implementation, the sample text belongs to the N preset categories; the first optimization function is used to ensure that the distance between each sample text on the hyperplane of each category in the N preset categories and its center point tends to be consistent; the second optimization function is used to ensure that the distance between each pair of sample texts on the hyperplane of each category in the N preset categories is as far as possible.

[0015] In a possible implementation, the hyperplane feature vectors corresponding to the N preset category center points are obtained by training the text classification model using the first optimization function and the second optimization function.

[0016] In a possible implementation, the method further includes:

[0017] Get verification text;

[0018] Inputting the verification text into the text classification model to obtain a classification result corresponding to the verification text;

[0019] When the classification result of the verification text is inconsistent with the classification label result corresponding to the verification text, the verification text is used as the sample text again to update the text classification model.

[0020] In a possible implementation, classifying the target text according to the judgment result to obtain the classification result of the target text includes:

[0021] When the judgment result is that the minimum Euclidean distance is not greater than the radius of the hypersphere where the target category is located, it is determined that the target text belongs to the target category, and a classification result of the target text is obtained.

[0022] Alternatively, when the judgment result is that the minimum Euclidean distance is greater than the radius of the hypersphere where the target category is located, it is determined that the target text belongs to an out-of-domain category, and a classification result of the target text is obtained.

[0023] In a possible implementation, after classifying the target text according to the judgment result to obtain the classification result of the target text, the method further includes:

[0024] The text classification model is updated using the target text and the classification result of the target text to improve the classification accuracy of the text classification model.

[0025] The present application also provides a text classification device, including:

[0026] A first acquisition unit, configured to acquire a target text to be classified;

[0027] A recognition unit, configured to input the target text into a pre-built text classification model and identify and obtain a hyperplane feature vector corresponding to the target text;

[0028] a calculation unit, configured to respectively calculate the Euclidean distances between the hyperplane feature vector corresponding to the target text and the hyperplane feature vectors corresponding to N preset category center points; wherein N is a positive integer greater than 0;

[0029] a judgment unit, configured to take a preset category corresponding to a minimum Euclidean distance among the obtained N Euclidean distances as a target category, and to determine a magnitude relationship between the minimum Euclidean distance and a radius of a hypersphere where the target category is located, to obtain a judgment result;

[0030] The classification unit is used to classify the target text according to the judgment result to obtain a classification result of the target text.

[0031] In a possible implementation, the apparatus further includes:

[0032] A second acquiring unit, configured to acquire a sample text;

[0033] The training unit is used to train the initial text classification model using the sample speech and the first optimization function and the second optimization function to obtain the text classification model.

[0034] In one possible implementation, the sample text belongs to the N preset categories; the first optimization function is used to ensure that the distance between each sample text on the hyperplane of each category in the N preset categories and its center point tends to be consistent; the second optimization function is used to ensure that the distance between each pair of sample texts on the hyperplane of each category in the N preset categories is as far as possible.

[0035] In a possible implementation, the hyperplane feature vectors corresponding to the N preset category center points are obtained by training the text classification model using the first optimization function and the second optimization function.

[0036] In a possible implementation, the apparatus further includes:

[0037] A third obtaining unit is used to obtain a verification text;

[0038] An input unit, configured to input the verification text into the text classification model to obtain a classification result corresponding to the verification text;

[0039] The first updating unit is configured to update the text classification model by reusing the verification text as the sample text when the classification result of the verification text is inconsistent with the classification label result corresponding to the verification text.

[0040] In a possible implementation, the classification unit is specifically configured to:

[0041] When the judgment result is that the minimum Euclidean distance is not greater than the radius of the hypersphere where the target category is located, it is determined that the target text belongs to the target category, and a classification result of the target text is obtained.

[0042] Alternatively, when the judgment result is that the minimum Euclidean distance is greater than the radius of the hypersphere where the target category is located, it is determined that the target text belongs to an out-of-domain category, and a classification result of the target text is obtained.

[0043] In a possible implementation, the apparatus further includes:

[0044] The second updating unit is used to update the text classification model using the target text and the classification result of the target text after classifying the target text according to the judgment result to improve the classification accuracy of the text classification model.

[0045] The embodiment of the present application also provides a text classification device, including: a processor, a memory, and a system bus;

[0046] The processor and the memory are connected via the system bus;

[0047] The memory is used to store one or more programs, where the one or more programs include instructions, and when the instructions are executed by the processor, the processor is enabled to perform any one of the implementations of the above-mentioned text classification method.

[0048] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions. When the instructions are executed on a terminal device, the terminal device executes any one of the implementations of the above-mentioned text classification method.

[0049] An embodiment of the present application further provides a computer program product, which, when executed on a terminal device, enables the terminal device to execute any one of the implementations of the above-mentioned text classification method.

[0050] The embodiment of the present application provides a text classification method, device, storage medium and equipment, which first obtains the target text to be classified and inputs the target text into a pre-built text classification model to identify the hyperplane feature vector corresponding to the target text; then respectively calculates the Euclidean distance between the hyperplane feature vector corresponding to the target text and the hyperplane feature vector corresponding to the center point of N preset categories; wherein N is a positive integer greater than 0, and then takes the preset category corresponding to the minimum Euclidean distance among the obtained N Euclidean distances as the target category, and judges the size relationship between the minimum Euclidean distance and the radius of the hypersphere where the target category is located to obtain a judgment result, and then can classify the target text according to the judgment result to obtain the classification result of the target text. It can be seen that since the embodiment of the present application uses the hyperplane feature vector of the target text output by the pre-built text classification model to calculate the distance with each preset category center point, if it is closest to the center point and the distance within the category hypersphere is not greater than the category hypersphere radius, then the target text is judged to belong to the category, otherwise it belongs to the out-of-domain category, so that when performing text classification, human intervention is no longer required, thereby improving the accuracy of the text classification result. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0052] Figure 1 A flowchart of a text classification method provided in an embodiment of the present application;

[0053] Figure 2 One of the network structure diagrams of the text classification model provided in the embodiment of the present application;

[0054] Figure 3 The second diagram of the network structure of the text classification model provided in the embodiment of the present application;

[0055] Figure 4 A schematic diagram of the hypersphere spatial distribution provided in an embodiment of the present application;

[0056] Figure 5 A schematic diagram of the composition of a text classification device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0057] In some text classification methods, a discriminant model is usually used to output a confidence score for all categories of the text, and then the category to which the maximum confidence score belongs is taken as the category corresponding to the text. This classification method cannot identify out-of-domain data of out-of-domain categories (that is, text that does not belong to the model's preset distinguishable types); if out-of-domain data of out-of-domain categories is to be identified, a confidence threshold needs to be set manually so that when the maximum category confidence score that fails is lower than the threshold, the text can be classified as an out-of-domain category. However, it is obvious that this threshold is difficult to determine and needs to be adjusted continuously as the data changes. It relies heavily on manual experience and is affected by the uncertainty of the data. Therefore, its out-of-domain category recognition accuracy is not high, and even makes the accuracy lower and lower during the iteration process or even not feasible. At the same time, the process of manually adjusting the threshold also brings more manpower costs.

[0058] Therefore, how to improve the accuracy of text classification results and reduce the cost of text classification is a technical problem that needs to be solved urgently.

[0059] In order to solve the above defects, the present application provides a text classification method, which first obtains the target text to be classified and inputs the target text into a pre-built text classification model to identify the hyperplane feature vector corresponding to the target text; then calculates the Euclidean distance between the hyperplane feature vector corresponding to the target text and the hyperplane feature vector corresponding to the center point of N preset categories; wherein N is a positive integer greater than 0, and then takes the preset category corresponding to the minimum Euclidean distance among the obtained N Euclidean distances as the target category, and judges the size relationship between the minimum Euclidean distance and the radius of the hypersphere where the target category is located to obtain a judgment result, and then can classify the target text according to the judgment result to obtain the classification result of the target text. It can be seen that since the embodiment of the present application uses the hyperplane feature vector of the target text output by the pre-built text classification model to calculate the distance with each preset category center point, the target text is judged to belong to the category if it is closest to the center point and the distance within the category hypersphere is not greater than the category hypersphere radius, otherwise it belongs to the out-of-domain category, so that when performing text classification, human intervention is no longer required, thereby improving the accuracy of the text classification results.

[0060] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0061] First embodiment

[0062] See also Figure 1 , is a flow chart of a text classification method provided in this embodiment, the method comprising the following steps:

[0063] S101: Acquire target speech to be recognized.

[0064] In this embodiment, any text for which text classification is implemented using this embodiment is defined as a target text. Furthermore, this embodiment does not limit the language type of the target text; for example, the target text can be Chinese text or English text; this embodiment does not limit the length of the target text; for example, the target text can be a sentence text or a chapter text; this embodiment does not limit the source of the target text; for example, the target text can be the result of speech recognition or log data collected from various business systems of the platform; this embodiment does not limit the type of the target text; for example, the target text can be a sentence in people's daily conversations or part of a speech, magazine article, literary work, etc.

[0065] It can be understood that sentence text refers to a sentence, which is a collection of individual words, and paragraph text refers to a collection of a series of sentences. After obtaining sentence text or paragraph text as the target text to be expressed, it can be classified according to subsequent steps.

[0066] S102: Input the target text into a pre-built text classification model to identify and obtain the hyperplane feature vector corresponding to the target text.

[0067] In this embodiment, after obtaining the target text to be classified in step S101, in order to quickly and accurately identify the classification result of the target text, the target text can be further input into a pre-built text classification model to identify and obtain the hyperplane feature vector corresponding to the target text, so as to perform the subsequent step S103. It should be noted that the specific format of the hyperplane feature vector corresponding to the target text can be set according to actual conditions (such as the architecture of the selected classification network model, etc.), and this embodiment does not limit this. For example, the hyperplane feature vector corresponding to the target text can be a 768-dimensional digital vector, etc.

[0068] Next, this embodiment will introduce the construction process of the text classification model, wherein an optional implementation method is that the construction process of the text classification model can specifically include: first obtaining sample text, and then using sample speech and the first optimization function and the second optimization function to train the initial text classification model to obtain the text classification model.

[0069] Specifically, in this implementation, in order to build a text classification model, a lot of preparatory work needs to be done in advance. First, a large amount of text data needs to be collected as sample text to constitute the model training data. For example, news texts, user reviews of a movie, and textbook texts can be collected in advance through online channels, and each collected text data can be used as sample text data to constitute the model training data, and the classification results corresponding to these sample voices can be manually marked. Then, based on these sample voices and the classification results corresponding to the sample voices, as well as the first optimization function and the second optimization function, the initial text classification model can be trained to generate the initial text classification model.

[0070] It should be noted that since the function of the text classification model is to vectorize the text, that is, to represent the text in a hyperdimensional space, when selecting the initial text classification model, it is sufficient to ensure that it supports parallel computing and backpropagation optimization algorithms. The specific model to be used and the dimension after the text is vectorized, that is, the dimension of the hyperplane feature vector, are not specified in this case. For example, the initial text classification model can be selected as a bidirectional long short-term memory network (BiLSTM) or BERT, and the dimension of the hyperplane feature vector can be set to 768, etc.; Figure 2 As shown in , a 768-dimensional digital vector can be obtained by inputting the text into the BiLSTM text classification model; or, as Figure 3 As shown in the figure, a 768-dimensional digital vector can also be obtained after inputting the text into the BERT text classification model.

[0071] During model training, a sample text can be extracted from the training data as model input in turn, and multiple rounds of model training can be performed until the values of the first optimization function and the second optimization function meet the preset conditions. Then, the updating of the model parameters is stopped, the training of the text classification model is completed, and a trained text classification model is generated.

[0072] Specifically, during model training, the center points of the hyperdimensional spaces of the N preset categories to which the sample text belongs are randomly initialized, and the distance from each sample text on the hyperplane of each category to its center point is gradually adjusted to be consistent through continuous updating of the first optimization function. The value of the first optimization function is as follows:

[0073]

[0074] Among them, L a represents the first optimization function; N represents the total number of preset categories; M represents the total number of sample texts; x ij represents the hyperplane feature vector of the i-th sample belonging to the j-th preset category; c j The hyperplane feature vector representing the center point of the hyperdimensional plane of the j-th preset category.

[0075] Then, the first optimization function L described in the above formula (1) is used a During training, gradually adjust L a When the value of is taken so that it reaches the minimum value, it can ensure that the distance from each sample text on the hyperplane of each category in the N preset categories to its center point tends to be consistent, that is, it ensures that the distance from each sample text in each category to its center point is equal to the average value. This formula is equivalent to the distance from each sample text to the center point in the category being equal. The reason why the first optimization function L described in formula (1) is used is a The purpose of performing calculations is also to parallelize the calculations and improve the efficiency.

[0076] Furthermore, it should be noted that, during model training, this application hopes that all data in each of the N preset categories can be evenly distributed on the hypersphere of the category (e.g. Figure 3 As shown in the left figure), rather than converging in a small area on the hypersphere (as shown in the left figure), Figure 3 This is because when all samples of a category tend to be in a small range, their representation in the feature space will be extremely compact, which reduces the model's feature representation and, in turn, reduces the model's recognition performance and accuracy.

[0077] Therefore, when using the first optimization function L described in the above formula (1) a While ensuring that the distance between each sample text on the hyperplane of each category in the N preset categories and its center point tends to be consistent, this application also uses a second optimization function to ensure that the distance between each pair of sample text on the hyperplane of each category in the N preset categories is as far as possible to improve classification accuracy. The values of the second optimization function are as follows:

[0078]

[0079] Among them, L b represents the second optimization function; N represents the total number of preset categories; M represents the total number of sample texts; x ij represents the hyperplane feature vector of the i-th sample belonging to the j-th preset category; x i'j Represents the hyperplane feature vector of the i'th sample belonging to the j'th preset category.

[0080] Then, the second optimization function L described in the above formula (2) is used b During training, gradually adjust L b When the value of is set to the largest possible value, the distance between the two sample texts on the hyperplane of each category in the N preset categories can be guaranteed to be as far as possible. The reason why the second optimization function L described in formula (2) is used is bThe purpose of performing calculations is to perform parallel calculations more simply and improve computing efficiency.

[0081] Furthermore, the first optimization function L a and the second optimization function L b By integrating, we can get the overall optimization function. The specific calculation formula is as follows:

[0082] L=L a -L b (3)

[0083] In this way, by conducting multiple rounds of model training, the labeled training data (i.e., the sample text extracted in each round) is continuously iterated using back propagation until the value of the overall optimization function L in the above formula (3) no longer decreases (reaches the minimum value) or the classification accuracy no longer increases, then the updating of the model parameters is stopped, the training of the text classification model is completed, and a trained text classification model is generated.

[0084] Furthermore, it should be noted that, in the process of training the text classification model, the first optimization function L a and the second optimization function L b Continuous adjustment not only updates the parameters of the classification model, but also updates the hyperplane feature vectors of the center points of the hyperdimensional space of the N preset categories. After the model training is completed, the optimized hyperplane feature vectors corresponding to the center points of each category in the N preset categories can be obtained for executing the subsequent step S103. At the same time, the maximum distance between the sample text of each category and the center point within the hypersphere can be used as the radius of the hypersphere where the category is located, for executing the subsequent step S104.

[0085] On this basis, after the text classification model is generated based on the sample text training, the generated text classification model can be further verified using the verification text. The specific verification process can include the following steps (1)-(3):

[0086] Step (1): Get the verification text.

[0087] In this embodiment, in order to verify the text classification model, it is first necessary to obtain verification texts. For example, 1,000 news texts can be collected through online channels as verification texts. The verification texts refer to text information that can be used to verify the text classification model. After obtaining these verification texts and the classification labels corresponding to each verification text, the subsequent step (2) can be continued.

[0088] Step (2): Input the verification text into the text classification model to obtain the classification result corresponding to the verification text.

[0089] After obtaining the verification text through step (1), the verification text can be further input into the text classification model to obtain the hyperplane feature vector corresponding to the verification text, and then the classification result corresponding to the verification text is obtained based on the hyperplane feature vector to execute the subsequent step (3).

[0090] Step (3): When the classification result of the verification text is inconsistent with the classification label result corresponding to the verification text, the verification text is used as a sample text again to update the text classification model.

[0091] After obtaining the classification result of the verification text through step (2), if the classification result of the verification text is inconsistent with the actual classification result corresponding to the verification text (such as the manually annotated classification label result), the verification text can be used as a sample text again to update the parameters of the text classification model.

[0092] Through the above embodiments, the text classification model can be effectively verified using verification text. When the classification result of the verification text is inconsistent with the actual classification result corresponding to the text (such as the manually annotated classification labeling result), the text classification model can be adjusted and updated in time, which helps to improve the recognition precision and accuracy of the classification model.

[0093] S103: Calculating the Euclidean distances between the hyperplane feature vector corresponding to the target text and the hyperplane feature vectors corresponding to N preset category center points respectively; wherein N is a positive integer greater than 0.

[0094] In this embodiment, the target text is input into a pre-built text classification model through step S102. After the hyperplane feature vector corresponding to the target text is identified, the Euclidean distance between the hyperplane feature vector corresponding to the target text and the hyperplane feature vectors corresponding to the N preset category center points can be further calculated to execute the subsequent step S104.

[0095] S104: taking the preset category corresponding to the minimum Euclidean distance among the obtained N Euclidean distances as the target category, and determining the size relationship between the minimum Euclidean distance and the radius of the hypersphere where the target category is located to obtain a determination result.

[0096] In this embodiment, after calculating the N Euclidean distances between the hyperplane feature vector corresponding to the target text and the hyperplane feature vectors corresponding to the center points of N preset categories respectively through step S103, the preset category corresponding to the minimum Euclidean distance can be further selected as the target category. It can be seen that the target category is the category closest to the target text, and then the relationship between the minimum Euclidean distance and the radius of the hypersphere where the target category is located can be further judged to obtain a judgment result for executing the subsequent step S105.

[0097] The judgment result may be that the minimum Euclidean distance is not greater than the radius of the hypersphere where the target category is located; or, the minimum Euclidean distance is greater than the radius of the hypersphere where the target category is located.

[0098] S105: Classify the target text according to the judgment result to obtain a classification result of the target text.

[0099] In this embodiment, when the judgment result is that the minimum Euclidean distance is not greater than the radius of the hypersphere where the target category is located, it can be determined that the target text is located within the hypersphere where the target category is located, and thus it can be determined that the target text belongs to the target category, that is, the classification result that the target text belongs to the target category is obtained.

[0100] Alternatively, when the judgment result is that the minimum Euclidean distance is greater than the radius of the hypersphere where the target category is located, it can be determined that the target text is not within the hypersphere where the target category is located, but the target category is already the category closest to the target text, so it can be determined that the target text belongs to the out-of-domain category, that is, the classification result of the target text belonging to the out-of-domain category is obtained. It can be seen that the classification judgment of text data of the out-of-domain category in this application no longer requires human intervention and adjustment (such as manually setting thresholds, etc.), and the subsequent online learning process can also be completed fully automatically, thereby improving the accuracy of the text classification results.

[0101] Furthermore, after obtaining the classification result of the target text, an optional implementation method is to use the target text and the classification result of the target text to update the text classification model to improve the classification accuracy of the text classification model.

[0102] In addition, it should be noted that if the model is iterated after each new target text is classified, it may result in an increase in equipment costs, such as hardware facilities and power consumption. Therefore, it is possible to wait until each category has accumulated a certain amount of text data before performing online batch updates. For example, when two thousand text data are collected online for each category, the model can be adaptively optimized online iteratively. Alternatively, two thousand data can be randomly sampled from different categories at regular intervals to perform adaptive optimization on the text classification model.

[0103] In summary, the present embodiment provides a text classification method, which first obtains the target text to be classified, and inputs the target text into a pre-built text classification model to identify the hyperplane feature vector corresponding to the target text; then respectively calculates the Euclidean distance between the hyperplane feature vector corresponding to the target text and the hyperplane feature vector corresponding to the center point of N preset categories; wherein N is a positive integer greater than 0, and then takes the preset category corresponding to the minimum Euclidean distance among the obtained N Euclidean distances as the target category, and judges the size relationship between the minimum Euclidean distance and the radius of the hypersphere where the target category is located to obtain a judgment result, and then can classify the target text according to the judgment result to obtain the classification result of the target text. It can be seen that since the embodiment of the present application uses the hyperplane feature vector of the target text output by the pre-built text classification model to calculate the distance with each preset category center point, the target text is judged to belong to the category if it is closest to the center point and the distance within the category hypersphere is not greater than the category hypersphere radius, otherwise it belongs to the out-of-domain category, so that when performing text classification, human intervention is no longer required, thereby improving the accuracy of the text classification results.

[0104] Second embodiment

[0105] This embodiment will introduce a text classification device. For related content, please refer to the above method embodiment.

[0106] See also Figure 5 , is a schematic diagram of the composition of a text classification device provided in this embodiment, the device 500 includes:

[0107] A first acquiring unit 501 is used to acquire a target text to be classified;

[0108] The recognition unit 502 is configured to input the target text into a pre-built text classification model to identify and obtain a hyperplane feature vector corresponding to the target text;

[0109] A calculation unit 503 is configured to calculate the Euclidean distances between the hyperplane feature vector corresponding to the target text and the hyperplane feature vectors corresponding to N preset category center points, where N is a positive integer greater than 0;

[0110] A judgment unit 504 is configured to determine a preset category corresponding to a minimum Euclidean distance among the obtained N Euclidean distances as a target category, and determine a magnitude relationship between the minimum Euclidean distance and a radius of a hypersphere where the target category is located, to obtain a judgment result;

[0111] The classification unit 505 is configured to classify the target text according to the judgment result to obtain a classification result of the target text.

[0112] In one implementation of this embodiment, the apparatus further includes:

[0113] A second acquiring unit, configured to acquire a sample text;

[0114] The training unit is used to train the initial text classification model using the sample speech and the first optimization function and the second optimization function to obtain the text classification model.

[0115] In one implementation of this embodiment, the sample text belongs to the N preset categories; the first optimization function is used to ensure that the distance from each sample text on the hyperplane of each category in the N preset categories to its center point tends to be consistent; the second optimization function is used to ensure that the distance between each pair of sample texts on the hyperplane of each category in the N preset categories is as far as possible.

[0116] In an implementation of this embodiment, the hyperplane feature vectors corresponding to the N preset category center points are obtained after training the text classification model using the first optimization function and the second optimization function.

[0117] In one implementation of this embodiment, the apparatus further includes:

[0118] A third obtaining unit is used to obtain a verification text;

[0119] An input unit, configured to input the verification text into the text classification model to obtain a classification result corresponding to the verification text;

[0120] The first updating unit is configured to update the text classification model by reusing the verification text as the sample text when the classification result of the verification text is inconsistent with the classification label result corresponding to the verification text.

[0121] In one implementation of this embodiment, the classification unit 505 is specifically configured to:

[0122] When the judgment result is that the minimum Euclidean distance is not greater than the radius of the hypersphere where the target category is located, it is determined that the target text belongs to the target category, and a classification result of the target text is obtained.

[0123] Alternatively, when the judgment result is that the minimum Euclidean distance is greater than the radius of the hypersphere where the target category is located, it is determined that the target text belongs to an out-of-domain category, and a classification result of the target text is obtained.

[0124] In one implementation of this embodiment, the apparatus further includes:

[0125] The second updating unit is used to update the text classification model using the target text and the classification result of the target text after classifying the target text according to the judgment result to improve the classification accuracy of the text classification model.

[0126] Furthermore, an embodiment of the present application also provides a text classification device, comprising: a processor, a memory, and a system bus;

[0127] The processor and the memory are connected via the system bus;

[0128] The memory is used to store one or more programs, and the one or more programs include instructions. When the instructions are executed by the processor, the processor executes any one of the implementation methods of the above-mentioned text classification method.

[0129] Furthermore, an embodiment of the present application also provides a computer-readable storage medium, in which instructions are stored. When the instructions are executed on a terminal device, the terminal device executes any one of the implementation methods of the above-mentioned text classification method.

[0130] Furthermore, an embodiment of the present application also provides a computer program product, which, when running on a terminal device, enables the terminal device to execute any one of the implementation methods of the above-mentioned text classification method.

[0131] Through the description of the above embodiments, it can be seen that those skilled in the art can clearly understand that all or part of the steps in the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network communication device such as a media gateway, etc.) to execute the methods described in each embodiment of the present application or certain parts of the embodiments.

[0132] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the methods.

[0133] It should also be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

[0134] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present application. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application is not limited to the embodiments shown herein, but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A text classification method, characterized in that: include: Get the target text to be classified; Inputting the target text into a pre-built text classification model to identify and obtain a hyperplane feature vector corresponding to the target text; Calculating the Euclidean distances between the hyperplane feature vector corresponding to the target text and the hyperplane feature vectors corresponding to N preset category center points, respectively; wherein N is a positive integer greater than 0; The preset category corresponding to the minimum Euclidean distance among the obtained N Euclidean distances is used as the target category, and the relationship between the minimum Euclidean distance and the radius of the hypersphere where the target category is located is determined to obtain a determination result; Classifying the target text according to the judgment result to obtain a classification result of the target text; The text classification model is constructed as follows: Get sample text; Using the sample text, the first optimization function, and the second optimization function, an initial text classification model is trained to obtain the text classification model; The sample text belongs to the N preset categories; the first optimization function is used to ensure that the distance between each sample text and its center point on the hyperplane of each category in the N preset categories tends to be consistent; the second optimization function is used to ensure that the distance between each two sample texts on the hyperplane of each category in the N preset categories is as far as possible; The hyperplane feature vectors corresponding to the N preset category center points are obtained after training the text classification model using the first optimization function and the second optimization function.

2. The method according to claim 1, characterized in that The method further comprises: Get verification text; Inputting the verification text into the text classification model to obtain a classification result corresponding to the verification text; When the classification result of the verification text is inconsistent with the classification label result corresponding to the verification text, the verification text is used as the sample text again to update the text classification model.

3. The method according to claim 1, characterized in that The target text is classified according to the judgment result to obtain the classification result of the target text, including: When the judgment result is that the minimum Euclidean distance is not greater than the radius of the hypersphere where the target category is located, determining that the target text belongs to the target category, and obtaining a classification result of the target text; Alternatively, when the judgment result is that the minimum Euclidean distance is greater than the radius of the hypersphere where the target category is located, it is determined that the target text belongs to an out-of-domain category, and a classification result of the target text is obtained.

4. The method according to claim 1, wherein After classifying the target text according to the judgment result to obtain the classification result of the target text, the method further includes: The text classification model is updated using the target text and the classification result of the target text to improve the classification accuracy of the text classification model.

5. A text classification device, characterized in that: include: A first acquisition unit, configured to acquire a target text to be classified; A recognition unit, configured to input the target text into a pre-built text classification model and identify and obtain a hyperplane feature vector corresponding to the target text; a calculation unit, configured to respectively calculate the Euclidean distances between the hyperplane feature vector corresponding to the target text and the hyperplane feature vectors corresponding to N preset category center points; wherein N is a positive integer greater than 0; a judgment unit, configured to take a preset category corresponding to a minimum Euclidean distance among the obtained N Euclidean distances as a target category, and to determine a magnitude relationship between the minimum Euclidean distance and a radius of a hypersphere where the target category is located, to obtain a judgment result; A classification unit, configured to classify the target text according to the judgment result to obtain a classification result of the target text; The device further comprises: A second acquiring unit, configured to acquire a sample text; A training unit, configured to train an initial text classification model using the sample text, the first optimization function, and the second optimization function to obtain the text classification model; The sample text belongs to the N preset categories; the first optimization function is used to ensure that the distance between each sample text and its center point on the hyperplane of each category in the N preset categories tends to be consistent; the second optimization function is used to ensure that the distance between each two sample texts on the hyperplane of each category in the N preset categories is as far as possible; The hyperplane feature vectors corresponding to the N preset category center points are obtained after training the text classification model using the first optimization function and the second optimization function.

6. A text classification device, characterized in that include: Processor, memory, system bus; The processor and the memory are connected via the system bus; The memory is configured to store one or more programs, wherein the one or more programs include instructions, and when the instructions are executed by the processor, the processor is enabled to perform the method according to any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that The computer-readable storage medium stores instructions, and when the instructions are executed on a terminal device, the terminal device executes the method according to any one of claims 1 to 4.

Images