Text classification model training method, text classification method and related device

Abstract

The present application discloses a text classification model training method, a text classification method, and related devices. The text classification model training method includes: obtaining multiple text samples, as well as classification labels and text feature complexity corresponding to each text sample, dividing the multiple text samples into multiple training sample groups according to the text feature complexity, determining embedding dimension values ​​corresponding to the training sample groups based on the text feature complexity, sorting the multiple training sample groups in ascending order of embedding dimension values, and training a text classification model using each text sample and its corresponding classification label in each training sample group in sequence according to the sorting of the multiple training sample groups to obtain a trained text classification model. The present application is conducive to improving the model training speed and improving the model training effect.

Description

Technical Field

[0001] The present application relates to the field of natural language processing technology, and in particular to a text classification model training method, a text classification method, and related devices. Background Art

[0002] With the development of computer hardware, more and more neural networks have been designed. By selecting training samples and combining flexible training methods, models can be trained for various practical scenarios.

[0003] With the growing demand for model applications, the structure of the model is becoming more and more complex, which leads to a significant increase in the time cost required for training. Models with complex structures often require more time and hardware resources to perform training. Once the training time is shortened or the training process is simplified, the trained model will often have poor results. How to improve the training speed of the text classification model and improve the training effect of the model is the technical problem to be solved by this application. Summary of the Invention

[0004] The purpose of the embodiments of the present application is to provide a text classification model training method, which is conducive to improving the model training speed and improving the model training effect.

[0005] In a first aspect, a text classification model training method is provided, comprising:

[0006] Acquire multiple text samples, as well as a classification label and text feature complexity corresponding to each text sample, wherein the classification label is used to indicate the classification category to which the text sample belongs, and the text feature complexity is used to indicate the complexity of the features of the text sample;

[0007] Dividing the plurality of text samples into a plurality of training sample groups according to the text feature complexity, wherein each training sample group includes at least two text samples, and the range of the text feature complexity corresponding to the at least two text samples is less than a preset range;

[0008] Determining embedding dimension values ​​corresponding to the plurality of training sample groups based on the text feature complexity, wherein the embedding dimension values ​​are positively correlated with the text feature complexity of each text sample in the corresponding training sample group;

[0009] The multiple training sample groups are sorted in ascending order of embedding dimension values, and according to the sorting of the multiple training sample groups, the text classification model is trained in turn using each text sample in each training sample group and its corresponding classification label to obtain a trained text classification model, and the trained text classification model is used to perform classification prediction on the text to be classified; wherein, when using the target training sample group in the multiple training sample groups to perform training, the dimension of the text feature embedding layer in the text classification model is set to the embedding dimension value corresponding to the target training sample group, and the text feature embedding layer is used to convert the input text sample into a text feature vector, and the number of dimensions of the text feature vector is the embedding dimension value.

[0010] In a second aspect, a text classification method is provided, comprising:

[0011] Get the text to be classified;

[0012] The text to be classified is input into a trained text classification model to obtain a classification result predicted by the trained text classification model for the text to be classified, wherein the trained text classification model is a text classification model obtained by the text classification model training method described in the first aspect.

[0013] In a third aspect, a text classification model training device is provided, comprising:

[0014] An acquisition module acquires a plurality of text samples, and a classification label and text feature complexity corresponding to each text sample, wherein the classification label is used to indicate the classification category to which the text sample belongs, and the text feature complexity is used to indicate the complexity of the features possessed by the text sample;

[0015] a division module, dividing the plurality of text samples into a plurality of training sample groups according to the text feature complexity, wherein each training sample group includes at least two text samples, and the range of the text feature complexity corresponding to the at least two text samples is less than a preset range;

[0016] A determination module, which determines embedding dimension values ​​corresponding to the plurality of training sample groups based on the text feature complexity, wherein the embedding dimension values ​​are positively correlated with the text feature complexity of each text sample in the corresponding training sample group;

[0017] A training module sorts the multiple training sample groups in ascending order of embedding dimension values, and according to the sorting of the multiple training sample groups, uses each text sample in each training sample group and its corresponding classification label to train the text classification model in turn to obtain a trained text classification model, and the trained text classification model is used to perform classification prediction on the text to be classified; wherein, when using a target training sample group among the multiple training sample groups to perform training, the dimension of the text feature embedding layer in the text classification model is set to the embedding dimension value corresponding to the target training sample group, and the text feature embedding layer is used to convert the input text sample into a text feature vector, and the number of dimensions of the text feature vector is the embedding dimension value.

[0018] In a fourth aspect, a text classification device is provided, comprising:

[0019] Get module to get the text to be classified;

[0020] The classification module inputs the text to be classified into a trained text classification model to obtain a classification result predicted by the trained text classification model for the text to be classified, wherein the trained text classification model is a text classification model obtained by the text classification model training method described in the first aspect.

[0021] In a fifth aspect, an electronic device is provided, which includes a processor, a memory, and a computer program stored in the memory and runnable on the processor. When the computer program is executed by the processor, the steps of the text classification model training method described in the first aspect are implemented, or the steps of the text classification method described in the second aspect are implemented.

[0022] In a sixth aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the computer program implements the steps of the text classification model training method as described in the first aspect, or executes the steps of the text classification method as described in the second aspect.

[0023] In an embodiment of the present application, first, a plurality of text samples, as well as a classification label and text feature complexity corresponding to each text sample are obtained, wherein the classification label is used to indicate the classification category to which the text sample belongs, and the text feature complexity is used to indicate the complexity of the features possessed by the text sample. Then, the plurality of text samples are divided into a plurality of training sample groups according to the text feature complexity, wherein each training sample group includes at least two text samples, and the range of the text feature complexity corresponding to the at least two text samples is less than a preset range. Subsequently, the embedding dimension value corresponding to the training sample group is determined based on the text feature complexity, and the embedding dimension value is positively correlated with the text feature complexity of each text sample in the corresponding training sample group. Finally, multiple training sample groups are sorted in order of embedding dimension values ​​from small to large, and the text classification model is trained using each text sample in each training sample group and its corresponding classification label in turn to obtain a trained text classification model, which is used to perform classification prediction on the text to be classified; wherein, when using a target training sample group among multiple training sample groups for training, the dimension of the text feature embedding layer in the text classification model is set to the embedding dimension value corresponding to the target training sample group, and the text feature embedding layer is used to convert the input text sample into a text feature vector, and the number of dimensions of the text feature vector is the embedding dimension value. The embedding dimension value determined by this scheme is positively correlated with the complexity of the text feature. Therefore, the model is trained in order of embedding dimension values ​​from small to large, that is, the training is performed in order of text samples from low to high feature complexity, so that the text classification model learns sample features from easy to difficult. Among them, the text feature embedding layer is used to convert text samples into text feature vectors, so that the text classification model can learn the features of the text samples. Since the dimension of the text feature embedding layer is too small, the sample features and text features will be lost, and if it is too long, a longer training time will be required. Therefore, this scheme dynamically adjusts the dimension value of the text feature embedding layer to enable the model to learn text features with appropriate dimensions, so that the model can learn the features of the text samples and shorten the training time, so as to achieve the purpose of improving the model training speed and improving the model training effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0025] Figure 1a This is a flowchart of a text classification model training method according to an embodiment of the present application;

[0026] Figure 1b This is one of the diagrams showing the relationship between the text length and the embedding dimension value of the sample in this article in an embodiment of the present application;

[0027] Figure 1c This is the second diagram of the relationship between the text length and the embedding dimension value of the sample in this article in an embodiment of the present application;

[0028] Figure 1d This is a third diagram of the relationship between the text length and the embedding dimension value of the sample in this article in an embodiment of the present application;

[0029] Figure 1e 1 is a flowchart of an embodiment of the present application for performing segmentation and sorting on multiple text samples to train a BERT model in sequence;

[0030] Figure 2 This is a schematic flow chart of a text classification method according to an embodiment of the present application;

[0031] Figure 3 This is a structural diagram of a text classification model training device according to an embodiment of the present application;

[0032] Figure 4 This is a structural diagram of a text classification device according to an embodiment of the present application;

[0033] Figure 5 It is a structural diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0034] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application. The figure numbers in this application are only used to distinguish the various steps in the scheme and are not used to limit the execution order of the various steps. The specific execution order is subject to the description in the specification.

[0035] To meet complex application requirements, larger neural network models may consist of dozens or even hundreds of layers. Model training requires tens of thousands of floating-point calculations per layer, resulting in a massive computational load. Current hardware developments struggle to fully meet the computational demands of neural networks, making some larger neural network models impractical due to prohibitive training costs, remaining limited to theoretical experiments. To improve model training efficiency, optimization efforts can be made in various areas, including data optimization and training speed.

[0036] Data optimization methods, such as data augmentation, active learning, prompted learning, and small-sample learning, can all improve model recognition performance to a certain extent. For example, classic data augmentation methods such as EDA (Exploratory Data Analysis), AEDA (Easier Data Augmentation), back-translation, and adversarial learning enhance the original data. While these methods can improve model recognition performance while keeping other conditions unchanged, they typically increase training time and resource expenditures.

[0037] Furthermore, to improve model training speed, techniques such as mixed-precision training, gradient accumulation, freezing certain network layers, and adjusting the batch size can all be used to increase model training speed to a certain extent. Taking mixed-precision training as an example, the inference and training of neural networks involve a large number of decimal calculations. These decimals are represented as 32-bit floating-point numbers to fully utilize the processor's floating-point processors for faster computation. While compressing 32-bit floating-point numbers to 16 bits will result in a certain loss of representation accuracy, it will significantly improve both parameter storage space and computational complexity. While other conditions remain unchanged, these methods can increase model training speed, but they often lead to reduced model effectiveness.

[0038] It can be seen from this that in the field of model training, it is often difficult to improve the training speed while ensuring the model training effect. There are problems such as slow model training speed and poor model training effect, and it is difficult to improve the model training efficiency.

[0039] In order to solve the problems existing in the prior art, the present invention provides a text classification model training method, such as Figure 1a Shown, including:

[0040] S11: Acquire multiple text samples, as well as a classification label and text feature complexity corresponding to each text sample, wherein the classification label is used to indicate the classification category to which the text sample belongs, and the text feature complexity is used to indicate the complexity of the features of the text sample.

[0041] The solution provided by this application relates to the field of natural language processing technology, and the trained text classification model can be used to classify text. In this step, the text sample is a sequential text with an order, and the text sample can specifically include one or more sentences.

[0042] The classification label corresponding to a text sample is used to express the classification category to which the corresponding text sample belongs. For example, the classification label is used to express the semantics of the corresponding text sample, or the classification label is used to express the emotion of the corresponding text sample. In practical applications, classification labels can be pre-set based on the actual application scenario, so that the trained text classification model can be used in classification scenarios that match the classification labels.

[0043] Among them, the text feature complexity is used to express the complexity of the features possessed by the corresponding text sample. For example, the text feature complexity can express the complexity of the corresponding text in terms of length, semantic content, sentence structure and other aspects, which can be specifically reflected in the form of a numerical value. Generally speaking, the lower the text feature complexity, the easier it is to be learned by the model. The text feature complexity corresponding to the text sample in this solution can be manually annotated in advance, or it can be calculated and generated according to a pre-set rule based on at least one feature of the text sample.

[0044] Based on the solution provided in the above embodiment, optionally, the text feature complexity can be determined according to the text features of the corresponding text sample, and the text features include at least one of the following: text length, text content complexity, text deviation degree, and text confidence.

[0045] Among them, text length can indicate the number of characters in the corresponding text sample. Text content complexity can indicate the types of words in the corresponding text sample. Text deviation degree can indicate the similarity between the corresponding text sample and other text samples. Text confidence can indicate the difficulty of the corresponding text sample for model training.

[0046] For example, suppose there are N text samples, denoted as text_1, text_2, ..., text_N. The corresponding sample categories expressed by the classification labels are m, namely c1, c2, ..., c m The lengths of N text samples are l1, l2, ..., l N , the text content complexity is μ1, μ2, ..., μ N , the text deviation values ​​are d1, d2, ..., d N , the text confidence values ​​are p1, p2, ..., p N The text samples and their corresponding text features can be stored in the following table format:

[0047]

[0048] Among them, text length, text content complexity and text confidence are positively correlated with text feature complexity, while text deviation degree is negatively correlated with text feature complexity.

[0049] In this step, the text feature complexity can be determined based on a combination of one or more of the above-mentioned text features. For example, if the text feature complexity is determined based on the text length, the text length of the text sample can be directly used as the text feature complexity. The longer the text, the greater the corresponding text feature complexity. If the text feature complexity is determined based on multiple text features, a weighted calculation can be performed on the multiple text features based on preset weights, or other statistical methods can be used to summarize the multiple text features to obtain the text feature complexity corresponding to the text sample.

[0050] Optionally, the text feature complexity is determined for multiple text samples in the same manner, for example, the text feature complexity is determined for multiple text samples based on text length and text content complexity, or the text feature complexity is determined for multiple samples based on text confidence and text deviation.

[0051] S12: Divide the multiple text samples into multiple training sample groups according to the text feature complexity, wherein each training sample group includes at least two text samples, and the range of the text feature complexity corresponding to the at least two text samples is less than a preset range.

[0052] In this step, multiple text samples are grouped according to the text feature complexity to obtain multiple training sample groups, which can be represented as batches. Among them, the range of the text feature complexity of the text samples in any training sample group is less than the preset range. The range can be used to represent the amount of variation in statistical data, usually referring to the difference between the maximum and minimum values. In this solution, it refers to the difference between the maximum value of the text feature complexity and the minimum value of the text feature complexity, which is used to express the difference in training difficulty of each text sample in the training sample group to which it belongs.

[0053] The text feature complexity of multiple text samples in any training sample group obtained by the solution provided in the embodiment of the present application is relatively small, that is, the difficulty of the model learning multiple text samples belonging to the same training sample is similar.

[0054] S13: Determine embedding dimension values ​​corresponding to the plurality of training sample groups based on the text feature complexity, wherein the embedding dimension value is positively correlated with the text feature complexity of each text sample in the corresponding training sample group.

[0055] Since the text samples belonging to the same group determined in step S12 above have little difference in training difficulty, in this step, for any training sample group, the embedding dimension value suitable for the training sample group can be determined based on multiple text samples with similar training difficulty. Generally speaking, text samples with higher feature complexity require larger embedding dimension values ​​to accommodate text features. Based on this, the embedding dimension value determined in this step is positively correlated with the text feature complexity of the corresponding training sample group to determine the embedding dimension value that reasonably accommodates the text features.

[0056] The embedding dimension value is used to set the dimension of the model's embedding layer during model training. The dimension of this embedding layer is closely related to the features that the model can learn. For example, if the embedding layer dimension is set too small, features of some dimensions will be lost during model training, resulting in poor model learning results. On the other hand, if the embedding layer dimension is set too large, while it can ensure that all features of the training samples are included, it will make model training take too long, resulting in slow training speed and low training efficiency.

[0057] In an embodiment of the present application, multiple text samples are first grouped according to the complexity of text features, and then embedding dimension values ​​are determined for text samples with similar training difficulty that belong to the same group. In this way, embedding dimension values ​​applicable to multiple text samples in the entire training sample group can be determined, so that the determined embedding dimension values ​​can accommodate the characteristics of the text samples and improve training efficiency.

[0058] The following example illustrates this solution. The embedding dimension can be expressed as the embedding value, and the text feature complexity is the text length. For example, suppose a text sample is "Today's weather is nice, perfect for a trip." The length of this text sample, including words and symbols, is 13. During subsequent training, this text is vectorized using the text feature embedding layer. The embedding dimension value represents the dimensionality of the text vectorization.

[0059] For this text sample, if the embedding value is 13, such as Figure 1b As shown in , the dimension size just covers the text length and can retain all the features of the text sample.

[0060] If the embedding value is 256, such as Figure 1c As shown in the figure, although the text length can be covered to retain all the features of the text sample, there is too much redundancy and the training time will increase many times, resulting in too long training time and low training efficiency.

[0061] If the embedding value is 10, such as Figure 1d As shown, the text length cannot be covered, the text will be truncated, and some information will be lost. Although the training speed is improved, the training effect is poor.

[0062] It can be seen that the embedding dimension value is closely related to the model training efficiency. Too large or too small an embedding dimension value will reduce the efficiency of model training.

[0063] Similarly, for a batch containing multiple text samples, assuming that 95% of the samples in a batch have a length less than or equal to a certain value of 128, then the embedding value corresponding to the batch can be set to 128, thereby improving the model training speed while retaining most sample features, thereby achieving the goal of improving model training efficiency.

[0064] S14: Sort the multiple training sample groups in ascending order of embedding dimension values, and according to the sorting of the multiple training sample groups, use each text sample in each training sample group and its corresponding classification label to train the text classification model in turn to obtain a trained text classification model, and the trained text classification model is used to perform classification prediction on the text to be classified; wherein, when using the target training sample group among the multiple training sample groups to perform training, the dimension of the text feature embedding layer in the text classification model is set to the embedding dimension value corresponding to the target training sample group, and the text feature embedding layer is used to convert the input text sample into a text feature vector, and the number of dimensions of the text feature vector is the embedding dimension value.

[0065] Since the embedding dimension value in this scheme is positively correlated with the complexity of text features, in this step, multiple training sample groups are sorted in ascending order of the embedding dimension value, and according to the sorting of the multiple training sample groups, the text classification model is trained using each text sample in each training sample group and its corresponding classification label in turn, which can achieve the effect of training the model using text samples from easy to difficult.

[0066] In this example, the text classification model can be obtained by training a Bidirectional Encoder Representation from Transformers (BERT) model. Figure 1e This diagram shows a flow chart of a text classification model training method based on multiple text samples. By grouping and sorting the acquired text samples, a set of text samples ranked from easy to difficult is obtained. Training the BERT model based on this ranking allows the model to learn features from easy to difficult, thereby optimizing model training results.

[0067] When training with any target training sample group, the model's text feature embedding layer's dimensions are set to the embedding dimension value corresponding to the target training sample group. This allows the text feature embedding layer to learn the text features in the target training sample group while simultaneously increasing the speed and effectiveness of model training. This avoids feature loss due to a text feature embedding layer with too small dimensions, and also avoids excessive training time due to a text feature embedding layer with too large dimensions.

[0068] Based on the solution provided in the above embodiment, optionally, the above step S11, obtaining multiple text samples, and the classification label and text feature complexity corresponding to each text sample, includes:

[0069] S21: Obtain the multiple text samples and the classification label corresponding to each text sample.

[0070] In this solution, we first obtain multiple text samples and the corresponding classification labels for each text sample. Each text sample corresponds to a classification label, and the number of classification labels is often smaller than the number of text samples.

[0071] Optionally, after obtaining text samples and corresponding classification labels, the text samples can be screened based on the classification labels so that the number of text samples corresponding to various classification labels is substantially consistent, thereby ensuring effective training of text samples of each classification.

[0072] S22: Determine a text feature of each text sample according to the multiple text samples.

[0073] Specifically, the text sample can be parsed to obtain one or more text features, such as text length, text content complexity, text deviation, text confidence, and text category. The text sample and its corresponding text features are stored in association, such as in a table, for easy access.

[0074] S23: Determine the text feature complexity corresponding to each text sample according to the text feature of each text sample.

[0075] In this step, the text feature complexity of each text sample is determined based on the one or more text features determined in the above steps. Optionally, a weighted summation of multiple text features can be performed based on preset weights to obtain the text feature complexity.

[0076] Based on the solution provided in the above embodiment, optionally, the text feature includes text content complexity;

[0077] The above step S22, determining the text features of each text sample based on the multiple text samples, includes:

[0078] S31: Based on a sliding window of a preset length, sliding interception is performed on a first target sample in sequence with a preset step size to obtain a plurality of byte segments of a preset length, where the first target sample is any one of the plurality of text samples.

[0079] This step can be implemented using the N-Gram model, a statistical language-based model algorithm. Specifically, a sliding window of a preset length is used to slide and intercept the text sample to obtain multiple byte segments of equal length. The preset length of the sliding window and the preset step size used for sliding can be pre-set according to actual needs.

[0080] The intercepted byte segments can be called gram, and the number of byte segments intercepted for text can be expressed as n_grams(text).

[0081] S32: Deduplication is performed on the plurality of byte segments of the preset length.

[0082] Deduplication is performed based on the contents of the byte segments, so that the contents of each byte segment after deduplication are different from each other.

[0083] For example, suppose the first target sample is "The weather is great today, perfect for an outing." With a preset step size of 1 and a preset length of 2, truncating this first target sample using a sliding window of length 2 yields 13 byte segments: "today," "every day," "weather," and "outing." These byte segments contain two occurrences of "today," and after deduplication, 12 deduplicated byte segments are obtained.

[0084] S33: Calculate the ratio of the number of byte segments after deduplication to the number of byte segments before deduplication, and determine the ratio as the text content complexity of the first target sample.

[0085] The text content complexity μ can be calculated by the following formula:

[0086]

[0087] Among them, "n_grams(text) deduplication" represents the number of byte segments after deduplication, and "n_grams(text)" represents the number of byte segments before deduplication. The text content complexity calculated by the solution provided in the embodiment of the present application can represent the repetition of characters in the corresponding text sample. For a text sample, the less repeated content, the greater the corresponding text content complexity. Correspondingly, the larger the numerical value of the text content complexity, the higher the complexity of the corresponding text sample, that is, the more difficult it is to be learned by the model. Through the solution provided in the embodiment of the present application, the text content complexity can be generated based on the content of the text, accurately expressing the difficulty of the corresponding text sample being learned by the model.

[0088] Based on the solution provided in the above embodiment, optionally, the text feature includes a text deviation degree;

[0089] In the above step S22, determining the text features of each text sample based on the multiple text samples includes:

[0090] S41: Determine text similarities between a second target sample and other text samples in the multiple text samples to obtain multiple text similarities, where the second target sample is any one text sample in the multiple text samples.

[0091] In this step, the similarity between the second target sample and each of the other text samples is determined. For example, assuming there are 100 text samples, in this step, the text similarity between one of the second target samples and the other 99 text samples is determined. Text similarity can indicate whether the second target sample is similar to the other text samples.

[0092] In practical applications, an appropriate method can be selected to calculate text similarity based on actual needs. For example, the similarity between text samples can be identified based on semantics. For example, the second target sample and other text samples can be input into a pre-trained text similarity recognition model, and the text similarity between the second target sample and the other input text samples can be determined based on the model output results. Alternatively, the text features of the second target sample and other text samples can be compared and calculated based on preset rules to determine text similarity.

[0093] S42: Determine an average value of the multiple text similarities as the text deviation degree of the second target sample.

[0094] In this step, an average value is calculated based on the multiple text similarities determined for the second target sample in the above step. Based on the above example, after 99 text similarities are determined, the average value of the 99 text similarities is calculated in this step as the text deviation degree of the second target sample.

[0095] In practical applications, the text deviation degree is determined for each of the plurality of text samples. The deviation degree values ​​corresponding to the plurality of text samples text_1, text_2, ..., text_N can be expressed as d1, d2, ..., d N .

[0096] The degree of deviation described in the examples of this application can also be called the degree of deviation. The degree of deviation is used to express the similarity between the corresponding text sample and other text samples. The greater the degree of deviation, the less similar it is. In the field of model training, the greater the difference between multiple training samples, the more difficult it is to be learned by the model. Through the solution provided by the embodiments of this application, the degree of deviation of each text sample can be determined based on the similarity between the text samples. The determined degree of deviation can objectively reflect the difficulty of the corresponding text sample being learned by the model.

[0097] Based on the solution provided in the above embodiment, optionally, the text similarity includes at least one of the following: Euclidean distance, cosine distance, edit distance, and Jaccard similarity.

[0098] In this solution, text can be first processed into numerical form. For example, text can be expressed as feature vectors, and then the distance between texts can be calculated based on the numerical form. Euclidean distance, also known as Euclidean distance, can be used to express text similarity in this solution by calculating the Euclidean distance between text vectors. Cosine distance, also known as cosine similarity, uses the cosine of the angle between two vectors in vector space as a measure of the difference between two individuals.

[0099] If you don't convert the text into numerical form, you can directly calculate the similarity between texts. Specifically, you can calculate the edit distance or Jaccard similarity as a measure of text similarity. Edit distance, also known as Levenshtein distance, is a quantitative measure of the degree of difference between strings. It is measured by the minimum number of operations required to transform one string into another. Jaccard similarity is the number of words in the intersection of two texts divided by the number of words in the union. Jaccard similarity is independent of the position or order of the text.

[0100] In practical applications, there are many ways to measure text similarity, and the method to be used can be selected according to actual needs. It is also possible to use multiple methods to calculate the distance between texts, and then summarize the distances between texts determined by multiple methods to determine text similarity.

[0101] Based on the solution provided in the above embodiment, optionally, the text feature includes text confidence.

[0102] The above step S22, determining the text features of each text sample based on the multiple text samples, includes:

[0103] S51: Using the multiple text samples, a preset basic text classification model is trained in a cross-validation manner to obtain a trained basic text classification model.

[0104] The preset basic text classification model described in this program can for example be a logistic regression model (Logisticregression, LR), a support vector machine (SVM) model, a bidirectional encoder representation from a transformer (BERT) model, etc. In this step, the above-mentioned preset basic text classification model is trained using the K-fold cross validation method. K-fold cross validation refers to dividing a plurality of the text samples into K groups, and each subset data is respectively done a verification set, and the remaining K-1 groups of subset data are used as training sets for training the preset basic text classification model to obtain the basic text classification model after training.

[0105] S52: Input a third target sample from the multiple text samples into the trained basic text classification model to obtain a text confidence of the third target sample, wherein the text confidence is the classification probability obtained after the trained basic text classification model performs classification prediction on the input third target text sample, and the third target sample is any one text sample from the multiple text samples.

[0106] In this step, the above-mentioned third target sample is input into the trained basic text classification model, so that the trained basic text classification model performs prediction on the category to which the third target sample belongs, and simultaneously outputs the probability that the third target sample belongs to the category, that is, the confidence level, and the confidence level output by the model is used as the confidence level of the third target sample.

[0107] In practical applications, the text confidences of the multiple text samples are determined respectively. The confidences of the multiple text samples text_1, text_2, ..., text_N can be expressed as p1, p2, ..., p N The larger the confidence value, the easier it is to train and identify samples.

[0108] Based on the solution provided in the above embodiment, optionally, the text features include text length l, text content complexity μ, text deviation degree d, and text confidence p.

[0109] The above step S23, determining the text feature complexity corresponding to each text sample according to the text feature of each text sample, includes:

[0110] S61: Perform weighted summation on the text length, the text content complexity, the text deviation degree and the text confidence based on preset weights to obtain the text feature complexity corresponding to the text sample.

[0111] In practical applications, the following formula can be used to calculate the text feature complexity γ corresponding to the text sample: i :

[0112]

[0113] Among them, l i represents the text length of the i-th text sample, μ i represents the text content complexity of the i-th text sample, d i Indicates the text deviation degree of the i-th text sample, p i represents the text confidence of the i-th text sample, l max represents the maximum length of the text of the plurality of text samples, d max represents the maximum value of the text deviation degree of the multiple text samples, α l Represents the preset weight of text length, α μ Indicates the preset weight of text content complexity, α d Indicates the preset weight of text deviation, α p Indicates the preset weight of text confidence.

[0114] Where i is a positive integer. Assuming that the total number of text samples is N, then l max represents l1, l2, ..., l N The maximum value in Indicates l i Perform normalization. max is d1, d2, ..., d N The maximum value in Indicates d i Perform normalization processing.

[0115] α l , α μ , α d , α p The values ​​of α in the present invention can be set to 0.5, 0.1, 0.1, and 0.3 respectively. The weight can be adjusted according to the actual application results, but it is necessary to satisfy that the sum of the weights of each sample feature is 1, that is, in this example, α l +α μ +α d+α p =1.

[0116] Among them, the relevant calculation of sample length The purpose is to express shorter text samples with smaller values, so that shorter samples can be arranged in the front. Related calculations of sample content complexity, sample deviation degree, and sample confidence The purpose is to express text samples that are easier to train with smaller values, so as to arrange samples with lower training recognition difficulty at the front. Through the solution provided by the embodiment of the present application, the determined text feature complexity γ can accurately represent the training difficulty of the text sample.

[0117] Based on the solution provided in the above embodiment, optionally, the above step S12, dividing the multiple text samples into multiple training sample groups according to the complexity of the text features, includes:

[0118] S71: According to the classification labels corresponding to the multiple text samples, multiple text samples belonging to the same classification category are sorted in ascending order of text feature complexity to obtain multiple text samples sorted in each classification category.

[0119] Based on the solution provided in the above embodiment, since the text feature complexity can accurately characterize the difficulty of the corresponding text sample for training, sorting is performed according to the text feature complexity in this step, and the shorter samples belonging to the same classification category with lower training recognition difficulty are placed in front, and the longer samples with higher training recognition difficulty are placed in the back.

[0120] In this step, sorting is performed based on the classification labels to obtain multiple text samples sorted by classification category. Optionally, multiple text samples are first clustered according to the classification labels, and then multiple text samples corresponding to the same classification label in each category are arranged in order of text feature complexity from smallest to largest, thereby obtaining multiple text samples sorted by classification category.

[0121] S72: For each classification category, the sorted multiple text samples are divided according to the sorting to obtain a preset number of division results, each division result corresponds to a sequence number.

[0122] In this step, it is assumed that the parameter batch_size used in the training model is n. The specific value of n can be a pre-set number, that is, the number of division results to be divided. When the total number of text samples is N, it is necessary to divide N samples into n batches, that is, n training sample groups. Assume that there are m types of classification labels, and the number of samples in each category is represented by q1, q2, ..., q m .

[0123] During actual implementation, for each classification category, the sorted text samples are divided into n parts according to the sort order, so that the training sample group can be constructed in the subsequent steps. For example, suppose a classification category contains 10 text samples, which are sorted from small to large based on the complexity of the text features. The order of arrangement is represented as a, b, c...i, j. Assume that 5 division results are required, that is, n is 5. Then, the 5 division results ab, cd, ef, gh, ij can be obtained by equal division. Then, each division result is sequentially assigned a number. For example, it is set as: ab→1, cd→2, ef→3, gh→4, ij→5.

[0124] S73: Constructing the text samples corresponding to the division results with the same sequence number in each classification category into a training sample group to obtain the preset number of training sample groups.

[0125] In this step, based on the text samples after the above division, the division results with the same sequence number in the m classification categories are combined to form a training sample, thereby obtaining n training sample groups. For example, the text samples corresponding to the division result with the sequence number 1 in each classification category among the m classification categories are constructed as the first training sample group, the text samples corresponding to the division result with the sequence number 2 in each classification category are constructed as the second training sample group, and so on until the text samples corresponding to the division result with the sequence number n in each classification category are constructed as the nth training sample group.

[0126] For example, based on the above example of this solution, assume that the training sample groups corresponding to each batch are represented as c1, c2, ..., c m Then, in the step of constructing the training sample group, the text samples of each category are first divided, and the text samples of each category in the m classification categories are q1, q2, ..., q m , after dividing the text samples of each category into n parts, the division results of the text samples of each category are Then, a sequence number is set based on the order of the division results, and a training sample group is constructed based on the sequence number from the text samples corresponding to the division results.

[0127] The division result is For example, a text sample with sequence number 1 in this category includes 1 to The text sample of the sequence number 2 includes to Text samples, ..., a text sample with sequence number n includes to text samples, therefore, used to construct each batch (i.e., c1, c2, ..., c m ) text samples include the text samples of this category from 1 to Article text sample, to Article sample, ..., to Similarly, each batch contains samples from other categories and is selected in the same way.

[0128] Then, the text samples corresponding to the division results are labeled in order to construct a training sample group. The text samples used to construct c1 include the text samples from the first category q1 to the first category q1. Text samples, the first to text samples, ..., the mth category q m In the text sample, the first to Sample text.

[0129] Accordingly, the text samples used to construct c2 include the first category of text samples to The second category of text samples to text samples, ..., the mth category of text samples to The process is analogous to the above, until m training sample groups are constructed.

[0130] The solution provided by the embodiments of this application can construct multiple training sample groups, each of which contains text samples from various classification categories to improve training effectiveness. Furthermore, this solution selects text samples based on the order of training difficulty from easy to difficult, so that the constructed training sample groups also have characteristics of easy to difficult. Based on this, using the training sample groups in the embodiments of this application to perform model training can effectively improve the effectiveness of model training, effectively improving both model quality and training speed.

[0131] Based on the solution provided in the above embodiment, optionally, the text feature complexity includes text length, and the above step S13, determining the embedding dimension values ​​corresponding to the multiple training sample groups based on the text feature complexity, includes:

[0132] S81: Determine an embedding dimension value of the training sample group according to the text length of each text sample in the training sample group, wherein the text length of a preset proportion of text samples in the training sample group is not greater than the embedding dimension value.

[0133] In this step, text length is determined as text feature complexity. In specific applications, the text lengths of each text sample in each training sample group are pre-calculated, that is, the text lengths of multiple text samples corresponding to each training sample are obtained. Subsequently, the embedding dimension value of the training sample group to which it belongs can be determined based on the text length. In the step of determining the embedding dimension value, an appropriate statistical method can be selected based on actual needs to determine the embedding dimension value.

[0134] For example, the maximum value of multiple text lengths corresponding to any training sample group is determined, and the maximum value is determined as the embedding dimension value of the training sample group to which it belongs.

[0135] For another example, if multiple text lengths in any training sample group are sorted by size and it is determined that a% of the sample lengths are less than a value ω, then this value ω is determined as the embedding dimension value of the training sample group. Specifically, a% can be pre-set according to actual needs. Assuming a% is 95%, then in this step, multiple text lengths can be sorted in ascending order, and the sample lengths of the first 95% of the sorted samples can be determined. The maximum value of these 95% sample lengths is then determined as ω, which is used as the embedding dimension value of the training sample group.

[0136] Through the solution provided by the embodiments of this application, the embedding dimension value determined during the training phase can accommodate the text features of the text samples. In practical applications, the appropriate embedding dimension value can be determined based on the specific length of the text in the training sample group according to actual needs. On the one hand, it can ensure that the model learns the features of the text samples during the training phase, and on the other hand, it can flexibly adjust the training speed according to actual needs, thereby improving the model training efficiency in terms of both model quality and training speed.

[0137] The technical effects of this solution are explained below by comparison.

[0138] Assume that a comparison scheme for comparison uses a random partitioning method to obtain batch samples and uses a fixed embedding dimension value (i.e., embedding value) to perform model training. Another scheme uses the scheme described in any of the above embodiments of this application to perform model training. Then, assuming that the existing training set has a total of 19,080 text samples, and when other training parameters and training conditions are consistent, the training results are compared as follows:

[0139] Experimental plan Precision Training time (h) Comparison plan 0.88 0.56 Solution of the present invention 0.90 0.45

[0140] Compared with the above-mentioned comparative scheme, the accuracy of the technical scheme of the present invention is improved by 2 percentage points and the training time is saved. In practical applications, the greater the difference in length between samples in the training sample set, the more time is saved. This shows that compared with the solution of randomly dividing batches and fixing the embedding layer dimension value, the solution provided by the embodiment of the application can significantly improve the effectiveness of model training.

[0141] The solution provided by the embodiments of this application has good versatility and portability and can be applied to a variety of text classification tasks. Taking the application scenario of text sentiment classification as an example, sentiment polarity can be specifically divided into positive sentiment, negative sentiment, and neutral sentiment. Further subcategorization can be further divided into preference, sadness, disgust, anger, happiness, surprise, fear, etc.

[0142] Taking the "happy" category as an example, scenarios expressing "happiness" include sentences like "Today is my birthday, an unforgettable day," or "I went on an outing today and felt very happy." Sentences expressing "happiness" vary greatly in length, such as "This shopping experience was perfect," and "Today, my beloved teacher, Li, retired. Teacher Li's diligent and meticulous work has earned the respect of all teachers and students. Today, Teacher Li's students are all over the world, cultivating batches of outstanding talents for the development of the motherland. I wish Teacher Li good health and all the best in the future." Furthermore, to cover a wide range of scenarios expressing "happiness," the training sample size is typically very large.

[0143] These factors often lead to long training times and low recognition accuracy for sentiment text classification models. This proposed solution can improve the recognition accuracy and training speed of the model by combining samples and adjusting the embedding dimension values ​​accordingly, while keeping the original dataset unchanged.

[0144] Furthermore, this solution offers excellent compatibility and can be used in conjunction with other data optimization and training acceleration methods during training to further improve training speed and recognition accuracy. Furthermore, this solution is applicable not only to text classification tasks but also to other tasks, demonstrating its versatility and transferability.

[0145] The solution provided in the embodiment of the present application improves the effectiveness of model training in terms of model training speed and model training quality through sample sorting, sample combination, model training and other methods. The improvement of the present solution in terms of recognition accuracy and training rate compared with the above-mentioned comparative solution was verified through the relevant comparative experimental module. Among them, the text samples in each category are sorted by calculating the size of the complexity of the text features. And the training sample group used for training the model is re-divided and combined. Therefore, in the process of training the model, the training is first performed using a batch with shorter samples and lower training recognition difficulty, and then the training is performed using a batch with longer samples and higher training recognition difficulty, thereby reducing the overall difficulty of model training and improving the recognition accuracy of the model. Among them, in the training model stage, different embedding values ​​are set for each batch, so that the model can shorten the model training time and improve the training efficiency on the basis of fully learning the text features. The application of the solution provided in the embodiment of the present application can effectively reduce the cost of resources such as manpower and time required for model training.

[0146] Figure 2 is a schematic flow chart of a text classification method according to an embodiment of the present application, such as Figure 2 As shown, the text classification method includes:

[0147] S201: Obtain the text to be classified.

[0148] The text to be classified can be one or more sentences, or an incomplete sentence, such as a combination of words. In practical applications, the text to be classified can be a single message sent by a user in a conversation. The text to be classified can be user-entered text or text converted from other forms of expression through methods such as speech recognition and multilingual translation.

[0149] S202: Input the text to be classified into a trained text classification model to obtain a classification result predicted by the trained text classification model for the text to be classified, wherein the trained text classification model is a text classification model obtained by any of the text classification model training methods described in the above embodiments.

[0150] The trained text classification model is obtained by training according to the text classification model training method provided in any of the above embodiments.

[0151] In this example, the text classification model predicts the input text to be classified and obtains a classification result. Optionally, the text classification model may also input a confidence level corresponding to the classification result. In actual applications, based on application requirements, the classification result with a confidence level greater than a preset confidence level may be determined as the classification result corresponding to the text to be classified.

[0152] If the confidence level of the text classification model's actual prediction output is too low, the text to be classified can be optimized. For example, the text to be classified can be grammatically checked and any errors corrected. The corrected text to be classified can then be re-input into the text classification model to obtain the classification result predicted by the text classification model for the optimized text to be classified.

[0153] Optionally, if the confidence level of the actual prediction output of the text classification model is too low, and the text to be classified contains a large amount of content, the text to be classified can be split to obtain multiple sub-texts to be classified. Specifically, a text segment containing multiple sentences can be split into multiple single sentences to obtain multiple sub-texts to be classified. Then, the multiple sub-texts to be classified are input into the text classification model one by one for classification prediction, thereby obtaining classification results corresponding to the multiple sub-texts to be classified. Subsequently, the classification results corresponding to the multiple sub-texts to be classified are statistically analyzed and integrated to obtain the classification result of the entire text to be classified.

[0154] Through the text classification method provided by the embodiment of the present application, the text classification model can quickly and accurately classify and predict the text to be classified. Among them, the embedding dimension value used by the text classification model in the training process is positively correlated with the complexity of the text features. Therefore, the model is trained in the order of the embedding dimension value from small to large, that is, the training is performed in the order of text samples from low to high feature complexity, so that the text classification model learns the sample features from easy to difficult. Among them, the text feature embedding layer is used to convert the text sample into a text feature vector, so that the text classification model learns the features of the text sample. Since the dimension of the text feature embedding layer is too small, the sample feature text features will be lost, and if it is too long, a long training time is required. Therefore, when the present solution trains the classification model, the dimension value of the text feature embedding layer is dynamically adjusted to enable the model to learn the text features with an appropriate dimension, so that the model can shorten the training time based on the features of the text sample, thereby achieving the purpose of improving the model training speed and improving the model training effect. Then, through the text classification model, the classification prediction of the text to be classified can be performed, and accurate classification results can be obtained quickly.

[0155] In order to solve the problems existing in the prior art, such as Figure 3 As shown, the embodiment of the present application further provides a text classification model training device 300, comprising:

[0156] An acquisition module 301 acquires a plurality of text samples, and a classification label and text feature complexity corresponding to each text sample, wherein the classification label is used to indicate the classification category to which the text sample belongs, and the text feature complexity is used to indicate the complexity of the features of the text sample;

[0157] A division module 302 divides the plurality of text samples into a plurality of training sample groups according to the text feature complexity, wherein each training sample group includes at least two text samples, and the range of the text feature complexity corresponding to the at least two text samples is less than a preset range;

[0158] A determination module 303 determines embedding dimension values ​​corresponding to the plurality of training sample groups based on the text feature complexity, wherein the embedding dimension values ​​are positively correlated with the text feature complexity of each text sample in the corresponding training sample group;

[0159] The training module 304 sorts the multiple training sample groups in order of embedding dimension values ​​from small to large, and according to the sorting of the multiple training sample groups, uses each text sample in each training sample group and its corresponding classification label to train the text classification model in turn to obtain a trained text classification model, and the trained text classification model is used to perform classification prediction on the text to be classified; wherein, when using the target training sample group among the multiple training sample groups to perform training, the dimension of the text feature embedding layer in the text classification model is set to the embedding dimension value corresponding to the target training sample group, and the text feature embedding layer is used to convert the input text sample into a text feature vector, and the number of dimensions of the text feature vector is the embedding dimension value.

[0160] The apparatus provided by the embodiments of the present application enables model training to be performed in a sequence from easy to difficult, while dynamically adjusting the dimension value of the text feature embedding layer. If this dimension value is too small, sample feature text features will be lost, while if it is too long, it will require longer training time. Dynamically adjusting this dimension value can improve model training efficiency and increase training speed while retaining most features.

[0161] The aforementioned modules in the apparatus provided in the embodiments of the present application may also implement the method steps provided in the aforementioned method embodiments. Alternatively, the apparatus provided in the embodiments of the present application may further include other modules in addition to the aforementioned modules to implement the method steps provided in the aforementioned method embodiments. Furthermore, the apparatus provided in the embodiments of the present application can achieve the technical effects achievable by the aforementioned method embodiments.

[0162] In addition, in order to solve the problems existing in the prior art, please refer to Figure 4 , this embodiment of the application further provides a text classification device 400, including:

[0163] Acquisition module 401, acquires the text to be classified;

[0164] The classification module 402 inputs the text to be classified into a trained text classification model to obtain a classification result predicted by the trained text classification model for the text to be classified, wherein the text classification model is a text classification model obtained by the text classification model training method described in the first aspect.

[0165] Through the text classification device provided by the embodiment of the present application, the text classification model can be used to quickly and accurately classify and predict the text to be classified. Among them, the embedding dimension value used by the text classification model during the training process is positively correlated with the complexity of the text features. Therefore, the model is trained in the order of the embedding dimension value from small to large, that is, the training is performed in the order of text samples from low to high feature complexity, so that the text classification model learns the sample features from easy to difficult. Among them, the text feature embedding layer is used to convert the text sample into a text feature vector, so that the text classification model learns the features of the text sample. Since the dimension of the text feature embedding layer is too small, the sample feature text features will be lost, and if it is too long, a long training time is required. Therefore, when the text classification model is trained, the dimension value of the text feature embedding layer is dynamically adjusted to enable the model to learn text features with an appropriate dimension, so that the model can shorten the training time based on the features of the text sample, thereby achieving the purpose of improving the model training speed and improving the model training effect. Then, by using this text classification model to perform classification prediction on the text to be classified, accurate classification results can be obtained quickly.

[0166] The aforementioned modules in the apparatus provided in the embodiments of the present application may also implement the method steps provided in the aforementioned method embodiments. Alternatively, the apparatus provided in the embodiments of the present application may further include other modules in addition to the aforementioned modules to implement the method steps provided in the aforementioned method embodiments. Furthermore, the apparatus provided in the embodiments of the present application can achieve the technical effects achievable by the aforementioned method embodiments.

[0167] Figure 5 This is a schematic diagram of the structure of an electronic device provided by an embodiment of this specification. Figure 5 At the hardware level, the electronic device includes a processor and, optionally, an internal bus, a network interface, and memory. The memory may include internal memory, such as high-speed random-access memory (RAM), or non-volatile memory, such as at least one disk drive. Of course, the electronic device may also include other hardware required for its services.

[0168] The processor, network interface, and memory can be interconnected via an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0169] The memory is used to store programs. Specifically, the program may include program code, which includes computer operating instructions. The memory may include internal memory and non-volatile memory, and provides instructions and data to the processor.

[0170] The processor reads the corresponding computer program from the non-volatile memory into the internal memory and then runs it, forming a text classification model training device at the logical level. The processor executes the program stored in the memory and is specifically used to perform the text classification model training method described in any of the above embodiments, or to perform the text classification method described in any of the above embodiments.

[0171] The text classification model training method or text classification method described in any of the above embodiments can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits in the processor or by software instructions. The above processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The various methods, steps, and logic block diagrams disclosed in one or more embodiments of this specification can be implemented or executed. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in conjunction with one or more embodiments of this specification can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium well-known in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the memory, and the processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the above method.

[0172] The electronic device may also execute the text classification model training method described in any one of the embodiments, or execute the text classification method described in any one of the above embodiments, which will not be described in detail in this specification.

[0173] Of course, in addition to software implementation, the electronic device in this specification does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc. That is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0174] The present application also provides a computer-readable storage medium having a computer program stored thereon. When executed by a processor, the computer program implements the various processes of the above-mentioned text classification model training method or text classification method embodiment, and can achieve the same technical effects. To avoid repetition, the details are not described here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0175] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0176] The present application is described with reference to the flow chart and / or block diagram of the method, device (system), and computer program product according to the embodiment of the present application. It should be understood that each flow process and / or box in the flow chart and / or block diagram and the combination of the flow process and / or box in the flow chart and / or block diagram can be realized by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processing machine or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for realizing the function specified in one flow chart flow or multiple flows and / or one box or multiple boxes of the block diagram.

[0177] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0178] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0179] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0180] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0181] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0182] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0183] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0184] The foregoing is merely an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. A text classification model training method, characterized in that: include: Acquire multiple text samples, as well as a classification label and text feature complexity corresponding to each text sample, wherein the classification label is used to indicate the classification category to which the text sample belongs, and the text feature complexity is used to indicate the complexity of the features of the text sample; Dividing the plurality of text samples into a plurality of training sample groups according to the text feature complexity, wherein each training sample group includes at least two text samples, and the range of the text feature complexity corresponding to the at least two text samples is less than a preset range; Determining embedding dimension values ​​corresponding to the plurality of training sample groups based on the text feature complexity, wherein the embedding dimension values ​​are positively correlated with the text feature complexity of each text sample in the corresponding training sample group; The multiple training sample groups are sorted in ascending order of embedding dimension values, and according to the sorting of the multiple training sample groups, the text classification model is trained in turn using each text sample in each training sample group and its corresponding classification label to obtain a trained text classification model, and the trained text classification model is used to perform classification prediction on the text to be classified; wherein, when using the target training sample group in the multiple training sample groups to perform training, the dimension of the text feature embedding layer in the text classification model is set to the embedding dimension value corresponding to the target training sample group, and the text feature embedding layer is used to convert the input text sample into a text feature vector, and the number of dimensions of the text feature vector is the embedding dimension value.

2. The method according to claim 1, wherein The obtaining of multiple text samples, and the classification labels and text feature complexity corresponding to each text sample, includes: Obtaining the multiple text samples and a classification label corresponding to each text sample; Determining a text feature of each text sample according to the multiple text samples; According to the text features of each text sample, the text feature complexity corresponding to each text sample is determined.

3. The method according to claim 2, wherein The text feature includes text content complexity; and determining the text feature of each text sample based on the multiple text samples includes: Based on a sliding window of a preset length, sequentially performing sliding interception on a first target sample with a preset step size to obtain a plurality of byte segments of a preset length, wherein the first target sample is any one text sample among the plurality of text samples; performing deduplication on a plurality of byte segments of the preset length; A ratio of the number of byte segments after deduplication is performed to the number of byte segments before deduplication is performed is calculated, and the ratio is determined as the text content complexity of the first target sample.

4. The method according to claim 2, wherein The text feature includes a text deviation degree; and determining the text feature of each text sample based on the multiple text samples includes: Determining text similarities between a second target sample and each other text sample in the plurality of text samples to obtain a plurality of text similarities, wherein the second target sample is any one text sample in the plurality of text samples; An average value of the multiple text similarities is determined as the text deviation degree of the second target sample.

5. The method according to claim 2, wherein The text feature includes a text confidence; and determining the text feature of each text sample based on the multiple text samples includes: Using the multiple text samples to perform training on a preset basic text classification model through a cross-validation method to obtain a trained basic text classification model; A third target sample among the multiple text samples is input into the trained basic text classification model to obtain a text confidence of the third target sample, wherein the text confidence is the classification probability obtained after the trained basic text classification model performs classification prediction on the input third target text sample, and the third target sample is any one text sample among the multiple text samples.

6. The method according to any one of claims 2 to 5, wherein: The text features include text length, text content complexity, text deviation degree, and text confidence; and determining the text feature complexity corresponding to each text sample based on the text features of each text sample includes: The text length, the text content complexity, the text deviation degree and the text confidence are weightedly summed based on preset weights to obtain the text feature complexity corresponding to the text sample.

7. The method according to any one of claims 1 to 5, wherein: The dividing the plurality of text samples into a plurality of training sample groups according to the complexity of the text features comprises: According to the classification labels corresponding to the multiple text samples, the multiple text samples belonging to the same classification category are sorted in ascending order of text feature complexity to obtain multiple text samples sorted in each classification category; For each classification category, the sorted multiple text samples are divided according to the sorting to obtain a preset number of division results, each division result corresponds to a sequence number; The text samples corresponding to the division results with the same sequence number in each classification category are constructed into a training sample group to obtain the preset number of training sample groups.

8. The method according to claim 1 or 2, wherein: The text feature complexity includes text length, and determining the embedding dimension values ​​corresponding to the plurality of training sample groups based on the text feature complexity includes: The embedding dimension value of the training sample group is determined according to the text length of each text sample in the training sample group, wherein the text length of the text samples with a preset proportion in the training sample group is not greater than the embedding dimension value.

9. A text classification method, characterized in that: include: Get the text to be classified; The text to be classified is input into a trained text classification model to obtain a classification result predicted by the trained text classification model for the text to be classified, wherein the trained text classification model is a text classification model obtained by the text classification model training method according to any one of claims 1 to 8.

10. A text classification model training device, characterized in that: include: An acquisition module acquires a plurality of text samples, and a classification label and text feature complexity corresponding to each text sample, wherein the classification label is used to indicate the classification category to which the text sample belongs, and the text feature complexity is used to indicate the complexity of the features possessed by the text sample; a division module, dividing the plurality of text samples into a plurality of training sample groups according to the text feature complexity, wherein each training sample group includes at least two text samples, and the range of the text feature complexity corresponding to the at least two text samples is less than a preset range; A determination module, which determines embedding dimension values ​​corresponding to the plurality of training sample groups based on the text feature complexity, wherein the embedding dimension values ​​are positively correlated with the text feature complexity of each text sample in the corresponding training sample group; A training module sorts the multiple training sample groups in ascending order of embedding dimension values, and according to the sorting of the multiple training sample groups, uses each text sample in each training sample group and its corresponding classification label to train the text classification model in turn to obtain a trained text classification model, and the trained text classification model is used to perform classification prediction on the text to be classified; wherein, when using a target training sample group among the multiple training sample groups to perform training, the dimension of the text feature embedding layer in the text classification model is set to the embedding dimension value corresponding to the target training sample group, and the text feature embedding layer is used to convert the input text sample into a text feature vector, and the number of dimensions of the text feature vector is the embedding dimension value.

11. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the text classification model training method according to any one of claims 1 to 8, or executes the steps of the text classification method according to claim 9.

12. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the text classification model training method according to any one of claims 1 to 8, or executes the steps of the text classification method according to claim 9.

Images