Text classification model training method, text classification method, device and apparatus

Abstract

The application provides a text classification model training method, a text classification method, a device, an electronic equipment and a computer readable storage medium; and relates to the technical field of artificial intelligence; the method comprises the following steps: calling a machine translation model based on a plurality of first text samples in a first language, so as to obtain a plurality of second text samples corresponding to the plurality of first text samples one by one; wherein the plurality of second text samples are in a second language different from the first language; training a first text classification model for the second language based on a plurality of third text samples in the second language and corresponding category labels; performing confidence-based screening processing on the plurality of second text samples by using the trained first text classification model; and training a second text classification model for the second language based on the second text samples obtained through the screening processing. Through the application, cross-language text samples can be automatically obtained, and the accuracy of text classification is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to a text classification model training method, a text classification method, a device, an electronic device and a computer readable storage medium. BACKGROUND

[0002] Artificial intelligence (AI) is a comprehensive technology of computer science, which makes machines have the functions of perception, reasoning and decision-making by studying the design principles and implementation methods of various intelligent machines. Artificial intelligence technology is a comprehensive discipline, involving a wide range of fields, such as natural language processing technology and machine learning / deep learning. With the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role.

[0003] A text classification model is one of the important applications in the field of artificial intelligence, which can identify the category to which a text belongs. The text classification model has a wide range of applications in news recommendation, intent recognition systems, etc., that is, the text classification model is a basic component of these complex systems.

[0004] However, the text classification model in the related art is for a certain language, and when the text classification model is extended to text classification of other languages, the text classification model will face the pressure of lack of labeled samples of other languages, and cannot smoothly perform the text classification task of other languages. SUMMARY

[0005] The embodiments of the present application provide a text classification model training method, a text classification method, a device, an electronic device and a computer readable storage medium, which can automatically obtain cross-language text samples and improve the accuracy of text classification.

[0006] The technical solutions of the embodiments of the present application are as follows:

[0007] The embodiments of the present application provide a text classification model training method, which comprises:

[0008] calling a machine translation model based on a plurality of first text samples in a first language to obtain a plurality of second text samples corresponding one-to-one to the plurality of first text samples;

[0009] Among them, the plurality of second text samples are in a second language different from the first language;

[0010] training a first text classification model for the second language based on a plurality of third text samples in the second language and corresponding category labels;

[0011] The trained first text classification model is used to perform confidence-based filtering on the multiple second text samples;

[0012] Based on the second text sample obtained through the filtering process, a second text classification model for the second language is trained.

[0013] The network depth of the second text classification model is greater than that of the first text classification model.

[0014] In the above technical solution, training a second text classification model for the second language based on the second text samples obtained through the filtering process includes:

[0015] The second text sample obtained by the filtering process is predicted by the second text classification model to obtain the predicted category corresponding to the second text sample obtained by the filtering process.

[0016] Based on the predicted category and corresponding category label of the second text sample obtained by the filtering process, the loss function of the second text classification model is constructed.

[0017] Update the parameters of the second text classification model until the loss function converges, and use the updated parameters of the second text classification model when the loss function converges as the parameters of the trained second text classification model.

[0018] In the above technical solution, the second text classification model includes multiple cascaded encoders;

[0019] The step of predicting the second text sample obtained through the filtering process using the second text classification model to obtain the predicted category corresponding to the second text sample obtained through the filtering process includes:

[0020] For any text sample in the second text sample obtained from the filtering process, perform the following processing:

[0021] The text sample is encoded by the first encoder of the multiple cascaded encoders.

[0022] The encoding result of the first encoder is output to the subsequent cascaded encoders so that encoding processing and encoding result output can continue in the subsequent cascaded encoders until the last encoder is output.

[0023] The encoding result output by the last encoder is used as the encoding vector of the corresponding text sample;

[0024] The encoding vector of the text sample is nonlinearly mapped to obtain the predicted category corresponding to the text sample.

[0025] In the above technical solution, the step of continuing encoding processing and outputting the encoding result in the subsequently cascaded encoder includes:

[0026] The following processing is performed by the y-th encoder of the plurality of cascaded encoders:

[0027] Perform self-attention processing on the encoding result of the (y-1)th encoder to obtain the yth self-attention vector;

[0028] The y-th self-attention vector and the encoding result of the (y-1)-th encoder are subjected to residual concatenation processing to obtain the y-th residual vector;

[0029] The y-th residual vector is subjected to nonlinear mapping to obtain the y-th mapping vector;

[0030] Perform residual concatenation processing on the y-th mapping vector and the y-th residual vector, use the result of the residual concatenation as the encoding result of the y-th encoder, and output the encoding result of the y-th encoder to the (y+1)-th encoder;

[0031] Where 2≤y≤H-1, y is a natural number, and H is the number of the multiple cascaded encoders.

[0032] This application provides a text classification method, including:

[0033] Get the text to be categorized;

[0034] The text to be classified is in a second language different from the first language;

[0035] The text to be classified is encoded using a second text classification model with a network depth greater than that of the first text classification model, to obtain the encoded vector of the text to be classified.

[0036] The encoding vector of the text to be classified is non-linearly mapped to obtain the category corresponding to the text to be classified;

[0037] The second text classification model is trained using text samples of the second language selected by the first text classification model, and the text samples of the second language are obtained by machine translation of the text samples of the first language.

[0038] This application provides a training apparatus for a text classification model, comprising:

[0039] The translation module is used to invoke a machine translation model based on multiple first text samples in a first language to obtain multiple second text samples that correspond one-to-one with the multiple first text samples; wherein, the multiple second text samples are in a second language different from the first language;

[0040] The first training module is used to train a first text classification model for the second language based on multiple third text samples of the second language and their corresponding category labels.

[0041] The filtering module is used to perform confidence-based filtering on the multiple second text samples using the trained first text classification model.

[0042] The second training module is used to train a second text classification model for the second language based on the second text samples obtained by the filtering process; wherein the network depth of the second text classification model is greater than the network depth of the first text classification model.

[0043] This application provides a text classification device, including:

[0044] An acquisition module is used to acquire the text to be classified; wherein the text to be classified is in a second language different from the first language;

[0045] The processing module is used to encode the text to be classified using a second text classification model with a network depth greater than that of the first text classification model, to obtain the encoding vector of the text to be classified; and to perform nonlinear mapping on the encoding vector of the text to be classified to obtain the category corresponding to the text to be classified; wherein, the second text classification model is trained using text samples of a second language selected by the first text classification model, and the text samples of the second language are obtained by machine translation of the text samples of the first language.

[0046] This application provides an electronic device for training a text classification model, the electronic device comprising:

[0047] Memory, used to store executable instructions;

[0048] The processor, when executing executable instructions stored in the memory, implements the training method or text classification method of the text classification model provided in the embodiments of this application.

[0049] This application provides a computer-readable storage medium storing executable instructions for inducing a processor to execute and implement the training method or text classification method of the text classification model provided in this application.

[0050] The embodiments of this application have the following beneficial effects:

[0051] By using a machine translation model, second text samples in a second language different from the first language are obtained. The second text samples are then filtered using a first text classification model, thereby automatically acquiring cross-language text samples and reducing the pressure caused by a lack of text samples. Furthermore, the high-quality text samples obtained through filtering are used to train the second text classification model, enabling it to perform accurate text classification and improving the accuracy of text classification. Attached Figure Description

[0052] Figure 1 This is a schematic diagram illustrating an application scenario of the text classification system provided in this application embodiment;

[0053] Figure 2 This is a schematic diagram of the structure of an electronic device for training a text classification model provided in an embodiment of this application;

[0054] Figures 3-5 This is a flowchart illustrating the training method based on a text classification model provided in an embodiment of this application.

[0055] Figure 6 This is a schematic diagram of the iterative training process provided in an embodiment of this application;

[0056] Figure 7 This is a schematic diagram of the hierarchical softmax provided in the embodiments of this application;

[0057] Figure 8 This is a schematic diagram of a cascaded encoder provided in an embodiment of this application;

[0058] Figure 9 This is a schematic diagram of text set A and text set B provided in an embodiment of this application;

[0059] Figure 10 This is a schematic diagram of text set B1 provided in an embodiment of this application;

[0060] Figure 11 This is a schematic diagram of the active learning process provided in an embodiment of this application;

[0061] Figure 12 This is a schematic diagram of the reinforcement learning process provided in the embodiments of this application. Detailed Implementation

[0062] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0063] In the following description, the terms "first" and "second" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first" and "second" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0064] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0065] In the implementation of this application, the collection and processing of relevant data should be strictly in accordance with the requirements of relevant laws and regulations, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of laws and regulations and the authorization of the personal information subject.

[0066] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0067] 1) Convolutional Neural Networks (CNNs): A class of feedforward neural networks (FNNs) that include convolutional computations and have a deep structure, CNNs are one of the representative algorithms of deep learning. CNNs possess representation learning capabilities, enabling them to perform shift-invariant classification of input images according to their hierarchical structure.

[0068] 2) Cross-language few-shot text classification: When migrating from a language A scenario to a language B scenario and having a limited budget for labeling language B samples, only a small amount of labeled language B text and a large amount of labeled language A text are needed to achieve large-scale labeling of language B text. The text classification model is then trained using the large-scale labeling of language B text to achieve language B text classification.

[0069] 3) Cross-language zero-shot text classification: When migrating from a language A scenario to a language B scenario and lacking budget (no manpower or tight product promotion time), it is impossible to annotate language B samples. That is, only a large amount of language A annotated text is used to achieve large-scale annotation of language B, and the text classification model is trained through the large-scale annotation of language B text to achieve language B text classification.

[0070] This application provides a training method for a text classification model, a text classification method, an apparatus, an electronic device, and a computer-readable storage medium, which can automatically acquire cross-language text samples and improve the accuracy of text classification.

[0071] The training method and text classification method of the text classification model provided in this application embodiment can be implemented by the terminal / server alone; or they can be implemented by the terminal and the server in collaboration. For example, the terminal can undertake the training method of the text classification model described below alone, or the terminal can send a text classification request for a certain language to the server, and the server can execute the training method of the text classification model according to the received text classification of the certain language, and perform the text classification task of that language based on the trained text classification model.

[0072] The electronic device for training the text classification model provided in this application can be various types of terminal devices or servers. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, etc., but is not limited thereto. The terminal and server can be directly or indirectly connected via wired or wireless communication, which is not limited herein.

[0073] Taking servers as an example, such as server clusters deployed in the cloud, AI as a Service (AIaaS) is offered to users. The AIaaS platform breaks down several common AI services and provides them as independent or packaged services in the cloud. This service model is similar to an AI-themed marketplace, where all users can access and use one or more AI services provided by the AIaaS platform through application programming interfaces.

[0074] For example, one type of AI cloud service could be a text classification model training service, where a cloud server encapsulates the text classification model training program provided in this application embodiment. Users invoke the text classification model training service in the cloud service through a terminal (running a client, such as a news client, reading client, etc.), causing the cloud-deployed server to call the encapsulated text classification model training program. Based on a first text sample in a first language, a machine translation model is used to obtain a second text sample in a second language different from the first language. The first text classification model is then used to filter the second text sample, and the filtered second text sample is used to train a second text classification model. The trained second text sample is then used for text classification for subsequent news applications, reading applications, etc. For example, in a news application, if the text is in English, the trained second text sample is used to classify the text for subsequent applications such as news and reading applications. This classification model (for English news classification) determines the category of each news item to be recommended, such as entertainment news, sports news, etc., and then filters the news items based on the category to obtain the news items for recommendation, and displays the recommended news to the user to achieve targeted news recommendations. For reading applications, the text is Chinese articles. A trained second text classification model (for Chinese article classification) determines the category of each article to be recommended, such as inspirational articles, legal articles, educational articles, etc., and then filters the articles to be recommended based on the category to obtain the articles for recommendation, and displays the recommended articles to the user to achieve targeted article recommendations.

[0075] See Figure 1 , Figure 1 This is a schematic diagram of the application scenario of the text classification system 10 provided in the embodiments of this application. The terminal 200 connects to the server 100 through the network 300. The network 300 can be a wide area network or a local area network, or a combination of the two.

[0076] Terminal 200 (running a client, such as a news client) can be used to obtain text to be categorized in a certain language. For example, developers can input text to be categorized in a certain language through the terminal, and the terminal will automatically obtain a text categorization request for that language.

[0077] In some embodiments, a text classification model training plugin can be embedded in the client running on the terminal to implement the text classification model training method locally on the client. For example, after the terminal 200 obtains text to be classified in a second language different from the first language, it calls the text classification model training plugin to implement the text classification model training method. It obtains a second text sample (in the second language) corresponding to the first text sample (in the first language) through a machine translation model, filters the second text sample through the first text classification model, trains the second text classification model through the filtered second text sample, and performs text classification based on the trained second text sample for subsequent news applications, reading applications, etc.

[0078] In some embodiments, after the terminal 200 requests text classification for a certain language, it calls the text classification model training interface of the server 100 (which can be provided as a cloud service, i.e., a text classification model training service). The server 100 obtains a second text sample (using a second language) corresponding to the first text sample (using a first language) through a machine translation model, and filters the second text sample through the first text classification model. The second text classification model is trained using the filtered second text sample, and text classification is performed based on the trained second text sample for subsequent news applications, reading applications, etc.

[0079] The structure of the electronic device for training a text classification model provided in the embodiments of this application is described below. See also... Figure 2 , Figure 2 This is a schematic diagram of the structure of an electronic device 500 for training a text classification model provided in an embodiment of this application. The example is a server. Figure 2 The illustrated electronic device 500 for training a text classification model includes at least one processor 510, a memory 550, at least one network interface 520, and a user interface 530. The various components in the electronic device 500 are coupled together via a bus system 540. It is understood that the bus system 540 is used to implement communication between these components. In addition to a data bus, the bus system 540 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 2 The general labeled all buses as Bus System 540.

[0080] The processor 510 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0081] Memory 550 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 550 described in this application embodiment is intended to include any suitable type of memory. Memory 550 may optionally include one or more storage devices physically located away from processor 510.

[0082] In some embodiments, memory 550 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0083] Operating system 551 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic business functions and handling hardware-based tasks;

[0084] The network communication module 553 is used to reach other computing devices via one or more (wired or wireless) network interfaces 520, exemplary network interfaces 520 including: Bluetooth, WiFi, and Universal Serial Bus (USB), etc.

[0085] In some embodiments, the training apparatus for the text classification model provided in this application can be implemented in software. For example, it can be a text classification model training plugin in the terminal mentioned above, or a text classification model training service in the server mentioned above. Of course, it is not limited to this. The training apparatus for the text classification model provided in this application can be provided in various software embodiments, including various forms such as applications, software, software modules, scripts, or code.

[0086] Figure 2 A training device 555 for a text classification model stored in a memory 550 is shown. It can be software in the form of programs and plug-ins, such as a text classification model training plug-in, and includes a series of modules, including a translation module 5551, a first training module 5552, a filtering module 5553, and a second training module 5554. The translation module 5551, the first training module 5552, the filtering module 5553, and the second training module 5554 are used to implement the training function of the text classification model provided in the embodiments of this application.

[0087] As mentioned above, the training method for the text classification model provided in this application can be implemented by various types of electronic devices. See alsoFigure 3 , Figure 3 This is a flowchart illustrating the training method for a text classification model provided in this application embodiment, combined with... Figure 3 The steps shown are explained.

[0088] In the following steps, the network depth of the second text classification model is greater than that of the first text classification model, meaning that the text classification ability of the second text classification model is stronger than that of the first text classification model. Therefore, the number of text samples required to train the second text classification model is greater than the number of text samples required to train the first text classification model.

[0089] In the following steps, the first text sample uses a first language, and the second and third text samples use a second language different from the first language. For example, the first text sample is a Chinese sample, and the second and third text samples are English samples.

[0090] In step 101, a machine translation model is invoked based on multiple first text samples of the first language to obtain multiple second text samples that correspond one-to-one with the multiple first text samples.

[0091] For example, when a developer inputs a text classification command in a second language through a terminal, the terminal automatically obtains a text classification request for the second language and sends it to the server. After receiving the text classification request, the server retrieves a large number of labeled first text samples from the sample library. However, the first text samples use a first language different from the second language. The server then calls a machine translation model to perform machine translation on the multiple first text samples to obtain multiple second text samples that correspond one-to-one with the multiple first text samples. The category labels of the second text samples inherit the category labels of the corresponding first text samples, which means that no manual labeling is required, greatly saving the labeling pressure caused by manual labeling.

[0092] In step 102, a first text classification model for the second language is trained based on multiple third text samples of the second language and their corresponding category labels.

[0093] Steps 101 and 102 do not have a clear order. After the server receives a text classification request in the second language, it obtains a small number of labeled third text samples from the sample library. Using multiple third text samples and their corresponding category labels, the first text classification model is trained, enabling the trained first text classification model to perform text classification based on the second language.

[0094] In some embodiments, training a first text classification model for a second language based on multiple third text samples in the second language and their corresponding category labels includes: training the first text classification model for the tth time based on multiple third text samples in the second language and their corresponding category labels; performing a tth-fold screening process on multiple second text samples based on confidence using the tth-fold trained first text classification model; training the first text classification model for the (t+1)th time based on the results of the previous t screenings, multiple third text samples, and their corresponding category labels; and using the tth-fold trained first text classification model as the trained first text classification model; where t is a value that satisfies 1. t T-1 is an integer, and its value increases from 1. T is an integer greater than 2, used to represent the total number of iterations for training.

[0095] For example, based on multiple third text samples in the second language and their corresponding category labels, the first text classification model is iteratively trained to select more high-quality third text samples through the gradually optimized first text classification model for subsequent enhancement training to train the second text classification model.

[0096] like Figure 6 As shown, the first text classification model is trained for the first time based on multiple third text samples of the second language and their corresponding category labels. The first text classification model trained for the first time is used to perform a first screening process based on confidence for multiple second text samples. Based on the first screening result, multiple third text samples and their corresponding category labels, the first text classification model is trained for the second time. The first text classification model trained for the second time is used to perform a second screening process based on confidence for the second text samples other than the first screening result. The first text classification model is trained for the third time using the first two screening results, multiple third text samples and their corresponding category labels. The above training process is iterated until the first text classification model is trained for the Tth time. The first text classification model trained for the Tth time is used as the trained first text classification model.

[0097] See Figure 4 , Figure 4 This is a schematic diagram of an optional process for training a text classification model provided in an embodiment of the present invention. Figure 4 Show Figure 3 Step 102 in the process can be achieved through Figure 4Steps 1021 to 1023 are implemented as follows: In step 1021, multiple third text samples of the second language are predicted using the first text classification model to obtain the confidence scores of the predicted categories corresponding to the multiple third text samples; In step 1022, a loss function of the first text classification model is constructed based on the confidence scores of the predicted categories and the category labels of the third text samples; In step 1023, the parameters of the first text classification model are updated until the loss function converges, and the updated parameters of the first text classification model when the loss function converges are used as the parameters of the trained first text classification model.

[0098] For example, after determining the value of the loss function of the first text classification model based on the confidence level of the predicted category and the category label of the third text sample, it can be determined whether the value of the loss function of the first text classification model exceeds a preset threshold. When the value of the loss function of the first text classification model exceeds the preset threshold, the error signal of the first text classification model is determined based on the loss function of the first text classification model, the error information is backpropagated in the first text classification model, and the model parameters of each layer are updated during the propagation process.

[0099] Here, we explain backpropagation. Training sample data is input into the input layer of the neural network model, passes through the hidden layers, and finally reaches the output layer to output the result. This is the forward propagation process of the neural network model. Since there is an error between the output result and the actual result, the error between the output result and the actual value is calculated and propagated back from the output layer to the hidden layers until it reaches the input layer. During the backpropagation process, the values ​​of the model parameters are adjusted according to the error. This process is iterated until convergence. The first text classification model is a neural network model.

[0100] In some embodiments, a first text classification model is used to predict multiple third text samples of a second language to obtain the confidence scores of the predicted categories corresponding to the multiple third text samples. This includes performing the following processing on any one of the multiple third text samples: performing the following processing on the first text classification model: encoding the third text sample to obtain an encoding vector of the third text sample; fusing the encoding vector of the third text sample to obtain a fused vector; and performing nonlinear mapping processing on the fused vector to obtain the confidence score of the predicted category corresponding to the third text sample.

[0101] For example, the first text classification model is a fast text classification model (fasttext). However, the first text classification model in this embodiment is not limited to fasttext. Fasttext includes an input layer, a hidden layer, and an output layer. It can be quickly trained using a small number of third text samples to enable it to perform second language text classification tasks quickly. For instance, the input layer encodes the third text sample to obtain its encoding vector; the hidden layer then fuses these encoding vectors to obtain a fused vector; finally, the output layer performs a non-linear mapping (i.e., mapping through an activation function (softmax)) on the fused vector to obtain the confidence score of the predicted category corresponding to the third text sample.

[0102] In some embodiments, the first text classification model includes multiple cascaded activation layers; performing nonlinear mapping processing on the fusion vector to obtain the confidence score of the predicted category corresponding to the third text sample includes: performing a mapping process on the fusion vector through the first activation layer of the multiple cascaded activation layers; outputting the mapping result of the first activation layer to subsequent cascaded activation layers, so as to continue the mapping processing and output the mapping result in subsequent cascaded activation layers until the last activation layer is output; and using the activation result output by the last activation layer as the confidence score of the predicted category corresponding to the third text sample.

[0103] like Figure 7 As shown, hierarchical softmax activation avoids obtaining the confidence score of the predicted class through a single activation operation, instead using multiple activation layers to reduce computational complexity. For example, hierarchical softmax includes T activation layers. Each activation layer performs a hierarchical softmax operation. The first activation layer maps the fusion vector to its first mapping result, which is then passed to the second activation layer. This process continues until the Tth activation layer is reached. The activation result from the Tth activation layer is used as the confidence score of the predicted class for the third text sample. Here, T represents the total number of activation layers.

[0104] In some embodiments, encoding a third text sample to obtain an encoding vector for the third text sample includes: performing window sliding processing on the third text sample to obtain multiple segment sequences; wherein the size of the window is N, and N is a natural number; mapping the multiple segment sequences based on a vocabulary to obtain sequence vectors corresponding to the multiple segment sequences respectively; and combining the sequence vectors corresponding to the multiple segment sequences to obtain the encoding vector for the third text sample.

[0105] Continuing with the above example, if the fragment sequence includes N characters, then perform a window sliding process on the third text sample to obtain multiple fragment sequences, including: perform the following process on the i-th character in the third text sample: obtain the i-th character to the (i + N - 1)-th character in the third text sample; combine the i-th character to the (i + N - 1)-th character, and use the combination result as the fragment sequence; where 0 < i ≤ M - N + 1, M is the number of characters in the third text sample, and M is a natural number. This can generate better encoding vectors for rare characters. In the vocabulary table, even if a character does not appear in the training corpus, it is still possible to construct the corresponding character-level encoding vector from the character-level window, and it can also enable the first text classification model to learn partial information about the local character order, which will enable the first text classification model to maintain character order information during training.

[0106] Continuing with the above example, if the fragment sequence includes N words, then perform a window sliding process on the third text sample to obtain multiple fragment sequences, including: perform the following process on the j-th word in the third text sample: obtain the j-th word to the (j + N - 1)-th word in the third text sample; combine the j-th word to the (j + N - 1)-th word, and use the combination result as the fragment sequence; where 0 < j ≤ K - N + 1, K is the number of words in the third text sample, and K is a natural number. This can generate better encoding vectors for rare words (sentences). In the vocabulary table, even if a sentence does not appear in the training corpus, it is still possible to construct the corresponding word-level encoding vector from the word-level window, and it can also enable the first text classification model to learn partial information about the local word order, which will enable the first text classification model to maintain word order information during training.

[0107] In step 103, perform a confidence-based screening process on multiple second text samples using the trained first text classification model.

[0108] After the server obtains the trained first text classification model through the third text sample, it can perform a confidence-based screening process on multiple second text samples using the trained first text classification model to screen out high-quality second text samples, so as to train the second text classification model with the high-quality second text samples.

[0109] In some embodiments, a confidence-based filtering process is performed on multiple second text samples using a trained first text classification model, including: performing the following processing on any one of the multiple second text samples: performing prediction processing on the second text sample using the trained first text classification model to obtain the confidence of multiple predicted categories corresponding to the second text sample; determining the category label of the first text sample corresponding to the second text sample as the category label of the second text sample; and, based on the confidence of the multiple predicted categories corresponding to the second text sample and the category label of the second text sample, selecting second text samples that exceed the confidence threshold as the second text samples obtained through the filtering process.

[0110] For example, the second text sample is encoded using the trained first text classification model to obtain the encoded vector of the second text sample. The encoded vector of the second text sample is then fused to obtain the fused vector. The fused vector is then subjected to nonlinear mapping to obtain the confidence scores of multiple predicted categories corresponding to the second text sample. From the multiple predicted categories corresponding to the second text sample, the predicted category that matches the category label of the second text sample is determined. When the confidence score of the matched predicted category exceeds the confidence score threshold, the second text sample is used as the second text sample obtained after the filtering process.

[0111] In step 104, a second text classification model for the second language is trained based on the second text samples obtained through the filtering process.

[0112] After the server filters out a large number of high-quality second text samples through the trained first text classification model, it automatically constructs cross-language text samples (i.e., second text samples in the second language, which have the corresponding category labels of the first text samples, i.e., no manual labeling is required). The second text classification model is trained with a large number of high-quality second text samples so that the trained second text classification model can accurately classify text based on the second language, thereby improving the accuracy of second language text classification.

[0113] Since sufficient second text samples can be obtained through the training method of this application embodiment for training the second text classification model, this application embodiment can train the second text classification model simply by filtering the obtained second text samples.

[0114] After the server obtains the trained second text classification model, it responds to the text classification request for the second language by classifying the text to be classified. That is, it encodes the text to be classified using the trained second text classification model to obtain the encoding vector of the text to be classified, and performs a non-linear mapping on the encoding vector of the text to be classified to obtain the category corresponding to the text to be classified. The category corresponding to the text to be classified can also be used for subsequent news applications, reading applications, etc.

[0115] See Figure 5 , Figure 5 This is a schematic diagram of an optional process for training a text classification model provided in an embodiment of the present invention. Figure 5 Show Figure 3 Step 104 in the process can be achieved through Figure 5 Steps 1041 to 1043 are implemented as follows: In step 1041, the distribution of the second text samples obtained through the screening process in multiple categories is determined; in step 1042, when the distribution of the second text samples obtained through the screening process in multiple categories satisfies the distribution equilibrium condition and the number in each category exceeds the corresponding category number threshold, text samples corresponding to the category number threshold are randomly extracted from the text samples of each category in the second text samples obtained through the screening process to construct a training set; in step 1043, a second text classification model for the second language is trained based on the training set.

[0116] For example, after the server obtains a large number of second text samples for training a second text classification model, the distribution of the filtered second text samples across multiple categories is analyzed to determine whether the distribution balance condition is met, i.e., the fluctuation in the number of samples in different categories. For example, the mean squared error is used to measure the fluctuation in the number of samples in different categories; the greater the fluctuation, the more unbalanced the distribution of text samples across multiple categories. When the distribution of the filtered second text samples across multiple categories meets the distribution balance condition, and the number of samples in each category exceeds the category number threshold, then text samples corresponding to the category number threshold are extracted from the text samples of each category in the filtered second text samples to construct a training set, thereby improving the accuracy of text classification.

[0117] In some embodiments, training a second text classification model for a second language based on the second text samples obtained through screening includes: when the distribution of the second text samples obtained through screening does not meet the distribution equilibrium condition in multiple categories, performing a synonym-based expansion process on the second text samples of the corresponding categories so that the distribution of the expanded second text samples in multiple categories meets the distribution equilibrium condition; constructing a training set based on the expanded second text samples; and training a second text classification model for the second language based on the training set.

[0118] In some embodiments, training a second text classification model for a second language based on the second text samples obtained through filtering includes: constructing a training set based on multiple third text samples and the second text samples obtained through filtering, and training a second text classification model for a second language based on the training set.

[0119] When the number of second text samples obtained from the screening process is lower than the corresponding category number threshold, the second text samples in the corresponding category are expanded based on synonyms so that the number of second text samples obtained from the expanded process in each category exceeds the corresponding category number threshold; a training set is constructed based on the second text samples obtained from the expanded process.

[0120] The specific expansion process is as follows: For any text sample from multiple third-party text samples and the filtered second-party text samples, the following processing is performed: A synonym dictionary (including correspondences between various synonyms) is matched with the words in the text sample to obtain matching words; based on the matching words, the words in the text sample are replaced to obtain a new text sample; the category label corresponding to the original text sample is used as the category label of the new text sample. By using synonym replacement, the second-language text samples can be greatly expanded to enable training of the second-language text classification model.

[0121] In some embodiments, constructing a training set based on multiple third text samples and second text samples obtained through filtering includes: traversing each category of the second text samples obtained through filtering and performing the following processing: when the number of second text samples in a category is lower than the category's category count threshold, randomly selecting third text samples from multiple third text samples to supplement the category's second text samples to update the second text samples obtained through filtering; and constructing a training set based on the updated second text samples obtained through filtering.

[0122] For example, when there are few text samples in some categories, or when the distribution of some categories is uneven, third text samples can be used to supplement them. For instance, if the number of second text samples in a category is lower than the category's number threshold, it indicates that there are few text samples in that category. Third text samples of that category can be randomly selected from multiple third text samples to supplement the second text samples of that category, thereby updating the second text samples obtained after the filtering process and making the second text samples more abundant in terms of the number of text samples of that category.

[0123] In some embodiments, to avoid overfitting caused by training a second classification model with a large number of samples, the computational power of the second text classification model can be matched with the corresponding number of text samples for appropriate training. Before training the second text classification model for a second language, based on the second text samples obtained through screening, a target number of samples matching the computational power available for training the second text classification model is determined according to the correspondence between the computational power (computing ability) of the text classification model and the number of text samples it can process per unit time. From the training set constructed based on the second text samples obtained through screening, text samples corresponding to the target number of samples are selected as samples for training the second text classification model for a second language.

[0124] In some embodiments, training a second text classification model for a second language based on the second text samples obtained through screening includes: performing prediction processing on the second text samples obtained through screening using the second text classification model to obtain the predicted category corresponding to the second text samples obtained through screening; constructing a loss function for the second text classification model based on the predicted category and corresponding category label of the second text samples obtained through screening; updating the parameters of the second text classification model until the loss function converges, and using the updated parameters of the second text classification model when the loss function converges as the parameters of the trained second text classification model.

[0125] For example, based on the predicted category and corresponding category label of the second text sample obtained through screening, after determining the value of the loss function of the second text classification model, it can be determined whether the value of the loss function of the second text classification model exceeds a preset threshold. When the value of the loss function of the second text classification model exceeds the preset threshold, the error signal of the second text classification model is determined based on the loss function of the second text classification model, the error information is backpropagated in the second text classification model, and the model parameters of each layer are updated during the propagation process.

[0126] In some embodiments, the second text classification model includes multiple cascaded encoders; the second text classification model is used to predict the second text samples obtained through the filtering process to obtain the predicted category corresponding to the second text samples obtained through the filtering process, including: performing the following processing on any text sample among the second text samples obtained through the filtering process: encoding the text sample through the first encoder of the multiple cascaded encoders; outputting the encoding result of the first encoder to the subsequent cascaded encoders to continue encoding processing and outputting encoding results in the subsequent cascaded encoders until the last encoder is output; using the encoding result output by the last encoder as the encoding vector of the corresponding text sample; performing a non-linear mapping on the encoding vector of the text sample to obtain the predicted category corresponding to the text sample.

[0127] like Figure 8 As shown, rich feature information of text samples can be extracted through cascaded encoders. For example, the first encoder encodes the text sample, obtaining the first encoding result. This first encoding result is then fed to the second encoder, which encodes the first result, obtaining the second encoding result, and so on, until the S-th encoder is reached. Finally, a non-linear mapping is applied to the encoded vector of the text sample to obtain the predicted category corresponding to the text sample. Here, S represents the total number of encoders.

[0128] Continuing with the example above, the encoding process and output of the encoding results are further performed in the subsequent cascaded encoders, including: the following processing is performed on the y-th encoder of the multiple cascaded encoders: self-attention processing is applied to the encoding result of the (y-1)-th encoder to obtain the y-th self-attention vector; residual concatenation processing is applied to the y-th self-attention vector and the encoding result of the (y-1)-th encoder to obtain the y-th residual vector; non-linear mapping processing is applied to the y-th residual vector to obtain the y-th mapping vector; residual concatenation processing is applied to the y-th mapping vector and the y-th residual vector, and the result of the residual concatenation is used as the encoding result of the y-th encoder, and the encoding result of the y-th encoder is output to the (y+1)-th encoder; where 2≤y≤H-1, y is a natural number, and H is the number of multiple cascaded encoders.

[0129] The following will describe an exemplary application of the embodiments of this application in a real-world application scenario.

[0130] Text classification is widely used in content-related products, such as news classification, article classification, intent classification, news feed products, forums, communities, e-commerce, and so on. Generally, text classification is applied to text in a single language, such as Chinese or English. However, when a product needs to expand into other languages, it may encounter insufficient labeled text in the early stages. For example, when expanding a news reading product from the Chinese market to the English market, it's necessary to quickly label news articles in the English-speaking world. Similarly, when performing sentiment analysis on Chinese user comments, as the user base grows or the product is launched in overseas markets, many comments that are not in Chinese will appear, and these comments also need to be labeled with their corresponding sentiment polarity.

[0131] While, over a longer timeframe, a substantial amount of labeled data in other languages ​​can be gradually accumulated through manual processes, enabling model training and prediction, manually labeling text in the early stages is extremely time-consuming and labor-intensive, hindering rapid product iteration. Therefore, the initial approach aims to achieve automated text labeling of large volumes through algorithms and leveraging the existing accumulation of labeled text in other languages.

[0132] The related technologies all revolve around few-shot or zero-shot text classification in the same language, which only solves the problem of insufficient labeled samples in the same language and lacks cross-language text classification.

[0133] To address the aforementioned issues, this application proposes a cross-language text classification method based on a translation model and active learning. It utilizes existing machine translation models to increase the number of language B samples using language A samples. However, since text translated by algorithms has inherent biases and errors, an active learning method is employed to select high-quality samples from the translated text for subsequent training.

[0134] The following describes in detail a cross-language text classification method based on a translation model and active learning proposed in this application. This method comprises two parts: A) data preparation, B) algorithm framework, and C) prediction.

[0135] A) Data Preparation

[0136] This application's embodiments address situations where there are no large numbers of samples (unlabeled), making it impossible to train large pre-trained models to extract text content. For example... Figure 9 As shown, in this embodiment of the application, there is a partial text set A (Text A, a text set in language A) (including the first text sample) and a small amount of text set B (Text B, a text set in language B) (including the third text sample). Text A and Text B are samples with category labels. Compared with Text A, Text B has only a small number of labels, so its proportion is very small.

[0137] Here, the labeled samples in Text A are denoted as<X_A,Y> Text B is denoted as<X_B,Y> In this context, X_A represents the text in Text A, and X_B represents the text in Text B. The labels for Text A and Text B are common and are both represented by Y. For example, category 0 (Y=0) represents entertainment news, and category 1 (Y=1) represents sports news. Here, 0 and 1 are universal and independent of language.

[0138] B) Algorithm Framework

[0139] The algorithm framework in this application includes: 1) sample augmentation, 2) active learning, and 3) augmentation training. The sample augmentation, active learning, and augmentation training are described in detail below:

[0140] 1) Sample Augmentation

[0141] First, such as Figure 10As shown, the machine translation model used (for translating language A into language B) converts each text X_A in language A into text in language B to form a corresponding text set B1 (Text B1, the text set in language B formed by translation).

[0142] This sample augmentation method yields two types of labeled text. One type is the original sample set Text B, which contains a small number of manually labeled samples and is very accurate. The other type is Text B1 (including the second text sample), which is a large number of labeled samples (their labels correspond to those in Text A) obtained after conversion by a machine translation model. Text B1 may contain noise, errors, etc., and is not as accurate as Text B.

[0143] 2) Active learning

[0144] In order to filter out high-quality samples from Text B1, an active learning method is adopted, and the entire process is as follows: Figure 11 As shown:

[0145] Step 1: First, train a weak classifier (a first-level text classification model) (such as a shallow classifier like FastText) using manually labeled Text B. Then, apply the weak classifier to Text B1 for prediction. Select samples with high confidence from Text B1. For example, assuming the confidence threshold is 0.8, if the confidence of a sample X_B1 in Text B1 predicting the label Y=2 is 0.87 (greater than 0.8), then the class of sample X_B1 is considered to be 2, thus obtaining labeled training samples.<X_B1,Y=2> .

[0146] Step 2: Use these high-confidence, labeled samples to form a new training sample set (text set B1', TextB1'). Based on Text B1' and Text B, continue to train the weak classifier. After training, repeat Step 1 and apply the weak classifier to the remaining samples after filtering Text B1 (the remaining samples refer to the text remaining after selecting high-confidence samples from Text B1).

[0147] Step 3: Continue training until the confidence level of the samples predicted in Text B1 can no longer exceed the specified confidence threshold. This means that the remaining samples after Text B1 screening are considered to be of poor quality. At this point, stop iterative training.

[0148] 3) Enhanced training

[0149] like Figure 12As shown, the Text B' and Text B obtained from the above steps are mixed together, and then a strong classifier (second text classification model) is trained (e.g., deep neural network (BERT, Bidirectional Encoder Representations from Transformers)).

[0150] C) Prediction

[0151] The trained strong classifier is used as the final text classification model for B language text classification. For example, when expanding a news reading product from the Chinese market to the English (B language) market, the trained strong classifier quickly labels English news articles. When users read news, the classifier can recommend news based on these labels, thus recommending English news that matches the user's interests. Similarly, when performing sentiment analysis on Chinese user comments, many comments appear in overseas markets (B language). The trained strong classifier quickly labels these English comments with relevant sentiment tags. When users comment, the classifier can provide appropriate emotional guidance based on the English comment tags, preventing users from developing negative emotions.

[0152] This concludes the explanation of the training method for the text classification model provided in this application, using the exemplary application and implementation of the server provided in the embodiments of this application. This application also provides a training apparatus for the text classification model. In practical applications, the functional modules in the training apparatus for the text classification model can be collaboratively implemented using the hardware resources of electronic devices (such as terminal devices, servers, or server clusters), such as computing resources like processors, communication resources (such as those used to support various communication methods like optical fiber and cellular), and memory. Figure 2 The training device 555 for the text classification model stored in the memory 550 is shown. It can be software in the form of programs and plug-ins, such as software modules designed in programming languages ​​such as C / C++ and Java, application software designed in programming languages ​​such as C / C++ and Java, or dedicated software modules, application programming interfaces, plug-ins, cloud services, etc. in large software systems. Examples of different implementation methods are given below.

[0153] Example 1: The training device for the text classification model is a mobile application and module.

[0154] The text classification model training device 555 in this embodiment can be provided as a software module designed using programming languages ​​such as C / C++ and Java, embedded into various mobile applications based on systems such as Android or iOS (stored as executable instructions in the storage medium of the mobile device and executed by the processor of the mobile device), thereby directly using the computing resources of the mobile device itself to complete the relevant information recommendation tasks, and periodically or irregularly transmitting the processing results to a remote server through various network communication methods, or saving them locally on the mobile device.

[0155] Example 2: The training device for the text classification model is a server application and platform.

[0156] The text classification model training device 555 in this embodiment can be provided as a dedicated software module in an application software or large software system designed using programming languages ​​such as C / C++ and Java. It runs on the server side (stored in the server-side storage medium in the form of executable instructions and executed by the server-side processor). The server uses its own computing resources to complete the relevant information recommendation tasks.

[0157] This application embodiment can also provide an information recommendation platform (for recommendation lists) for use by individuals, groups or organizations by mounting a customized, easy-to-interact web interface or other user interfaces (UI) on a distributed, parallel computing platform composed of multiple servers.

[0158] Example 3: The training device for the text classification model is a server-side application programming interface (API) and plugins.

[0159] The text classification model training device 555 in this application embodiment can be provided as a server-side API or plugin for users to call, to execute the text classification model training method of this application embodiment, and to be embedded in various applications.

[0160] Example 4: The training device for the text classification model is a mobile device client API and plugin.

[0161] The training device 555 for the text classification model in this application embodiment can be provided as an API or plugin on a mobile device for users to call in order to execute the training method of the text classification model in this application embodiment.

[0162] Example 5: The training device for the text classification model is an open cloud service.

[0163] The text classification model training device 555 in this embodiment can provide a cloud service for information recommendation developed for users, allowing individuals, groups, or organizations to obtain recommendation lists.

[0164] The text classification model training device 555 includes a series of modules, including a translation module 5551, a first training module 5552, a filtering module 5553, and a second training module 5554. The following description further illustrates the training scheme of the text classification model provided in this embodiment, where the various modules cooperate to implement the text classification model training.

[0165] Translation module 5551 is used to call a machine translation model based on multiple first text samples in a first language to obtain multiple second text samples that correspond one-to-one with the multiple first text samples; wherein the multiple second text samples are in a second language different from the first language; first training module 5552 is used to train a first text classification model for the second language based on multiple third text samples in the second language and their corresponding category labels; filtering module 5553 is used to perform confidence-based filtering on the multiple second text samples using the trained first text classification model; second training module 5554 is used to train a second text classification model for the second language based on the second text samples obtained from the filtering process; wherein the network depth of the second text classification model is greater than the network depth of the first text classification model.

[0166] In some embodiments, the first training module 5552 is further configured to: train the first text classification model for the tth time based on multiple third text samples of the second language and their corresponding category labels; perform a confidence-based filtering process on the multiple second text samples using the first text classification model trained for the tth time; train the first text classification model for the (t+1)th time based on the filtering results of the previous t times, the multiple third text samples, and their corresponding category labels; and use the first text classification model trained for the Tth time as the trained first text classification model; wherein, t is a value that satisfies the condition 1. t T-1 is an integer, and its value increases from 1. T is an integer greater than 2, used to represent the total number of iterations for training.

[0167] In some embodiments, the second training module 5554 is further configured to determine the distribution of the second text samples obtained by the filtering process in multiple categories; when the distribution of the second text samples obtained by the filtering process in multiple categories satisfies the distribution equilibrium condition and the number in each category exceeds the corresponding category number threshold, text samples corresponding to the category number threshold are randomly extracted from the text samples of each category in the second text samples obtained by the filtering process to construct a training set; and a second text classification model for the second language is trained based on the training set.

[0168] In some embodiments, the second training module 5554 is further configured to perform a synonym-based expansion process on the second text samples of the corresponding category when the distribution of the second text samples obtained by the filtering process in multiple categories does not meet the distribution equilibrium condition, so that the distribution of the second text samples obtained by the expansion process in multiple categories meets the distribution equilibrium condition; construct a training set based on the second text samples obtained by the expansion process; and train a second text classification model for the second language based on the training set.

[0169] In some embodiments, the second training module 5554 is further configured to construct a training set based on the plurality of third text samples and the second text samples obtained by the filtering process, and to train a second text classification model for the second language based on the training set.

[0170] In some embodiments, the second training module 5554 is further configured to traverse each category of the second text samples obtained by the filtering process and perform the following processing: when the number of second text samples in the category is lower than the category number threshold, the third text samples of the category are randomly selected from the plurality of third text samples to supplement the second text samples of the category to update the second text samples obtained by the filtering process; and a training set is constructed based on the updated second text samples obtained by the filtering process.

[0171] In some embodiments, the second training module 5554 is further configured to determine the number of target samples that match the computing power that can be used to train the second text classification model, based on the correspondence between the computing power of the text classification model and the number of text samples that can be processed per unit time; and to select text samples corresponding to the number of target samples from the training set constructed based on the second text samples obtained by the filtering process, so as to use them as samples for training the second text classification model for the second language.

[0172] In some embodiments, the first training module 5552 is further configured to perform prediction processing on multiple third text samples of the second language using the first text classification model to obtain the confidence scores of the predicted categories corresponding to the multiple third text samples respectively; construct a loss function of the first text classification model based on the confidence scores of the predicted categories and the category labels of the third text samples; update the parameters of the first text classification model until the loss function converges, and use the updated parameters of the first text classification model when the loss function converges as the parameters of the trained first text classification model.

[0173] In some embodiments, the first training module 5552 is further configured to perform the following processing on any one of the plurality of third text samples: perform the following processing through the first text classification model: encode the third text sample to obtain the encoding vector of the third text sample; fuse the encoding vector of the third text sample to obtain the fusion vector; and perform nonlinear mapping processing on the fusion vector to obtain the confidence of the predicted category corresponding to the third text sample.

[0174] In some embodiments, the first text classification model includes multiple cascaded activation layers; the first training module 5552 is further configured to perform mapping processing on the fusion vector through the first activation layer of the multiple cascaded activation layers; output the mapping result of the first activation layer to subsequent cascaded activation layers, so as to continue mapping processing and mapping result output in the subsequent cascaded activation layers until the last activation layer is output; and use the activation result output by the last activation layer as the confidence level of the predicted category corresponding to the third text sample.

[0175] In some embodiments, the filtering module 5553 is further configured to perform the following processing on any one of the plurality of second text samples: perform prediction processing on the second text sample using the trained first text classification model to obtain the confidence scores of the plurality of predicted categories corresponding to the second text sample; determine the category label of the first text sample corresponding to the second text sample as the category label of the second text sample; and, based on the confidence scores of the plurality of predicted categories corresponding to the second text sample and the category label of the second text sample, use the second text sample that exceeds the confidence score threshold as the second text sample obtained by the filtering process.

[0176] In some embodiments, the second training module 5554 is further configured to perform prediction processing on the second text sample obtained by the filtering process using the second text classification model to obtain the predicted category corresponding to the second text sample obtained by the filtering process; construct the loss function of the second text classification model based on the predicted category and the corresponding category label of the second text sample obtained by the filtering process; update the parameters of the second text classification model until the loss function converges, and use the updated parameters of the second text classification model when the loss function converges as the parameters of the trained second text classification model.

[0177] In some embodiments, the second text classification model includes multiple cascaded encoders; the second training module 5554 is further configured to perform the following processing on any text sample in the second text samples obtained by the filtering process: encoding the text sample using the first encoder of the multiple cascaded encoders; outputting the encoding result of the first encoder to subsequent cascaded encoders to continue encoding processing and outputting encoding results in the subsequent cascaded encoders until output to the last encoder; using the encoding result output by the last encoder as the encoding vector corresponding to the text sample; performing a nonlinear mapping on the encoding vector of the text sample to obtain the predicted category corresponding to the text sample.

[0178] In some embodiments, the second training module 5554 is further configured to perform the following processing on the y-th encoder of the plurality of cascaded encoders: perform self-attention processing on the encoding result of the (y-1)-th encoder to obtain the y-th self-attention vector; perform residual concatenation processing on the y-th self-attention vector and the encoding result of the (y-1)-th encoder to obtain the y-th residual vector; perform nonlinear mapping processing on the y-th residual vector to obtain the y-th mapping vector; perform residual concatenation processing on the y-th mapping vector and the y-th residual vector, use the result of the residual concatenation as the encoding result of the y-th encoder, and output the encoding result of the y-th encoder to the (y+1)-th encoder; where 2≤y≤H-1, y is a natural number, and H is the number of the plurality of cascaded encoders.

[0179] This application also provides a text classification device, which includes a series of modules, including an acquisition module and a processing module. The acquisition module acquires text to be classified, wherein the text to be classified is in a second language different from the first language. The processing module encodes the text to be classified using a second text classification model with a network depth greater than that of the first text classification model to obtain an encoding vector of the text to be classified; and performs a non-linear mapping on the encoding vector of the text to be classified to obtain the category corresponding to the text to be classified. The second text classification model is trained using text samples of the second language selected by the first text classification model, and the text samples of the second language are obtained by machine translation of text samples of the first language.

[0180] This application provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the text classification model training method or text classification method described above in this application.

[0181] This application provides a computer-readable storage medium storing executable instructions. When these executable instructions are executed by a processor, they cause the processor to execute the artificial intelligence-based information recommendation method or text classification method provided in this application. For example... Figures 3-5 The training method for the text classification model is shown.

[0182] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0183] In some embodiments, executable instructions may take the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0184] As an example, executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple collaborating files (e.g., a file that stores one or more modules, subroutines, or code sections).

[0185] As an example, executable instructions can be deployed to execute on a single computing device, or on multiple computing devices located in one location, or on multiple computing devices distributed across multiple locations and interconnected via a communication network.

[0186] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. A training method for a text classification model, characterized in that, include: Based on multiple first text samples in the first language, a machine translation model is invoked to obtain multiple second text samples that correspond one-to-one with the multiple first text samples; The plurality of second text samples are written in a second language different from the first language. A first text classification model for the second language is trained based on multiple third text samples of the second language and their corresponding category labels. For any one of the plurality of second text samples, the following processing is performed: The trained first text classification model is used to predict the second text sample to obtain the confidence scores of multiple predicted categories corresponding to the second text sample. The category label of the first text sample corresponding to the second text sample is determined as the category label of the second text sample; Based on the confidence scores of multiple predicted categories corresponding to the second text sample and the category labels of the second text sample, the second text samples that exceed the confidence score threshold are used as the second text samples obtained through filtering. Based on the second text sample obtained through the filtering process, a second text classification model for the second language is trained. The network depth of the second text classification model is greater than that of the first text classification model.

2. The method according to claim 1, characterized in that, The training of a first text classification model for the second language based on multiple third text samples in the second language and their corresponding category labels includes: Based on multiple third text samples of the second language and their corresponding category labels, the first text classification model is trained for the tth time. The first text classification model, trained for the tth time, performs a confidence-based filtering process on the multiple second text samples. Based on the results of the previous t screenings, the multiple third text samples, and their corresponding category labels, the first text classification model is trained for the (t+1)th time. The first text classification model trained for the Tth time is used as the first text classification model after training. Where t is the range of values ​​that satisfy 1 t T-1 is an integer, and its value increases from 1. T is an integer greater than 2, used to represent the total number of iterations for training.

3. The method according to claim 1, characterized in that, The second text sample obtained based on the filtering process is used to train a second text classification model for the second language, including: Determine the distribution of the second text sample obtained from the filtering process across multiple categories; When the distribution of the second text samples obtained by the filtering process satisfies the distribution balance condition in multiple categories and the number in each category exceeds the corresponding category number threshold, text samples corresponding to the category number threshold are randomly extracted from the text samples of each category in the second text samples obtained by the filtering process to construct a training set. A second text classification model for the second language is trained based on the training set.

4. The method according to claim 1, characterized in that, The second text sample obtained based on the filtering process is used to train a second text classification model for the second language, including: When the distribution of the second text samples obtained from the filtering process does not meet the distribution equilibrium condition across multiple categories, the second text samples in the corresponding categories undergo expansion processing based on synonyms to ensure that... The distribution of the second text sample obtained by the augmentation process across multiple categories satisfies the distribution equilibrium condition. A training set is constructed based on the second text sample obtained from the augmentation process. A second text classification model for the second language is trained based on the training set.

5. The method according to claim 1, characterized in that, The second text sample obtained based on the filtering process is used to train a second text classification model for the second language, including: A training set is constructed based on the multiple third text samples and the second text samples obtained through the filtering process, and a second text classification model for the second language is trained based on the training set.

6. The method according to claim 5, characterized in that, The construction of the training set based on the plurality of third text samples and the second text samples obtained through the filtering process includes: For each category of the second text sample obtained from the filtering process, perform the following processing: When the number of second text samples in the category is lower than the category number threshold, third text samples of the category will be randomly selected from the plurality of third text samples to supplement the second text samples of the category, so as to update the second text samples obtained by the filtering process; A training set is constructed based on the second text sample obtained from the updated filtering process.

7. The method according to claim 1, characterized in that, Before training a second text classification model for the second language based on the second text sample obtained from the filtering process, the method further includes: Based on the correspondence between the computing power of the text classification model and the number of text samples that can be processed per unit time, determine the number of target samples that match the computing power that can be used to train the second text classification model. From the training set constructed based on the second text samples obtained through the filtering process, text samples corresponding to the target number of samples are selected as samples for training a second text classification model for the second language.

8. The method according to claim 1, characterized in that, The training of a first text classification model for the second language based on multiple third text samples in the second language and their corresponding category labels includes: The first text classification model is used to predict multiple third text samples of the second language to obtain the confidence scores of the predicted categories corresponding to the multiple third text samples. Based on the confidence level of the predicted category and the category label of the third text sample, the loss function of the first text classification model is constructed. Update the parameters of the first text classification model until the loss function converges, and use the updated parameters of the first text classification model when the loss function converges as the parameters of the trained first text classification model.

9. The method according to claim 8, characterized in that, The step of predicting multiple third text samples of the second language using the first text classification model to obtain the confidence scores of the predicted categories corresponding to the multiple third text samples includes: Perform the following processing on any one of the plurality of third text samples: The following processing is performed using the first text classification model: The third text sample is encoded to obtain the encoded vector of the third text sample; The encoded vector of the third text sample is fused to obtain a fused vector; The fusion vector is subjected to nonlinear mapping processing to obtain the confidence level of the predicted category corresponding to the third text sample.

10. The method according to claim 9, characterized in that, The first text classification model includes multiple cascaded activation layers; The step of performing nonlinear mapping processing on the fusion vector to obtain the confidence score of the predicted category corresponding to the third text sample includes: The fusion vector is mapped by the first activation layer of the multiple cascaded activation layers. The mapping result of the first activation layer is output to the subsequent cascaded activation layers, so that mapping processing and mapping result output can continue in the subsequent cascaded activation layers until the last activation layer is output. The activation result output by the last activation layer is used as the confidence level of the predicted category corresponding to the third text sample.

11. A text classification method, characterized in that, The method includes: Get the text to be categorized; The text to be classified is in a second language different from the first language; The text to be classified is encoded using a second text classification model with a network depth greater than that of the first text classification model, to obtain the encoded vector of the text to be classified. The encoding vector of the text to be classified is non-linearly mapped to obtain the category corresponding to the text to be classified; Wherein, the second text classification model is trained using text samples of the second language selected by the first text classification model, and the text samples of the second language are obtained by machine translation of the text samples of the first language. Wherein, the first text classification model and the second text classification model are trained using the method described in any one of claims 1-10.

12. A training device for a text classification model, characterized in that, The device includes: The translation module is used to invoke a machine translation model based on multiple first text samples in a first language to obtain multiple second text samples that correspond one-to-one with the multiple first text samples; wherein, the multiple second text samples are in a second language different from the first language; The first training module is used to train a first text classification model for the second language based on multiple third text samples of the second language and their corresponding category labels. The filtering module is configured to perform the following processing on any one of the plurality of second text samples: perform prediction processing on the second text sample using the trained first text classification model to obtain the confidence scores of multiple predicted categories corresponding to the second text sample; determine the category label of the first text sample corresponding to the second text sample as the category label of the second text sample; and, based on the confidence scores of the multiple predicted categories corresponding to the second text sample and the category label of the second text sample, select the second text sample that exceeds the confidence score threshold as the second text sample obtained through the filtering process. The second training module is used to train a second text classification model for the second language based on the second text samples obtained by the filtering process; wherein the network depth of the second text classification model is greater than the network depth of the first text classification model.

13. The apparatus according to claim 12, characterized in that, The first training module is specifically used for: Based on multiple third text samples of the second language and their corresponding category labels, the first text classification model is trained for the tth time. The first text classification model, trained for the tth time, performs a confidence-based filtering process on the multiple second text samples. Based on the results of the previous t screenings, the multiple third text samples, and their corresponding category labels, the first text classification model is trained for the (t+1)th time. The first text classification model trained for the Tth time is used as the first text classification model after training. Where t is an integer whose value ranges from 1 to t to T-1 and increases from 1, and T is an integer greater than 2, used to represent the total number of iterations for training.

14. The apparatus according to claim 12, characterized in that, The second training module is specifically used for: Determine the distribution of the second text sample obtained from the filtering process across multiple categories; When the distribution of the second text samples obtained by the filtering process satisfies the distribution balance condition in multiple categories and the number in each category exceeds the corresponding category number threshold, text samples corresponding to the category number threshold are randomly extracted from the text samples of each category in the second text samples obtained by the filtering process to construct a training set. A second text classification model for the second language is trained based on the training set.

15. The apparatus according to claim 12, characterized in that, The second training module is specifically used for: When the distribution of the second text samples obtained from the filtering process does not meet the distribution equilibrium condition across multiple categories, the second text samples in the corresponding categories undergo expansion processing based on synonyms to ensure that... The distribution of the second text sample obtained by the augmentation process across multiple categories satisfies the distribution equilibrium condition. A training set is constructed based on the second text sample obtained from the augmentation process. A second text classification model for the second language is trained based on the training set.

16. The apparatus according to claim 12, characterized in that, The second training module is specifically used for: A training set is constructed based on the multiple third text samples and the second text samples obtained through the filtering process, and a second text classification model for the second language is trained based on the training set.

17. The apparatus according to claim 16, characterized in that, The second training module is also used for: For each category of the second text sample obtained from the filtering process, perform the following processing: When the number of second text samples in the category is lower than the category number threshold, third text samples of the category will be randomly selected from the plurality of third text samples to supplement the second text samples of the category, so as to update the second text samples obtained by the filtering process; A training set is constructed based on the second text sample obtained from the updated filtering process.

18. An electronic device, characterized in that, The electronic device includes: Memory, used to store executable instructions; The processor, when executing executable instructions stored in the memory, implements the training method of the text classification model according to any one of claims 1 to 10, or the text classification method according to claim 11.

19. A computer-readable storage medium, characterized in that, It stores executable instructions for implementing, when executed by a processor, the training method of the text classification model according to any one of claims 1 to 10, or the text classification method according to claim 11.

20. A computer program product comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by the processor, they implement the training method of the text classification model according to any one of claims 1 to 10, or the text classification method according to claim 11.

Images