Method for training a text classification model, text classification method, device, equipment, storage medium, and computer program

Abstract

A method, a device, an electronic device, and a computer-readable storage medium for training a text classification model related to artificial intelligence technology, the method including: performing a machine translation process on a plurality of first text samples in a first language through a machine translation model to obtain a plurality of second text samples that correspond one-to-one to the plurality of first text samples, where the plurality of second text samples are in a second language different from the first language (101); training a first text classification model for the second language based on a plurality of third text samples in the second language and their corresponding category labels (102); performing a confidence-based sorting process on the plurality of second text samples through the trained first text classification model (103); and training a second text classification model for the second language based on the second text samples obtained by the sorting process (104).

Description

[Technical field]

[0001] (CROSS-REFERENCE TO RELATED APPLICATIONS) This application claims priority to a Chinese patent application bearing application number 202011217057.9, filed with the China Patent Office on November 4, 2020, the entire contents of which are incorporated herein by reference.

[0002] The present application relates to artificial intelligence technology, and in particular to a method for training a text classification model, a text classification method, an apparatus, an electronic device, a computer-readable storage medium, and a computer program product. [Background technology]

[0003] Artificial intelligence (AI) is a comprehensive technology of computer science that can endow machines with the functions of perception, reasoning, and decision-making by studying various design principles and implementation methods for intelligent machines. AI technology is a comprehensive field that encompasses a wide range of fields, including natural language processing technology and machine learning / deep learning. As technology develops, AI technology is expected to be applied to more fields and demonstrate increasingly important value.

[0004] Text classification models are one of the important applications in the field of artificial intelligence, and can identify the category to which a text belongs. Text classification models are widely used in news recommendation systems, intent recognition systems, etc., that is, text classification models are a fundamental component of these complex systems.

[0005] However, the text classification models in the related art are targeted at a specific language, and when the text classification models are extended to text classification in other languages, the text classification models face the problem of a lack of labeling samples in other languages, and are unable to smoothly perform text classification tasks in other languages. Summary of the Invention [Problem to be solved by the invention]

[0006] The present embodiments provide a method for training a text classification model, a text classification method, an apparatus, an electronic device, a computer-readable storage medium, and a computer program product, which can automatically obtain cross-lingual text samples and improve the accuracy of text classification. [Means for solving the problem]

[0007] The technical solution according to the embodiment of the present application can be realized as follows.

[0008] An embodiment of the present application provides a method for training a text classification model, the method comprising: performing a machine translation process on a plurality of first text samples in a first language to obtain a plurality of second text samples in one-to-one correspondence with the plurality of first text samples, the plurality of second text samples adopting a second language different from the first language; training a first text classification model for the second language based on a third plurality of text samples in the second language and their corresponding category labels; performing a confidence-based classification process on the second plurality of text samples via the trained first text classification model; training a second text classification model for the second language based on the second text samples obtained by the screening process; The network depth of the second text classification model is greater than the network depth of the first text classification model.

[0009] An embodiment of the present application provides a text classification method, the method comprising: obtaining a classification target text, the classification target text adopting a second language different from the first language; performing an encoding process on the text to be classified through a second text classification model, the second text classification model having a network depth greater than that of the first text classification model, to obtain an encoded vector of the text to be classified; performing nonlinear mapping on the coded vector of the text to be classified to obtain a category corresponding to the text to be classified; The second text classification model is obtained by training a second language text sample obtained by filtering using the first text classification model, and the second language text sample is obtained by machine translating the first language text sample.

[0010] An embodiment of the present application provides an apparatus for training a text classification model, the apparatus comprising: a translation module configured to perform a machine translation process on a first plurality of text samples in a first language to obtain a second plurality of text samples in one-to-one correspondence with the first plurality of text samples; a first training module configured to train a first text classification model for the second language based on a third plurality of text samples in the second language and their corresponding category labels; a filtering module configured to perform a confidence-based filtering process on the second plurality of text samples via the trained first text classification model; and a second training module configured to train a second text classification model for the second language based on second text samples obtained by the screening process, wherein a network depth of the second text classification model is greater than a network depth of the first text classification model.

[0011] An embodiment of the present application provides a text classification device, the device comprising: an acquisition module configured to acquire a text to be classified, the text to be classified employing a second language different from the first language; a processing module configured to perform an encoding process on the text to be classified through a second text classification model having a network depth greater than that of a first text classification model, obtain an encoded vector of the text to be classified, and perform nonlinear mapping on the encoded vector of the text to be classified to obtain a category corresponding to the text to be classified, wherein the second text classification model is obtained by training a text sample in a second language obtained by sorting using the first text classification model, and the text sample in the second language is obtained by machine translating the text sample in the first language.

[0012] An embodiment of the present application provides an electronic device for training a text classification model, the electronic device comprising: a memory for storing executable instructions; A processor that, when executing executable instructions stored in the memory, performs a method for training a text classification model or a method for text classification provided by an embodiment of the present disclosure.

[0013] An embodiment of the present application provides a computer-readable storage medium having executable instructions stored thereon, the executable instructions causing a processor to perform a method for training a text classification model or a method for text classification provided by an embodiment of the present application.

[0014] An embodiment of the present application provides a computer program product including a computer program or instructions, which, when executed by a processor, implements the method for training a text classification model or the method for text classification provided by the embodiment of the present application.

[0015] The present embodiment has the following beneficial effects:

[0016] By obtaining a second text sample adopting a second language different from the first language through machine translation, and sorting the second text sample through the first text classification model, automatic acquisition of cross-lingual text samples is realized and the problem of insufficient text samples is alleviated. In addition, by training the second text classification model using the high-quality text sample obtained through sorting, the second text classification model can be enabled to perform accurate text classification, and the accuracy of text classification can be improved. [Brief description of the drawings]

[0017] [Figure 1] FIG. 1 is a schematic diagram of an application scene of a text classification system according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a schematic structural diagram of an electronic device for training a text classification model according to an embodiment of the present application. [Diagram 3] 1 is a flowchart of a training method based on a text classification model according to an embodiment of the present disclosure. [Figure 4] 1 is a flowchart of a training method based on a text classification model according to an embodiment of the present disclosure. [Diagram 5] 1 is a flowchart of a training method based on a text classification model according to an embodiment of the present disclosure. [Figure 6] 1 is a flowchart of repetitive training according to an embodiment of the present invention. [Figure 7] FIG. 2 is a schematic diagram of hierarchical softmax according to an embodiment of the present invention. [Figure 8] FIG. 2 is a schematic diagram of a cascaded encoder according to an embodiment of the present invention; [Figure 9] FIG. 2 is a schematic diagram of text set A and text set B according to an embodiment of the present invention. [Figure 10] FIG. 2 is a schematic diagram of a text set B1 according to an embodiment of the present invention. [Figure 11] 1 is a flowchart of active learning according to an embodiment of the present invention. [Figure 12]1 is a flowchart of reinforcement learning according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0018] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be described in more detail below with reference to the drawings. The embodiments described below do not limit the present application, and all other embodiments that can be obtained by those skilled in the art without any creative efforts are within the scope of protection of the present application.

[0019] Terms such as "first / second" in the embodiments of the present application are not intended to limit a particular order but to distinguish between similar objects, and "first / second" can be used to interchange a particular order or order when appropriate, so that it will be understood that the embodiments of the present application described herein can be performed in orders other than those shown or described herein.

[0020] Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art. The terms used herein are used only to describe the present application and are not intended to limit the present application.

[0021] Before describing the present embodiment in more detail, the nouns and terms used in the present embodiment will be explained. The nouns and terms used in the present embodiment will be explained as follows.

[0022] 1) Convolutional Neural Networks (CNN): A type of feedforward neural network (FNN) that includes convolutional calculations and has a deep structure, and is one of the representative algorithms of deep learning. Convolutional neural networks have a representation learning function and can perform shift-invariant classification on input images according to their hierarchical structure.

[0023] 2) Cross-lingual few-shot text classification: When moving from scenes in language A to scenes in language B and the budget for labeling language B samples is small, large-scale labeling of language B text can be achieved with only a small amount of labeling text in language B and a large amount of labeling text in language A, and a text classification model can be trained based on the large-scale labeling of language B text to achieve text classification in language B.

[0024] 3) Cross-lingual zero-shot text classification: When moving from a language A scene to a language B scene and there is a lack of budget (no manpower or time for product promotion), it is not possible to label language B samples. That is, large-scale labeling of language B can be realized only with a large amount of labeling text of language A, and a text classification model can be trained by the large-scale labeling of language B text to realize text classification of language B.

[0025] Text classification is widely used in content-related products, such as news classification, sentence classification, intent classification, infomercial products, forums, communities, e-commerce, etc. Generally, text classification targets specific languages ​​such as Chinese and English, but when the product needs to expand its other language services, the problem of insufficient labeling text will occur in the early stages of the product. For example, when selling a news reading product from the Chinese market to the English market, it is necessary to quickly attach corresponding labels to news in the English field. When analyzing the positive and negative emotions of Chinese users' comments, as the number of users increases or when selling the product to overseas markets, there will be more comments other than Chinese, so it is necessary to label the corresponding sentiment polarity for these comments as well.

[0026] In the long run, such other language texts can be trained and predicted after slowly accumulating a certain amount of labeling data through manual or other methods. However, in the early stages, labeling texts only manually takes a lot of time and effort and is not conducive to rapid product iteration. Therefore, in the early stages, it is expected that automatic labeling of large amounts of text can be realized through the accumulation of algorithms and labeling texts in existing languages.

[0027] All related technologies focus on single-language few-shot or zero-shot text classification, that is, they only solve the problem of insufficient single-language labeling samples, and lack cross-lingual text classification.

[0028] In order to solve the above problems, embodiments of the present application provide a method for training a text classification model, a text classification method, an apparatus, an electronic device, a computer-readable storage medium, and a computer program product, which can automatically obtain cross-lingual text samples and improve the accuracy of text classification.

[0029] The text classification model training method and text classification method provided by the embodiments of the present application may be independently implemented by a terminal / server, or may be implemented cooperatively by a terminal and a server. For example, a terminal may independently execute a text classification model training method described below, or a terminal may send a text classification request for a specific language to a server, and the server may execute a text classification model training method based on the received text classification for the language, and perform a text classification task for the language based on the trained text classification model.

[0030] The electronic device for training the text classification model provided by the embodiment of the present application may be various types of terminal devices or servers, where the server may be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The terminal may be, but is not limited to, a smartphone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, etc. The terminal and the server may be directly or indirectly connected via wired or wireless communication, and the present application is not limited thereto.

[0031] Taking a server as an example, it can be, for example, a server cluster deployed in the cloud, and artificial intelligence cloud services (AIaaS: AI as a Service) can be opened to users. The AIaaS platform splits some common AI services and provides them as standalone or packaged services on the cloud. This service model is similar to an AI theme mall, and all users can access one or more AI services provided by the AIaaS platform through an application programming interface.

[0032] For example, one of the artificial intelligence cloud services may be a text classification model training service, that is, a program for training the text classification model provided by the embodiment of the present application is encapsulated in a server of the cloud. A user calls the text classification model training service in the cloud service through a terminal (running a client such as a news client or a reading client), and the server deployed in the cloud calls the encapsulated text classification model training program, obtains a second text sample adopting a second language different from the first language through a machine translation model based on a first text sample of a first language, selects the second text sample through the first text classification model, trains the second text classification model using the second text sample obtained by the selection, and performs text classification with the trained second text sample, thereby executing a subsequent news app, reading app, etc. For example, in the case of a news app, the text is English news, and the category of each piece of recommended news, such as entertainment news or sports news, is determined through the trained second text classification model (for English news classification), and then each piece of recommended news is selected based on the news category, the recommended news is obtained, and the recommended news is displayed to the user, thereby realizing tailored news recommendation; in the case of a reading app, the text is Chinese text, and the category of each piece of recommended text (e.g., encouraging text, legal text, educational text, etc.) is determined through the trained second text classification model (for Chinese text classification), and then each piece of recommended text is selected based on the text category, the recommended text is obtained, and the recommended text is displayed to the user, thereby realizing tailored text recommendation.

[0033] Referring to FIG. 1, FIG. 1 is a schematic diagram of an application scene of the text classification system 10 provided by an embodiment of the present application, in which a terminal 200 is connected to a server 100 via a network 300, and the network 300 may be a wide area network, a LAN, or a combination of the two.

[0034] The terminal 200 (running a client such as a news client) may be used to obtain text to be classified in a specific language; for example, when a developer inputs text to be classified in a specific language via the terminal, the terminal automatically obtains a text classification request in the specific language.

[0035] In some embodiments, a client executed on a terminal may incorporate a text classification model training plug-in to locally implement the text classification model training method on the client. For example, after the terminal 200 obtains a text to be classified in a second language different from the first language, the terminal 200 calls the text classification model training plug-in to realize the text classification model training method, obtains a second text sample (using a second language) corresponding to the first text sample (using the first language) through a machine translation model, selects the second text sample through the first text classification model, trains the second text classification model using the selected text sample, performs text classification based on the trained second text sample, and then executes a news application, a reading application, etc.

[0036] In some embodiments, after a text classification of a specific language is requested, the terminal 200 calls a text classification model training interface (which can be provided as a cloud service, i.e., a text classification model training service) of the server 100, and the server 100 obtains a second text sample (using a second language) corresponding to the first text sample (using a first language) through a machine translation model, selects the second text sample through the first text classification model, trains the second text classification model using the text sample obtained by the selection, performs text classification based on the trained second text sample, and then executes a news application, a reading application, etc.

[0037] The structure of the electronic device for training a text classification model provided by the embodiment of the present application will be described below. Referring to FIG. 2, FIG. 2 is a schematic structural diagram of an electronic device 500 for training a text classification model provided by the embodiment of the present application. Taking the electronic device 500 as a server as an example, the electronic device 500 for training a text classification model shown in FIG. 2 includes at least one processor 510, a memory 550, at least one network interface 520, and a user interface 530. Each component in the electronic device 500 is coupled via a bus system 540. It can be understood that the bus system 540 is used to realize connection communication between these components. The bus system 540 includes a power bus, a control bus, and a status signal bus in addition to a data bus. However, for clarity of explanation, various buses are represented as the bus system 540 in FIG. 2.

[0038] The processor 510 may be, for example, an integrated circuit chip with signal processing capabilities, such as a general purpose processor, a digital signal processor (DSP), a programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, etc., where the general purpose processor may be a microprocessor or any conventional processor, etc.

[0039] The memory 550 may include volatile memory or non-volatile memory, or may include both volatile and non-volatile memory, where the non-volatile memory may be a read only memory (ROM) and the volatile memory may be a random access memory (RAM). The memory 550 described in the embodiments of the present application includes any suitable type of memory. The memory 550 includes one or more storage devices that are physically separate from the processor 510.

[0040] In some embodiments, memory 550 may store data to support various operations, examples of which include programs, modules, and data structures, or a subset or superset thereof, as illustratively described below.

[0041] The operating system 551 includes system programs for processing various basic system services and executing hardware-related tasks, such as a framework layer, a core library layer, and a drive layer, and is configured to realize various basic operations and process hardware-based tasks.

[0042] The network communications module 553 is configured to reach other computing devices via one or more (wired or wireless) network interfaces 520; exemplary network interfaces 520 include Bluetooth, Wireless Compatibility (WiFi), Universal Serial Bus (USB), and the like.

[0043] In some embodiments, the text classification model training apparatus provided by the embodiments of the present application may be implemented in software, for example, a text classification model training plug-in in the above-mentioned terminal, or a text classification model training service in the above-mentioned server, but is of course not limited thereto, and the text classification model training apparatus provided by the embodiments of the present application may be provided as various forms of software embodiments, including an app, software, a software module, a script, or code.

[0044] FIG. 2 shows a text classification model training device 555 stored in a memory 550, which may be software in the form of a program or plug-in, such as a text classification model training plug-in, and includes a series of modules including a translation module 5551, a first training module 5552, a screening module 5553, and a second training module 5554, where the translation module 5551, the first training module 5552, the screening module 5553, and the second training module 5554 are configured to realize the text classification model training function provided by the embodiment of the present application.

[0045] As mentioned above, the training method of the text classification model provided by the embodiment of the present application can be implemented by various types of electronic devices. Referring to Figure 3, Figure 3 is a flow chart of the training method of the text classification model provided by the embodiment of the present application, which will be described with reference to the steps shown in Figure 3.

[0046] In the following steps, the network depth of the second text classification model is greater than the network depth of the first text classification model, i.e., the text classification ability of the second text classification model is stronger than the classification ability of the first text classification model, so the number of text samples required for training the second text classification model is greater than the number of text samples required for training the first text classification model.

[0047] In the steps below, the first text sample adopts a first language, and the second text sample and the third text sample adopt a second language different from the first language, for example, the first text sample is a Chinese sample, and the second text sample and the third text sample are English samples.

[0048] In step 101, a machine translation process is performed on a plurality of first text samples in a first language to obtain a plurality of second text samples that correspond one-to-one to the plurality of first text samples.

[0049] For example, when a developer inputs a text classification command for a second language through a terminal, the terminal automatically obtains a text classification request for the second language and sends the text classification request for the second language to a server. After receiving the text classification request for the second language, the server obtains a large number of labeled first text samples from a sample library, where the first text samples adopt a first language different from the second language, and invokes a machine translation model to perform machine translation on the multiple first text samples to obtain multiple second text samples that correspond one-to-one to the multiple first text samples, where the category labeling of the second text samples inherits the category labeling corresponding to the first text samples, that is, there is no need for manual labeling, and the labeling load caused by frequent manual labeling can be greatly reduced.

[0050] In step 102, a first text classification model for the second language is trained based on a third plurality of text samples in the second language and their corresponding category labels.

[0051] Here, there is no clear priority between step 101 and step 102. After receiving the text classification request in the second language, the server obtains a small number of labeled third text samples from a sample library, and uses the third text samples and corresponding category labels to train a first text classification model, so that the trained first text classification model can perform text classification based on the second language.

[0052] In some embodiments, the step of training a first text classification model for the second language based on the third text samples in the second language and their corresponding category labels includes: training the first text classification model a t-th time based on the third text samples in the second language and their corresponding category labels; performing a t-th screening process on the second text samples through the first text classification model after the t-th training based on a confidence level; training the first text classification model a t+1-th time based on the previous t screening results, the third text samples, and their corresponding category labels; and setting the first text classification model after the T-th training as the trained first text classification model, where t is a positive integer that increases sequentially and a value range satisfies 1≦t≦T−1, and T is an integer greater than 2 and is used to represent a total number of iterative trainings.

[0053] For example, by repeatedly training a first text classification model based on a plurality of third text samples in a second language and their corresponding category labels, more and more good third text samples are selected by the gradually optimized first text classification model, and subsequent reinforcement training is performed to train a second text classification model.

[0054] As shown in FIG. 6 , a first training is performed on a first text classification model based on a plurality of third text samples in a second language and their corresponding category labels; a first selection process is performed on a plurality of second text samples of the first text classification model after the first training based on a reliability; a second training is performed on the first text classification model based on a result of the first selection, the plurality of third text samples, and their corresponding category labels; a second selection process is performed on the second text samples excluding the first selection result among the plurality of second text samples based on a reliability using the first text classification model after the second training; a third training is performed on the first text classification model based on the results of the previous two selections, the plurality of third text samples, and their corresponding category labels; and the above training process is repeated until the first text classification model is trained for the Tth time, and the first text classification model after the Tth training is a trained first text classification model.

[0055] 4, FIG. 4 is an exemplary flowchart of a method for training a text classification model provided by an embodiment of the present application. FIG. 4 shows that step 102 in FIG. 3 can be realized by steps 1021 to 1023 shown in FIG. 4. In step 1021, a prediction process is performed on a plurality of third text samples in a second language through a first text classification model, and confidences of predicted categories corresponding to each of the plurality of third text samples are obtained. In step 1022, a loss function of the first text classification model is constructed based on the confidences of the predicted categories and the category labels of the third text samples. In step 1023, parameters of the first text classification model are updated until the loss function converges, and the updated parameters of the first text classification model when the loss function converges are set as parameters of the trained first text classification model.

[0056] For example, after determining a value of a loss function of the first text classification model based on the confidence of the predicted category and the category label of the third text sample, it can be determined whether the value of the loss function of the first text classification model exceeds a predetermined threshold, and if the value of the loss function of the first text classification model exceeds the predetermined threshold, determine an error signal of the first text classification model based on the loss function of the first text classification model, back-propagate the error information in the first text classification model, and update model parameters of each layer during the propagation process.

[0057] Here, the backpropagation is explained as follows: the training sample data is input to the input layer of the neural network model, passes through the hidden layer, and finally reaches the output layer to output the result, which is the forward propagation process of the neural network model; since there is an error between the output result of the neural network model and the actual result, the error between the output result and the actual value is calculated, and the error is backpropagated from the output layer to the hidden layer and propagated to the input layer; in the backpropagation process, the value of the model parameters is adjusted according to the error, and the above process is continuously repeated until convergence. Here, the first text classification model belongs to the neural network model.

[0058] In some embodiments, the step of performing a prediction process on a plurality of third text samples in a second language through a first text classification model and obtaining a confidence level of a predicted category corresponding to each of the plurality of third text samples includes performing, for any third text sample of the plurality of third text samples, a process of performing an encoding process on the third text sample through the first text classification model to obtain an encoded vector of the third text sample, performing a fusion process on the encoded vector of the third text sample to obtain a fusion vector, performing a nonlinear mapping process on the fusion vector, and obtaining a confidence level of a predicted category corresponding to the third text sample.

[0059] For example, the first text classification model is a fast text classification model (fasttext), and the first text classification model in the embodiment of the present application is not limited to fasttext, and fasttext includes an input layer, a hidden layer, and an output layer, and fasttext can be quickly trained with a small number of third text samples, so that fasttext can quickly perform the text classification task of the second language. For example, the third text sample is encoded through the input layer to obtain the encoded vector of the third text sample, then the encoded vector of the third text sample is fused through the hidden layer to obtain the fused vector, and finally, a nonlinear mapping is performed on the fused vector through the output layer (i.e., a mapping process is performed by an activation function (such as softmax)) to obtain the confidence of the predicted category corresponding to the third text sample.

[0060] In some embodiments, the first text classification model includes a plurality of cascaded activation layers, and the step of performing a nonlinear mapping process on the fusion vector to obtain a confidence of a predicted category corresponding to the third text sample includes: performing a mapping process of a first activation layer on the fusion vector through a first activation layer of the plurality of cascaded activation layers; outputting the mapping result of the first activation layer to a subsequent cascaded activation layer, and continuing the mapping process and outputting the mapping result through the subsequent cascaded activation layers until the mapping result is output to a last activation layer; and setting the activation result output by the last activation layer as the confidence of the category corresponding to the third text sample.

[0061] As shown in FIG. 7, by performing activation calculation by hierarchical softmax, it is possible to avoid obtaining the confidence of the predicted category by a one-time activation calculation, and the calculation complexity can be reduced by multi-layer activation calculation. For example, hierarchical softmax includes T activation layers, performs one hierarchical softmax calculation for each activation layer, performs mapping of the first activation layer on the fusion vector through the first activation layer, obtains the first mapping result, outputs the first mapping result to the second activation layer, performs mapping of the second activation layer on the first mapping result through the second activation layer, obtains the second mapping result, and continues until outputting to the Tth activation layer, and the activation result output by the Tth activation layer is the confidence of the category corresponding to the third text sample. Here, T is the total number of activation layers.

[0062] In some embodiments, the step of performing an encoding process on a third text sample and obtaining an encoded vector of the third text sample includes performing a window sliding process on the third text sample to obtain a plurality of segment sequences, where the size of the window is N, and N is a natural number; performing a mapping process on the plurality of segment sequences based on a thesaurus to obtain sequence vectors corresponding to the plurality of segment sequences respectively; and performing a concatenation process on the sequence vectors corresponding to the plurality of segment sequences respectively to obtain an encoded vector of the third text sample.

[0063] Subsequent to the above example, when the segment sequence includes N characters, the step of performing a window sliding process on the third text sample to obtain a plurality of segment sequences includes, for the i-th character in the third text sample, performing the following operations: obtaining the i-th character to the (i + N - 1)-th character in the third text sample, and concatenating the i-th character to the (i + N - 1)-th character to obtain a concatenation result as the segment sequence, where 0 < i ≤ M - N + 1, M is the number of characters in the third text sample, and M is a natural number. Thereby, a better encoded vector can be generated for rare characters, and in the thesaurus, even if a character does not appear in the training corpus, an encoded vector corresponding to the character granularity can be constructed from the window of the character granularity, and the first text classification model can also learn partial information of the local character order, whereby the first text classification model can maintain the character order information during training.

[0064] Following the above example, when the segment sequence contains N words, for the third text sample, performing window sliding processing to obtain a plurality of segment sequences includes executing the following processing for the j-th word in the third text sample: obtaining the j-th word to the (j + N - 1)-th word in the third text sample, combining the j-th word to the (j + N - 1)-th word, and using the combined result as the segment sequence, where 0 < j ≦ K - N + 1, K is the number of words in the third text sample, and K is a natural number. Thereby, a better encoded vector is generated for rare words (phrases), and in the thesaurus, even if a phrase does not appear in the training corpus, an encoded vector corresponding to the word granularity can be constructed from a window of word granularity, and local word order partial information can also be learned by the first text classification model, whereby the first text classification model can maintain word order information during training.

[0065] In step 103, through the trained first text classification model, screening processing based on reliability is performed on a plurality of second text samples.

[0066] For example, after the server obtains the third text sample and the trained first text classification model, by performing screening processing based on reliability on a plurality of second text samples through the trained first text classification model, high-quality second text samples can be selected, and the second text classification model can be trained with the high-quality second text samples.

[0067] In some embodiments, the step of performing a selection process based on the confidence levels for the plurality of second text samples via the trained first text classification model includes performing the following processes for any second text sample of the plurality of second text samples: performing a prediction process for the second text sample via the trained first text classification model, obtaining confidence levels of a plurality of predicted categories corresponding to the second text sample, determining a category label of the first text sample corresponding to the second text sample as a category label of the second text sample, and determining a second text sample that exceeds a confidence threshold based on the confidence levels of the plurality of predicted categories corresponding to the second text sample and the category label of the second text sample as the second text sample obtained by the selection process.

[0068] For example, through the trained first text classification model, an encoding process is performed on the second text sample to obtain an encoded vector of the second text sample, a fusion process is performed on the encoded vector of the second text sample to obtain a fusion vector, a nonlinear mapping process is performed on the fusion vector to obtain confidence levels of multiple predicted categories corresponding to the second text sample, a predicted category that matches the category label of the second text sample is determined from the multiple predicted categories corresponding to the second text sample, and if the confidence level of the matched predicted category exceeds a confidence threshold, the second text sample is regarded as the second text sample obtained by the sorting process.

[0069] In step 104, a second text classification model for the second language is trained based on the second text samples obtained from the screening process.

[0070] For example, the server screens a large number of high-quality second text samples through the trained first text classification model, and then realizes automatic construction of cross-lingual text samples (i.e., second text samples in a second language that have the category labels of the corresponding first text samples, i.e., no manual labeling is required), and trains the second text classification model with a large number of high-quality second text samples, so that the trained second text classification model can accurately perform text classification based on the second language, and improve the accuracy of text classification in the second language.

[0071] The training method of the present embodiment can obtain sufficient second text samples for training the second text classification model, so that the present embodiment only needs to train the second text classification model with the second text samples obtained by the sorting process.

[0072] Here, after the server obtains the trained second text classification model, it performs text classification on the text to be classified in response to a text classification request for the second language, i.e., performs an encoding process on the text to be classified through the trained second text classification model to obtain an encoded vector of the text to be classified, performs nonlinear mapping on the encoded vector of the text to be classified to obtain a category corresponding to the text to be classified, and can perform subsequent news applications, reading applications, etc. using the category corresponding to the text to be classified.

[0073] Referring to FIG. 5, FIG. 5 is an exemplary flowchart of a method for training a text classification model provided by an embodiment of the present application. FIG. 5 shows that step 104 in FIG. 3 can be realized by steps 1041 to 1043 shown in FIG. 5. In step 1041, a distribution of a plurality of categories of the second text sample obtained by the sorting process is determined. In step 1042, if the distribution of a plurality of categories of the second text sample obtained by the sorting process satisfies a distribution equilibrium condition and the number of each category exceeds a corresponding category number threshold, a text sample corresponding to the category number threshold is randomly extracted from the text samples of each category in the second text sample obtained by the sorting process to construct a training set. In step 1043, a second text classification model for a second language is trained based on the training set.

[0074] For example, after the server obtains a large number of second text samples for training the second text classification model, the distribution of the second text samples in multiple categories obtained by the sorting process is analyzed to check whether it meets the distribution balance condition, i.e., the jitter of the number of different categories, for example, when the standard deviation is used to measure the jitter of the number of different categories, the larger the jitter, the more uneven the distribution of the text samples in multiple categories. If the distribution of the second text samples in multiple categories obtained by the sorting process meets the distribution balance condition and the number of each category exceeds the category number threshold, a text sample corresponding to the category number threshold is extracted from the text samples of each category in the second text samples obtained by the sorting process to construct a training set, thereby improving the accuracy of text classification.

[0075] In some embodiments, the step of training a second text classification model for a second language based on the second text samples obtained by the sorting process includes the steps of: if the distribution of the second text samples obtained by the sorting process in multiple categories does not satisfy a distribution equilibrium condition, performing a synonym-based extension process on the second text samples of the category with a smaller distribution so that the distribution of the second text samples obtained by the extension process in multiple categories satisfies the distribution equilibrium condition; constructing a training set based on the second text samples obtained by the extension process; and training a second text classification model for the second language based on the training set.

[0076] If the number of second text samples obtained by the selection process in each category is smaller than the corresponding category number threshold, an extension process based on synonyms is performed on the second text samples in the corresponding category so that the number of second text samples obtained by the extension process in each category exceeds the corresponding category number threshold, and a training set is constructed based on the second text samples obtained by the extension process.

[0077] Here, the specific extension process is as follows: for any one of the multiple third text samples and the second text samples obtained by the selection process, the following process is performed: a matching process is performed between the thesaurus (including the correspondence between various synonyms) and the words in the text sample, a matching word corresponding to the word in the text sample is obtained, and a replacement process is performed on the words in the text sample based on the matching word to obtain a new text sample, and the category label corresponding to the text sample is the category label of the new text sample. Through the synonym replacement method, the text samples of the second language can be greatly expanded to realize the training of the second text classification model.

[0078] In some embodiments, training a second text classification model for the second language based on the second text samples obtained by the screening process includes constructing a training set based on the third plurality of text samples and the second text samples obtained by the screening process, and training a second text classification model for the second language based on the training set.

[0079] For example, the step of constructing a training set based on a plurality of third text samples and the second text samples obtained by the sorting process includes traversing each category of the second text samples obtained by the sorting process, and performing the following process: if the number of second text samples in a category is lower than a category number threshold of the category, updating the second text samples obtained by the sorting process by supplementing the second text samples of the category with third text samples of the category randomly extracted from the plurality of third text samples, and performing a process of constructing a training set based on the updated second text samples obtained by the sorting process.

[0080] Following the above example, when the text samples of a certain category are relatively few, or the distribution of a certain category is unbalanced, it can be supplemented with a third text sample. For example, when the number of second text samples in a category is less than the category number threshold of the category, it indicates that the text samples of the category are relatively few, and the third text samples of the category are randomly extracted from multiple third text samples to supplement the second text samples of the category, so that the text samples of the category in the second text samples are more sufficient.

[0081] In some embodiments, in order to avoid the problem of overfitting caused by training the second classification model with a large number of samples, appropriate training can be performed based on the computing power of the second text classification model according to the number of corresponding text samples that match the computing power. Before training the second text classification model for the second language based on the second text samples obtained by the sorting process, the number of target samples that match the computing power available for training the second text classification model is determined based on the corresponding relationship between the computing power (computing function) of the text classification model and the number of text samples that can be calculated per unit time, and text samples corresponding to the number of target samples are selected from the training set constructed based on the second text samples obtained by the sorting process to be used as samples for training the second text classification model for the second language.

[0082] In some embodiments, the step of training a second text classification model for the second language based on the second text samples obtained by the screening process includes the steps of: performing a prediction process on the second text samples obtained by the screening process through the second text classification model to obtain a predicted category corresponding to the second text sample obtained by the screening process; constructing a loss function of the second text classification model based on the predicted category corresponding to the second text sample obtained by the screening process and the corresponding category label; updating parameters of the second text classification model until the loss function converges, and setting the updated parameters of the second text classification model when the loss function converges as parameters of the trained second text classification model.

[0083] For example, after determining a value of a loss function of the second text classification model based on the predicted category and the corresponding category label corresponding to the second text sample obtained by the sorting process, it is possible to determine whether the value of the loss function of the second text classification model exceeds a predetermined threshold; if the value of the loss function of the second text classification model exceeds the predetermined threshold, it is possible to determine an error signal of the second text classification model based on the loss function of the second text classification model, back-propagate the error information in the second text classification model, and update the model parameters of each layer during the propagation process.

[0084] In some embodiments, the second text classification model includes a plurality of cascaded encoders, and the step of performing a prediction process on the second text sample obtained by the sorting process through the second text classification model and obtaining a predicted category corresponding to the second text sample obtained by the sorting process includes performing the following process for any text sample of the second text sample obtained by the sorting process: performing an encoding process of the first encoder on the text sample through a first encoder of the cascaded plurality of encoders, outputting the encoding result of the first encoder to a subsequent cascaded encoder, continuing the encoding process and outputting the encoding result through the subsequent cascaded encoders until the encoding result is output to a last encoder, taking the encoding result output by the last encoder as an encoded vector corresponding to the text sample, performing a nonlinear mapping on the encoded vector of the text sample, and obtaining a predicted category corresponding to the text sample.

[0085] As shown in FIG. 8, by performing encoding operations by cascaded encoders, rich feature information of a text sample can be extracted. For example, a first encoder performs encoding processing on the text sample, obtains a first encoding result, outputs the first encoding result to a second encoder, and the second encoder performs encoding processing on the first encoding result, obtains a second encoding result, and continues until the encoding result is output to the S-th encoder, and finally performs nonlinear mapping on the encoding vector of the text sample to obtain a predicted category corresponding to the text sample, where S is the total number of encoders.

[0086] Following the above example, the step of continuing the encoding process and outputting the encoding result in the subsequent cascaded encoder includes performing the following process through the y-th encoder of the cascaded multiple encoders: performing self-attention processing on the encoding result of the y-1th encoder, obtaining the y-th self-attention vector, performing residual connection processing on the y-th self-attention vector and the encoding result of the y-1th encoder, obtaining the y-th residual vector, performing a nonlinear mapping process on the y-th residual vector, obtaining the y-th mapping vector, performing residual connection processing on the y-th mapping vector and the y-th residual vector, taking the result of the residual connection as the encoding result of the y-th encoder, and outputting the encoding result of the y-th encoder to the y+1th encoder, where y is a positive integer that increases sequentially, and the value range satisfies 2≦y≦H-1, and H is an integer greater than 2, and is used to represent the number of the cascaded multiple encoders.

[0087] In addition, after obtaining the trained second text classification model, a method for performing text classification in a second language through the trained second text classification model includes the steps of obtaining a text to be classified, the text to be classified being a second language different from the first language; performing an encoding process on the text to be classified through the second text classification model having a network depth greater than that of the first text classification model to obtain an encoded vector of the text to be classified; and performing nonlinear mapping on the encoded vector of the text to be classified to obtain a category corresponding to the text to be classified, wherein the second text classification model is obtained by training text samples in the second language obtained by screening using the first text classification model, and the text samples in the second language are obtained by machine translating the text samples in the first language.

[0088] Following the above example, the second text classification model includes a plurality of cascaded encoders, and performs the following process on the text to be classified: through a first encoder of the plurality of cascaded encoders, the first encoder performs an encoding process on the text to be classified, and the encoding result of the first encoder is output to a subsequent cascaded encoder, and the encoding process and the output of the encoding result are continued through the subsequent cascaded encoders, until the encoding result is output to the last encoder, and the encoding result output by the last encoder is used as the encoding vector of the corresponding text to be classified, and a nonlinear mapping is performed on the encoding vector of the text to be classified to obtain a category corresponding to the text to be classified.

[0089] For example, by performing encoding operations using cascaded encoders, it is possible to extract rich feature information of the text to be classified. For example, a first encoder performs encoding processing on the text to be classified, obtains a first encoding result, outputs the first encoding result to a second encoder, and the second encoder performs encoding processing on the first encoding result, obtains a second encoding result, and continues until the encoding result is output to the S-th encoder, and finally performs nonlinear mapping on the encoding vector of the text to be classified to obtain a category corresponding to the text to be classified, where S is the total number of encoders.

[0090] Following the above example, the step of continuing the encoding process and outputting the encoding result through the subsequent cascaded encoder includes performing the following process through the y-th encoder of the cascaded multiple encoders: performing self-attention processing on the encoding result of the y-1th encoder, obtaining the y-th self-attention vector, performing residual connection processing on the y-th self-attention vector and the encoding result of the y-1th encoder, obtaining the y-th residual vector, performing a nonlinear mapping process on the y-th residual vector, obtaining the y-th mapping vector, performing residual connection processing on the y-th mapping vector and the y-th residual vector, as the encoding result of the y-th encoder, and outputting the encoding result of the y-th encoder to the y+1th encoder, where y is a positive integer that increases sequentially, and the value range satisfies 2≦y≦H-1, and H is an integer greater than 2, and is used to represent the number of the cascaded multiple encoders.

[0091] Next, an application example of the present embodiment in an actual application scene will be described.

[0092] Text classification is widely used in content-related products such as news classification, sentence classification, intent classification, infomercial products, forums, communities, e-commerce, etc., so that text recommendations, emotion induction, etc. are performed based on the text classification category. Generally, text classification targets specific languages ​​such as Chinese and English, and there is a need to expand the product's other language services (for example, to sell news reading products from the Chinese market to the English market). When a user reads news, recommendations are made based on the labels of English news, so that English news that matches the user's interests can be recommended. When analyzing the positive and negative emotions of Chinese users' comments, when a product is sold to an overseas market, the user's emotions can be appropriately induced based on the labels of English comments when the user comments, so that the user can be prevented from continuously generating negative emotions.

[0093] In the following, the training method of the text classification model and the text classification method proposed in the embodiment of the present application are specifically described with reference to the above application scenarios, which are used to increase the amount of samples in language B from samples in language A by the existing machine translation model. However, since the text translated by the algorithm will have some deviations and errors, an active learning method is adopted to select high-quality samples from the translated text for subsequent training.

[0094] The following is a detailed description of the text classification model training method and text classification method proposed in the present embodiment, which includes two parts: A) data preparation, B) algorithm framework, and C) prediction.

[0095] A) Data preparation The present embodiment is intended for a situation where there are no large number of samples (no labels), so it is not possible to train a large-scale pre-training model to extract text content. As shown in Figure 9, in the present embodiment, there is a partial text set A (Text A: a text set of language A) (including a first text sample) and a small amount of text set B (Text B: a text set of language B) (including a third text sample), where Text A and Text B are samples with category labels, and Text B has a smaller number of labels than Text A, so its proportion is very small.

[0096] Here is a labeling sample of Text A:<X_A,Y> , Text B<X_B,Y> Here, X_A represents text in Text A, and X_B represents text in Text B. The labels of Text A and Text B are common and are both represented by Y. For example, category 0 (Y=0) represents entertainment type news, and category 1 (Y=1) represents sports type news, where 0 and 1 are common and have nothing to do with language.

[0097] B) Algorithmic Framework Here, the algorithm framework in the embodiment of the present application includes 1) sample reinforcement, 2) active learning, and 3) reinforcement training. In the following, sample reinforcement, active learning, and reinforcement training will be specifically described.

[0098] 1) Sample Enhancement First, as shown in FIG. 10, a machine translation model (used to translate language A into language B) converts each text X_A in language A in Text A into a text in language B to form a corresponding text set B1 (Text B1: a text set in language B formed by translation).

[0099] Through this method of sample enrichment, we obtain two kinds of labeled texts: one is the original sample set with a small amount of manual labeling, whose manual labeling is very accurate; the other is a sample Text B1 (including the second text sample) with a large amount of labeling (whose labeling corresponds to the labeling in Text A) obtained after being converted by a machine translation model; Text B1 may contain noise, errors, etc., and is not as accurate as the content of Text B.

[0100] 2) Active learning To select high-quality samples in Text B1, we adopt an active learning method, and the whole process is shown in Figure 11.

[0101] In step 1, first, a weak classifier (first text classification model) (e.g., a shallow classifier such as fasttext) is trained using the manually labeled Text B, and then the weak classifier is applied to Text B1 to make predictions and select high-confidence samples from Text B1. For example, assuming that the confidence threshold is 0.8, if the confidence of the predicted label Y=2 of a certain sample X_B1 in Text B1 is 0.87 (greater than 0.8), the category of the sample X_B1 is deemed to be 2, and thus the labeled training samples<X_B1,Y=2> Get the.

[0102] In step 2, construct a new training sample set (Text Set B1', Text B1') with these high-confidence, labelled samples, and continue training the weak classifier based on Text B1' and Text B. Once training is complete, repeat step 1 and apply the weak classifier to the remaining samples after sorting out Text B1 (the remaining samples refer to the text remaining after sorting out the high-confidence samples from Text B1).

[0103] In step 3, when the confidence obtained by predicting samples in Text B1 does not exceed a specified confidence threshold, the samples of Text B1 remaining after the selection are considered to be low-quality samples, and the iterative training is stopped.

[0104] 3) Strength training As shown in Figure 12, Text B' obtained in the above steps is mixed with Text B, and then a strong classifier (second text classification model) (such as a deep neural network (BERT: Bidirectional Encoder Representations from Transformers)) is trained.

[0105] C) Prediction The strong classifier obtained by training is used as the final text classification model for text classification of language B. For example, when a news reading product is sold from the Chinese market to the English (language B) market, the strong classifier obtained by training can quickly attach a corresponding label to English news, and when a user reads news, news recommendations are made based on the labels of English news, so that English news that matches the user's interests can be recommended; when analyzing the positive and negative emotions of Chinese user comments, when a product is sold to an overseas market (language B), there are many comments that are not Chinese, i.e., English comments, so the strong classifier obtained by training can quickly attach a corresponding emotion label to English news, and when a user makes a comment, the user's emotion can be appropriately induced based on the labels of the English comments, so that the user can be prevented from continuously generating negative emotions.

[0106] In summary, the text classification model training method and text classification method of the present application embodiment adopt a language B different from language A to obtain a second text sample through a machine translation model, and select the second text sample through a weak classifier, thereby realizing automatic acquisition of cross-lingual text samples and alleviating the problem of insufficient text samples. In addition, the high-quality text samples obtained by the selection are used to train a strong classifier, enabling the strong classifier to perform accurate text classification, and improving the accuracy of text classification.

[0107] Above, the training method of the text classification model provided by the embodiment of the present application has been described in connection with the exemplary application and implementation of the server provided by the embodiment of the present application. The embodiment of the present application further provides a training device of the text classification model, and in practical application, each functional module in the training device of the text classification model can be implemented cooperatively by hardware resources of electronic equipment (such as terminal equipment, server or server cluster), such as computing resources such as processors, communication resources (for example, used to support the realization of various types of communication such as optical cable, cellular, etc.), and memory. FIG. 2 shows a training device 555 of the text classification model stored in memory 550, and the training device 555 of the text classification model can be software in the form of programs, plug-ins, etc., for example, applications designed in programming languages ​​such as software C / C++, Java, or dedicated software modules in large-scale software systems, application interfaces, plug-ins, cloud services, etc., and examples of different realization methods will be described below.

[0108] Example 1: When the training device for the text classification model is a mobile app and module The text classification model training device 555 in the embodiment of the present application can be designed in a programming language such as software C / C++, Java, etc., and provided as a software module embedded in various mobile apps based on systems such as Android or iOS (stored in the form of executable instructions in a storage medium on the mobile side and executed by a processor on the mobile side), and can directly use the computing resources of the mobile side itself to complete related information recommendation tasks, and can periodically or irregularly transmit the processing results to a remote server through various network communication methods, or can be stored locally on the mobile side.

[0109] Example 2: Training a text classification model When the device is a server application and platform The text classification model training device 555 in the present embodiment can be designed in a programming language such as software C / C++, Java, etc., and provided as a software module executed on the server side (stored in a server-side storage medium in the form of executable instructions and executed by a server-side processor), and the server can use its own computing resources to complete related information recommendation tasks.

[0110] The present embodiments can also be provided as a customized, easy-to-interact network (Web) interface or other user interface (UI) mounted on a distributed parallel computing platform consisting of multiple servers, forming an information recommendation platform (for recommendation lists) for individuals, groups, or organizations.

[0111] Example 3: When the training device for a text classification model is a server-side application program interface (API) and a plug-in The text classification model training device 555 in the present embodiment can be provided as a server-side API or plug-in and can be incorporated into various applications, and a user can call the server-side API or plug-in to execute the text classification model training method in the present embodiment.

[0112] Example 4: Training a text classification model: When the device is a client API and plug-in for a mobile device The text classification model training device 555 in the present embodiment can be provided as an API and plug-in on the mobile device side, and a user can call the API and plug-in on the mobile device side to execute the text classification model training method according to the present embodiment.

[0113] Example 5: When the training device for a text classification model is a cloud open service The text classification model training device 555 in the embodiment of the present application can be provided as an information recommendation cloud service developed for users, which can be used by individuals, groups, or organizations to obtain recommendation lists.

[0114] Here, the text classification model training device 555 includes a series of modules, including a translation module 5551, a first training module 5552, a selection module 5553, and a second training module 5554. Next, a method for training the text classification model through the cooperation of each module in the text classification model training device 555 provided by the embodiment of the present application will be described.

[0115] The translation module 5551 is configured to perform a machine translation process on a plurality of first text samples in a first language through a machine translation model to obtain a plurality of second text samples having one-to-one correspondence with the plurality of first text samples, the plurality of second text samples adopting a second language different from the first language; the first training module 5552 is configured to train a first text classification model for the second language based on a plurality of third text samples in the second language and their corresponding category labels; the sorting module 5553 is configured to perform a sorting process based on a confidence level on the plurality of second text samples through the trained first text classification model; and the second training module 5554 is configured to train a second text classification model for the second language based on the second text samples obtained by the sorting process, the network depth of the second text classification model being greater than the network depth of the first text classification model.

[0116] In some embodiments, the first training module 5552 is further configured to: train the first text classification model a t-th time based on the third text samples in the second language and their corresponding category labels; perform a t-th classification process on the second text samples through the first text classification model after the t-th training based on a confidence level; train the first text classification model a t+1-th time based on the previous t classification results and the third text samples and their corresponding category labels; and set the first text classification model after the T-th training as the first text classification model after the training, where t is a positive integer that increases sequentially, and a value range satisfies 1≦t≦T−1, and T is an integer greater than 2, and is used to represent a total number of iterative trainings.

[0117] In some embodiments, the second training module 5554 is further configured to determine a distribution of a plurality of categories of the second text sample obtained by the sorting process; if the distribution of the plurality of categories of the second text sample obtained by the sorting process satisfies a distribution balance condition and the number of each category exceeds a corresponding category number threshold, construct a training set based on text samples randomly extracted from text samples of each category in the second text sample obtained by the sorting process corresponding to the category number threshold; and train a second text classification model for the second language based on the training set.

[0118] In some embodiments, the second training module 5554 is further configured to: perform a synonym-based expansion process on the second text samples of the less-distributed categories when the distribution of the second text samples obtained by the screening process in the multiple categories does not satisfy a distribution balance condition, so that the distribution of the second text samples obtained by the expansion process in the multiple categories satisfies the distribution balance condition; construct a training set based on the second text samples obtained by the expansion process; and train a second text classification model for the second language based on the training set.

[0119] In some embodiments, the second training module 5554 is further configured to construct a training set based on the third plurality of text samples and a second text sample obtained by the sorting process, and train a second text classification model for the second language based on the training set.

[0120] In some embodiments, the second training module 5554 is further configured to traverse each category of second text samples obtained by the sorting process and perform the following processes: if the number of second text samples in the category is lower than a category number threshold for the category, update the second text samples obtained by the sorting process by supplementing the second text samples of the category with third text samples of the category randomly extracted from the plurality of third text samples, and construct a training set based on the updated second text samples obtained by the sorting process.

[0121] In some embodiments, the second training module 5554 is further configured to determine a number of target samples matching the computing power available for training the second text classification model based on a correspondence between the computing power of the text classification model and the number of text samples that can be computed per unit time, and to select text samples corresponding to the number of target samples from a training set constructed based on the second text samples obtained by the selection process to use them as samples for training a second text classification model for the second language.

[0122] In some examples, the first training module 5552 is further configured to perform a prediction process on a plurality of third text samples in the second language via the first text classification model, obtain a confidence of a predicted category corresponding to each of the plurality of third text samples, construct a loss function of the first text classification model based on the confidence of the predicted category and the category label of the third text sample, update parameters of the first text classification model until the loss function converges, and set the updated parameters of the first text classification model when the loss function converges as parameters of the trained first text classification model.

[0123] In some embodiments, the first training module 5552 is further configured to perform the following processes on any third text sample of the plurality of third text samples: perform an encoding process on the third text sample via the first text classification model to obtain an encoded vector for the third text sample, perform a fusion process on the encoded vector for the third text sample to obtain a fusion vector, perform a nonlinear mapping process on the fusion vector, and obtain a confidence level of a predicted category corresponding to the third text sample.

[0124] In some embodiments, the first text classification model includes a plurality of cascaded activation layers, and the first training module 5552 is further configured to perform a mapping process of the first activation layer on the fusion vector through a first activation layer of the plurality of cascaded activation layers, output the mapping result of the first activation layer to a subsequent cascaded activation layer, continue the mapping process and output the mapping result through the subsequent cascaded activation layers, until the mapping result is output to a last activation layer, and the activation result output by the last activation layer is the confidence of the category corresponding to the third text sample.

[0125] In some embodiments, the sorting module 5553 is further configured to perform the following processes for any second text sample of the plurality of second text samples: perform a prediction process on the second text sample via the trained first text classification model, obtain confidence levels of multiple predicted categories corresponding to the second text sample, determine the category label of the first text sample corresponding to the second text sample as the category label of the second text sample, and based on the confidence levels of the multiple predicted categories corresponding to the second text sample and the category label of the second text sample, determine the second text sample that exceeds a confidence threshold as the second text sample obtained by the sorting process.

[0126] In some embodiments, the second training module 5554 is further configured to perform a prediction process on the second text sample obtained by the screening process via the second text classification model, obtain a predicted category corresponding to the second text sample obtained by the screening process, construct a loss function of the second text classification model based on the predicted category corresponding to the second text sample obtained by the screening process and a corresponding category label, update parameters of the second text classification model until the loss function converges, and set the updated parameters of the second text classification model when the loss function converges as parameters of the trained second text classification model.

[0127] In some embodiments, the second text classification model includes a plurality of cascaded encoders, and the second training module 5554 is further configured to perform the following process for any text sample of the second text samples obtained by the sorting process: performing an encoding process of the text sample through a first encoder of the cascaded plurality of encoders, outputting the encoding result of the first encoder to a subsequent cascaded encoder, continuing the encoding process and outputting the encoding result through the subsequent cascaded encoders until the encoding result is output to a last encoder, using the encoding result output by the last encoder as an encoded vector corresponding to the text sample, and performing a nonlinear mapping on the encoded vector of the text sample to obtain a predicted category corresponding to the text sample.

[0128] In some embodiments, the second training module 5554 is further configured to perform the following process through the y-th encoder of the cascaded multiple encoders: perform self-attention processing on the encoding result of the y-1th encoder, obtain a y-th self-attention vector, perform a residual connection processing on the y-th self-attention vector and the encoding result of the y-1th encoder, obtain a y-th residual vector, perform a nonlinear mapping process on the y-th residual vector, obtain a y-th mapping vector, perform a residual connection processing on the y-th mapping vector and the y-th residual vector, take the result of the residual connection as the encoding result of the y-th encoder, and output the encoding result of the y-th encoder to the y+1th encoder, where y is a positive integer that increases sequentially, and the value range satisfies 2≦y≦H-1, and H is an integer greater than 2, and is used to represent the number of the cascaded multiple encoders.

[0129] Here, the present embodiment further provides a text classification apparatus, which comprises a series of modules including an acquisition module and a processing module, where the acquisition module is configured to acquire a text to be classified, the text to be classified adopting a second language different from a first language, the processing module is configured to perform an encoding process on the text to be classified through a second text classification model having a network depth greater than that of the first text classification model, obtain an encoded vector of the text to be classified, and perform nonlinear mapping on the encoded vector of the text to be classified to obtain a category corresponding to the text to be classified, the second text classification model is obtained by training a text sample in the second language obtained by filtering using the first text classification model, and the text sample in the second language is obtained by machine translating the text sample in the first language.

[0130] The present embodiment provides a computer program product or a computer program, the computer program product or the computer program including computer instructions, the computer instructions being stored in a computer-readable storage medium, a processor of an electronic device reading the computer instructions from the computer-readable storage medium, and the processor executing the computer instructions to cause the electronic device to perform the above-mentioned method for training a text classification model or the method for text classification in the present embodiment.

[0131] The present embodiment further provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to perform the artificial intelligence based information recommendation method or text classification method provided by the present embodiment, such as the method for training a text classification model shown in FIGS. 3 to 5.

[0132] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic memory, optical disk, or CD-ROM, or may be a variety of devices including one or any combination of the above memories.

[0133] In some embodiments, the executable instructions may take the form of a program, software, software module, script, or code, and may be written in any type of programming language (including compiled or interpreted, declarative or procedural languages), and may be arranged in any form, such as as a separate program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0134] As an example, the executable instructions do not necessarily correspond to a file in a file system, but may be stored as part of a file that stores other programs or data, such as one or more scripts within a HyperText Markup Language (HTML) document, a single file dedicated to the program being discussed, or multiple joint files (such as files that store one or more modules, subroutines, or code).

[0135] As an example, the executable instructions may be configured to execute on one computing device, or may be configured to execute on multiple computing devices located at a single location, or may be distributed across multiple locations and interconnected via a communications network.

[0136] The above are merely examples of the present application and do not limit the scope of protection of the present application. Any modifications, equivalent replacements and improvements made within the spirit and scope of the present application shall be included in the scope of protection of the present application.

Claims

1. 1. A method for training a text classification model, comprising: performing a machine translation process on a plurality of first text samples in a first language to obtain a plurality of second text samples in one-to-one correspondence with the plurality of first text samples, the plurality of second text samples adopting a second language different from the first language; training a first text classification model for the second language based on a third plurality of text samples in the second language and their corresponding category labels; performing a confidence-based classification process on the second plurality of text samples via the trained first text classification model, For any second text sample of the plurality of second text samples, performing a prediction process on the second text sample through the trained first text classification model, and obtaining confidence levels of a plurality of predicted categories corresponding to the second text sample; determining a category label of a first text sample corresponding to the second text sample as a category label of the second text sample; performing a process of selecting a second text sample that exceeds a confidence threshold based on confidences of a plurality of predicted categories corresponding to the second text sample and a category label of the second text sample as the second text sample obtained by the selection process; training a second text classification model for the second language based on the second text samples obtained by the screening process; A method for training a text classification model, wherein a network depth of the second text classification model is greater than a network depth of the first text classification model.

2. training a first text classification model for the second language based on a third plurality of text samples in the second language and their corresponding category labels, comprising: training the first text classification model for a tth time based on a third plurality of text samples in the second language and their corresponding category labels; performing a t-th classification process based on confidence on the second text samples through the first text classification model after the t-th training; training the first text classification model for a (t+1)th time based on the previous (t) times of filtering results and the third plurality of text samples and their corresponding category labels; and determining the first text classification model after the Tth training as the first text classification model after the training. t is a positive integer that increases sequentially, and the value range satisfies 1≦t≦T−1, and T is an integer greater than 2 and is used to represent the total number of repeated training. The method for training a text classification model according to claim 1 .

3. training a second text classification model for the second language based on the second text samples obtained by the screening process, determining a distribution of a second text sample obtained by the filtering process across a plurality of categories; If the distribution of the categories in the second text sample obtained by the sorting process satisfies a distribution equilibrium condition and the number of each category exceeds a corresponding category number threshold, constructing a training set based on text samples randomly extracted from the text samples of each category in the second text sample obtained by the sorting process, the text samples corresponding to the category number threshold; training a second text classification model for the second language based on the training set. The method for training a text classification model according to claim 1 .

4. training a second text classification model for the second language based on the second text samples obtained by the screening process, When the distribution of the second text sample in the multiple categories obtained by the selection process does not satisfy a distribution balance condition, performing an expansion process based on synonyms on the second text sample in the category with a small distribution, in which the distribution of the second text sample in the multiple categories obtained by the expansion process satisfies the distribution balance condition; constructing a training set based on second text samples obtained by the augmentation process; training a second text classification model for the second language based on the training set. The method for training a text classification model according to claim 1 .

5. training a second text classification model for the second language based on the second text samples obtained by the screening process, constructing a training set based on the third plurality of text samples and the second text sample obtained by the screening process; and training a second text classification model for the second language based on the training set. The method for training a text classification model according to claim 1 .

6. The step of constructing a training set based on the third text samples and the second text samples obtained by the screening process includes: traversing each category of the second text sample obtained by the screening process; If the number of second text samples in the category is less than a category number threshold for the category, updating the second text samples obtained by the sorting process by supplementing the second text samples of the category with third text samples of the category randomly extracted from the plurality of third text samples; and performing a process of constructing a training set based on the updated second text samples obtained by the screening process. The method for training a text classification model according to claim 5.

7. Before training a second text classification model for the second language based on the second text samples obtained by the screening process, the method for training the text classification model includes: determining a target number of samples that matches the computational capacity available for training the second text classification model based on a correspondence relationship between the computational capacity of the text classification model and the number of text samples that can be computed per unit time; and selecting text samples corresponding to the number of target samples from a training set constructed based on the second text samples obtained by the selection process, and using the text samples as samples for training a second text classification model for the second language. The method for training a text classification model according to claim 1 .

8. training a first text classification model for the second language based on a third plurality of text samples in the second language and their corresponding category labels, comprising: performing a prediction process on a plurality of third text samples in the second language through the first text classification model, and obtaining a confidence of a predicted category corresponding to each of the plurality of third text samples; constructing a loss function for the first text classification model based on the confidence of the predicted category and the category label of the third text sample; updating parameters of the first text classification model until the loss function converges, and setting the updated parameters of the first text classification model when the loss function converges as parameters of the trained first text classification model. The method for training a text classification model according to claim 1 .

9. performing a prediction process on a plurality of third text samples in the second language through the first text classification model and obtaining a confidence of a predicted category corresponding to each of the plurality of third text samples; For any third text sample of the plurality of third text samples, via the first text classification model, performing an encoding process on the third text sample to obtain an encoded vector of the third text sample; Perform a fusion process on the coding vector of the third text sample to obtain a fusion vector; performing a nonlinear mapping process on the fusion vector to obtain a confidence level of a predicted category corresponding to the third text sample; The method for training a text classification model according to claim 8.

10. the first text classification model includes a plurality of cascaded activation layers; performing a nonlinear mapping process on the fusion vector to obtain a confidence level of a predicted category corresponding to the third text sample; performing a mapping process of the first activation layer on the fusion vector through a first activation layer of the plurality of cascaded activation layers; outputting the mapping result of the first activation layer to a subsequent cascaded activation layer, and continuing the mapping process and outputting the mapping result through the subsequent cascaded activation layers until the mapping result is output to a last activation layer; and determining the activation result output by the final activation layer as the confidence of the category corresponding to the third text sample. The method for training a text classification model according to claim 9.

11. training a second text classification model for the second language based on the second text samples obtained by the screening process, performing a prediction process on the second text sample obtained by the filtering process through the second text classification model to obtain a predicted category corresponding to the second text sample obtained by the filtering process; constructing a loss function of the second text classification model based on predicted categories and corresponding category labels corresponding to the second text samples obtained by the filtering process; updating parameters of the second text classification model until the loss function converges, and setting the updated parameters of the second text classification model when the loss function converges as parameters of the trained second text classification model. The method for training a text classification model according to claim 1 .

12. the second text classification model includes a plurality of cascaded encoders; performing a prediction process on the second text sample obtained by the filtering process through the second text classification model to obtain a predicted category corresponding to the second text sample obtained by the filtering process; For any one of the second text samples obtained by the selection process, performing an encoding process of a first encoder of the cascaded plurality of encoders on the text sample; outputting the encoding result of the first encoder to a subsequent encoder in the cascade, and continuing the encoding process and outputting the encoding result through the subsequent cascaded encoders until the encoding result is output to a final encoder; The encoding result output by the last encoder is defined as an encoded vector corresponding to the text sample; performing a non-linear mapping on the encoded vectors of the text samples to obtain a predicted category corresponding to the text samples; The method for training a text classification model according to claim 11.

13. Continuing the encoding process and outputting the encoding result by the subsequent cascaded encoders includes: via a y-th encoder of the cascaded plurality of encoders, Perform self-attention processing on the encoding result of the y-1th encoder to obtain the yth self-attention vector; A residual connection process is performed on the y-th self-attention vector and the encoding result of the y-1-th encoder to obtain a y-th residual vector; Perform a nonlinear mapping process on the y-th residual vector to obtain a y-th mapping vector; performing a residual connection process on the y-th mapping vector and the y-th residual vector, setting the residual connection result as an encoding result of the y-th encoder, and outputting the encoding result of the y-th encoder to a y+1-th encoder; y is a positive integer that increases sequentially, and the value range satisfies 2≦y≦H−1, and H is an integer greater than 2, and is used to represent the number of the cascaded encoders; The method for training a text classification model according to claim 12.

14. 1. A method for text classification, comprising: obtaining a text to be classified, the text to be classified employing a second language different from the first language; 14. A method for text classification, comprising: obtaining a category corresponding to the text to be classified using a second text classification model among text classification models including a first text classification model trained by the method of any one of claims 1 to 13 and a second text classification model having a network depth greater than that of the first text classification model.

15. 1. An apparatus for training a text classification model, comprising: a translation module configured to perform a machine translation process on a plurality of first text samples in a first language to obtain a plurality of second text samples in one-to-one correspondence with the plurality of first text samples, the plurality of second text samples employing a second language different from the first language; a first training module configured to train a first text classification model for the second language based on a third plurality of text samples in the second language and their corresponding category labels; a filtering module configured to perform a confidence-based filtering process on the second plurality of text samples via the trained first text classification model, For any second text sample of the plurality of second text samples, performing a prediction process on the second text sample through the trained first text classification model, and obtaining confidence levels of a plurality of predicted categories corresponding to the second text sample; determining a category label of a first text sample corresponding to the second text sample as a category label of the second text sample; a filtering module configured to perform a process of selecting a second text sample that exceeds a confidence threshold based on confidences of a plurality of predicted categories corresponding to the second text sample and a category label of the second text sample as the second text sample obtained by the filtering process; and a second training module configured to train a second text classification model for the second language based on second text samples obtained by the screening process, wherein a network depth of the second text classification model is greater than a network depth of the first text classification model.

16. A text classification device configured to perform the text classification method of claim 14.

17. a memory for storing executable instructions; An electronic device comprising: a processor that, when executing executable instructions stored in the memory, performs the method for training a text classification model according to any one of claims 1 to 13.

18. A method for manufacturing a computer-implemented computer-implemented system comprising: a memory for storing executable instructions; 15. An electronic device comprising: a processor that, when executing executable instructions stored in the memory, performs the method of text classification of claim 14.

19. A computer-readable storage medium having executable instructions stored thereon for causing a processor to perform the method for training a text classification model according to any one of claims 1 to 13.

20. A computer-readable storage medium having executable instructions stored thereon for causing a processor to execute the text classification method of claim 14.

21. A computer program arranged to cause a computer to carry out the method for training a text classification model according to any one of claims 1 to 13.

22. A computer program configured to cause a computer to perform the text classification method of claim 14.

Images