Multilingual text classification method, apparatus, device, and medium

Abstract

The application discloses a multilingual text classification method, device and equipment and a medium. The method comprises the following steps: obtaining a target text and a pre-trained learning model, wherein the learning model comprises a shared feature extraction network and a plurality of sub-task identification networks; obtaining a sentence vector representation of the target text and a language identification prediction result through the shared feature extraction network; and calling a sub-task identification network corresponding to the language to process the sentence vector representation according to the language identification prediction result, so as to obtain a classification result of the target text, wherein a language self-learning module in the sub-task identification network learns the correlation between a plurality of languages corresponding to the language. The application can integrate the correlation knowledge between a plurality of languages into the model for learning, and classify the multilingual text through the model, so that the multilingual text can be better classified. Accordingly, the application also provides a multilingual text classification device, equipment and medium.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of text classification, and in particular to a multilingual text classification method, device, equipment and medium. BACKGROUND

[0002] Text classification is a basic research topic in natural language processing, which aims to provide a text for a classifier and return a label according to the text. Since the text classification task can help businesses automatically build and enhance data-driven decisions, it has been widely used in many fields, such as sentiment analysis and attack recognition. With the development of the Internet around the world, people increasingly use social media to share their views by writing comments in different languages.

[0003] However, the types of languages used by people to express their views and comments on social media are not limited to common languages such as Chinese and English. Some other countries' less common languages also appear on the Internet, such as Dravidian, the fourth largest language family in the world, which has 26 languages and has been widely used on social media. Current text classification technology still only implements text classification for a single language, cannot consider the language characteristics of multiple languages, and cannot achieve classification and recognition of multiple language texts. SUMMARY

[0004] The embodiments of the present application provide a multilingual text classification method, device, equipment and medium, which can integrate the correlation knowledge between multiple languages into the model for learning, and classify the multilingual text through the model, thereby better classifying the multilingual text.

[0005] The first aspect of the embodiments of the present application provides a multilingual text classification method, comprising:

[0006] obtaining a target text and a pre-trained learning model, the learning model comprising a shared feature extraction network and a plurality of sub-task recognition networks; wherein each of the sub-task recognition networks corresponds to each language; wherein the sub-task recognition network comprises a language self-learning module, a splicing module and a classifier, and the parameters of the language self-learning modules in the plurality of sub-task recognition networks are shared;

[0007] calling a representation module in the shared feature extraction network to process the target text to obtain a sentence vector representation of the target text; and calling a language classification module in the shared feature extraction network to process the target text to obtain a language recognition prediction result of the target text;

[0008] determining the language corresponding to the target text based on the language recognition prediction result;

[0009] based on the corresponding language, calling a language self-learning module in a sub-task recognition network of the corresponding language in the learning model to process the corresponding language, obtaining a language descriptor of the corresponding language, and based on the language descriptor of the corresponding language, calling a splicing module in the sub-task recognition network of the corresponding language to process the language descriptor of the corresponding language and the sentence vector representation, obtaining a multi-dimensional feature vector of the target text;

[0010] based on the multi-dimensional feature vector, calling a classifier in the sub-task recognition network of the corresponding language to process the multi-dimensional feature vector, obtaining a classification result of the target text.

[0011] The second aspect of the embodiment of the application provides a multi-language text classification device, which comprises:

[0012] The acquisition module is used to acquire a target text and a pre-trained learning model, and the learning model comprises a shared feature extraction network and a plurality of sub-task recognition networks; wherein each sub-task recognition network corresponds to each language one by one; wherein the sub-task recognition network comprises a language self-learning module, a splicing module and a classifier, and the parameters of the language self-learning modules in the plurality of sub-task recognition networks are shared.

[0013] The language recognition module is used to call a representation module in the shared feature extraction network to process the target text, obtain a sentence vector representation of the target text, and call a language classification module in the shared feature extraction network to process the target text, obtain a language recognition prediction result of the target text, and further determine the corresponding language of the target text based on the language recognition prediction result.

[0014] The characteristic vector acquisition module is used to, based on the corresponding language, call a language self-learning module in a sub-task recognition network of the corresponding language in the learning model to process the corresponding language, obtain a language descriptor of the corresponding language, and based on the language descriptor of the corresponding language, call a splicing module in the sub-task recognition network of the corresponding language to process the language descriptor of the corresponding language and the sentence vector representation, obtain a multi-dimensional feature vector of the target text.

[0015] The classification module is used to, based on the multi-dimensional feature vector, call a classifier in the sub-task recognition network of the corresponding language to process the multi-dimensional feature vector, obtain a classification result of the target text.

[0016] The third aspect of the embodiment of the application provides a terminal device, which comprises a processor, a memory and a computer program stored in the memory and configured to be executed by the processor, and the processor implements the multi-language text classification method provided by the above embodiment when executing the computer program.

[0017] A fourth aspect of the embodiment of the present application provides a storage medium, the storage medium comprising a stored computer program, wherein the computer program controls a device in which the storage medium is located to perform the multilingual text classification method provided by the above-mentioned embodiments when the computer program is running.

[0018] Compared with the prior art, the multilingual text classification method provided by the embodiment of the present application acquires a target text and a pre-trained learning model, the learning model comprising a shared feature extraction network and a plurality of sub-task recognition networks; the target text is processed by the shared feature extraction network to obtain a sentence vector representation of the target text and a language recognition prediction result, and according to the language recognition prediction result, a corresponding language sub-task recognition network is called to process the sentence vector representation module, wherein the language self-learning module in the sub-task recognition network learns the correlation between a plurality of languages for a corresponding language to obtain a language descriptor of the corresponding language, and the language descriptor of the corresponding language and the sentence vector representation are spliced by a splicing module in the sub-task recognition network to obtain a multi-dimensional feature vector of the target text, and finally the multi-dimensional feature vector is processed by a classifier in the corresponding language sub-task recognition network to obtain a classification result of the target text. The embodiment of the present application can integrate the correlation knowledge between a plurality of languages into the model for learning, and classify the multilingual text through the model, so that the multilingual text is better classified. Accordingly, the embodiment of the present application also provides a multilingual text classification device, equipment and medium. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 is a flowchart of the multilingual text classification method provided by the embodiment of the present application;

[0020] Figure 2 is a multilingual text classification framework structure diagram provided by the embodiment of the present application. DETAILED DESCRIPTION

[0021] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0022] Multilingual text classification is one of the research hotspots in the field of natural language processing. Although there are currently some researches on multilingual text classification, the existing models for classifying multilingual text mainly focus on a single language and cannot solve the common problems of multiple languages. Secondly, for some language families, there are similar vocabularies and common language commonalities among multiple languages covered by the language families, such as the Dravidian language family. There is a certain correlation between different languages. The existing models only study a single language without learning the correlation between languages, and it is difficult to reflect the deep rules and characteristics behind the corresponding language, so it cannot provide necessary help for the systematic study of the entire language family.

[0023] In order to solve the problem that the existing multilingual text classification model ignores the correlation between multiple languages, the embodiment of the present application provides a multilingual text classification method, by acquiring a target text and a pre-trained learning model, the learning model includes a shared feature extraction network and a plurality of sub-task recognition networks; the target text is processed through the shared feature extraction network to obtain a sentence vector representation of the target text and a language recognition prediction result, and according to the language recognition prediction result, the corresponding language sub-task recognition network is called to process the sentence vector representation module, wherein the language self-learning module in the sub-task recognition network learns the correlation between multiple languages for the corresponding language to obtain a language descriptor of the corresponding language, and the language descriptor of the corresponding language is spliced with the sentence vector representation through the splicing module in the sub-task recognition network to obtain a multi-dimensional feature vector of the target text, and finally the multi-dimensional feature vector is processed based on the classifier in the sub-task recognition network of the corresponding language to obtain the classification result of the target text. The embodiment of the present application can integrate the correlation knowledge between multiple languages into the model for learning, and classify the multilingual text through the model, so as to better classify the multilingual text.

[0024] The multilingual text classification method of the embodiment of the present application will be described in detail below with reference to the accompanying drawings.

[0025] Referring to Figure 1 The multilingual text classification method provided by the embodiment of the present application comprises:

[0026] S11, acquiring a target text and a pre-trained learning model, the learning model includes a shared feature extraction network and a plurality of sub-task recognition networks; wherein each sub-task recognition network corresponds to each language one by one; wherein the sub-task recognition network includes a language self-learning module, a splicing module and a classifier, and the parameters of the language self-learning module in the plurality of sub-task recognition networks are shared.

[0027] In the embodiment of the present application, multi-task learning is introduced for multi-lingual text classification problems. Multi-task learning is a machine learning method that puts multiple related tasks together to learn based on shared representations.

[0028] Specifically, in the embodiment of the present application, the learning model includes a shared feature extraction network and a plurality of sub-task identification networks, the shared feature extraction network is connected to each sub-task identification network through a branch node; the shared feature extraction network is a network shared by sub-task identification networks; the number of sub-task identification networks is the same as the number of tasks of multi-task learning. Wherein, the shared feature extraction network is used for feature extraction of text, after feature extraction of the target text through the shared feature extraction network, the execution process of multiple different tasks can be conveniently performed, such as text sentiment classification and identification tasks of multiple different languages.

[0029] S12, calling the representation module in the shared feature extraction network to process the target text to obtain the sentence vector representation of the target text; and calling the language classification module in the shared feature extraction network to process the target text to obtain the language identification prediction result of the target text; based on the language identification prediction result, determining the language corresponding to the target text.

[0030] In the embodiment of the present application, when the shared feature extraction network processes the target text, not only the sentence vector representation of the target text is obtained, but also the language information corresponding to the target text. For example, the shared feature extraction network outputs the language prediction result of the target text as follows: the probability of Kannada language is 1 / 4, the probability of Tamil language is 1 / 4, and the probability of Malayalam language is 1 / 2. Then, the language prediction result with larger probability is selected to determine the language corresponding to the target text. By identifying the corresponding language information of the target text, the correlation learning between languages in the subsequent process can be facilitated, and the corresponding language information can be used as an auxiliary information to assist the machine learning to select the sub-task identification network of the corresponding language to further process the target text.

[0031] S13, based on the corresponding language, calling the language self-learning module in the sub-task identification network of the corresponding language in the learning model to process the corresponding language to obtain the language descriptor of the corresponding language, and based on the language descriptor of the corresponding language, calling the splicing module in the sub-task identification network of the corresponding language to process the language descriptor of the corresponding language and the sentence vector representation to obtain the multi-dimensional feature vector of the target text.

[0032] In the embodiment of the present application, after the language information contained in the target text is extracted by the shared feature extraction network, the machine learning module selects the sub-task recognition network corresponding to the language to process the corresponding language information and the sentence vector representation, and obtains the classification result of the target text. For example, if the language corresponding to the target text is identified by the shared feature extraction network as the Malayalam language, the sub-task recognition network corresponding to the Malayalam language is selected to process the multi-dimensional feature vector, and the classification result of the target text is obtained.

[0033] Specifically, the sub-task recognition network includes a language self-learning module, a splicing module, and a classifier, and the parameters of the language self-learning modules in the plurality of sub-task recognition networks are shared. The language self-learning module is used to learn the mutual information between each language contained in the corresponding language, the splicing module is used to splice the sentence vector representation output by the representation module and the language descriptor of the corresponding language output by the language self-learning module, so as to obtain the correlation information between the semantic representation of the target text and the language. The classifier is used for sentiment classification of the text.

[0034] S14, based on the multi-dimensional feature vector, calling the classifier in the sub-task recognition network corresponding to the language to process the multi-dimensional feature vector, and obtaining the classification result of the target text.

[0035] Based on the multilingual text classification method provided by the embodiment of the present application, the target text and the pre-trained learning model are obtained, the learning model includes a shared feature extraction network and a plurality of sub-task recognition networks; the target text is processed by the shared feature extraction network to obtain the sentence vector representation and the language recognition prediction result of the target text, and according to the language recognition prediction result, the sub-task recognition network corresponding to the language is called to process the sentence vector representation module, wherein the language self-learning module in the sub-task recognition network learns the correlation between a plurality of languages corresponding to the language to obtain the language descriptor of the corresponding language, and the language descriptor of the corresponding language is spliced with the sentence vector representation by the splicing module in the sub-task recognition network to obtain the multi-dimensional feature vector of the target text. Finally, the multi-dimensional feature vector is processed by the classifier in the sub-task recognition network corresponding to the language to obtain the classification result of the target text. The embodiment of the present application can integrate the correlation knowledge between a plurality of languages into the model for learning, and classify the multilingual text through the model, so as to better classify the multilingual text.

[0036] In an alternative embodiment, the representation model employs a LaBSE model. LaBSE is a multilingual BERT embedding model that is pre-trained using 17 billion monolingual sentences and 6 billion bilingual sentence pairs. The trained model can generate language-agnostic cross-lingual sentence embeddings for 109 languages, and is also effective for low-resource languages for which no data was available during training. LaBSE employs a slightly different training method than BERT, in that it removes the NSP training task and modifies the MLM training to concatenated translation pairs, TLM (or Translation Language Modeling, TLM). The final training is performed using both MLM and TLM.

[0037] Specifically, for a text classification problem, the first position in the target text is marked as a [CLS] start symbol, and the final hidden state h i is typically taken as the aggregated sequence representation for the classification task. For each token in a given sentence, its input representation is constructed by summing the corresponding token, segment, and position embeddings.

[0038] In the embodiment of the present application, the LaBSE model represents the sentence encoding S by taking the first token ([CLS]) of the final hidden state, to obtain the sentence vector representation of the text.

[0039] In an alternative embodiment, the corresponding language is processed by calling the language self-learning module in the sub-task recognition network of the corresponding language in the learning model, to obtain the language descriptor of the corresponding language, which specifically comprises:

[0040] Based on the corresponding language, the language self-learning module in the sub-task recognition network of the corresponding language in the learning model is called, so that the language self-learning module initializes the language description matrix, and inputs the language description matrix and the initial language descriptor of the corresponding language into the self-attention module, to obtain the language descriptor of the corresponding language.

[0041] Specifically, in the embodiment of the present application, in order to better recognize the interaction between different languages, the embodiment introduces language descriptors for language representation self-learning based on the self-attention mechanism, to simulate the interaction between languages.

[0042] Specifically, it is assumed that a language descriptor represents the label of a language i ∈R m where m is the dimension obtained from the representation module, and the language descriptors of all languages can form a matrix N ∈ R n×mwherein n represents the number of languages. Thus, a language descriptor of a certain language i obtained by a language self-learning method can be obtained as

[0043]

[0044] wherein N i is the initial language descriptor of language i.

[0045] According to the above formula, the steps of the language self-learning module for language self-learning in the embodiment include: firstly, randomly initializing the language descriptor matrix using the normal distribution function, then performing dot product on the initial language descriptor N i and the transpose matrix of the language descriptor matrix, and normalizing it using the softmax activation function to obtain the language weight corresponding to each language under the current language i, and finally performing dot product on the weight and the language descriptor matrix to obtain the weighted sum of all languages relative to language i, that is, the new language descriptor containing the mutual information of each language

[0046] It can be seen that in the optional embodiment, the language descriptor is used for language information self-learning to simulate the interaction between languages based on the self-attention mechanism, and thus the multi-language text classification can be better performed for multi-language texts with greater correlation between languages such as the Dravidian language family.

[0047] In an optional implementation, the subtask recognition network further includes a language information perturbation module, and the method further includes:

[0048] calling the language information perturbation module in the subtask recognition network of the corresponding language in the learning model to process the corresponding language, generating a perturbation value of each vocabulary in the target text, and superimposing the perturbation value of each vocabulary on the basis of the sentence vector representation to generate an adversarial sentence vector representation; wherein the target text includes a plurality of vocabularies;

[0049] Then, the calling of the splicing module in the subtask recognition network of the corresponding language to process the language descriptor of the corresponding language and the sentence vector representation includes:

[0050] The calling of the splicing module in the subtask recognition network of the corresponding language to process the language descriptor of the corresponding language and the adversarial sentence vector representation.

[0051] In the embodiment of the application, the perturbation value is added to the original target text to generate the interference information of the target text.

[0052] In an optional embodiment, the perturbation value of each vocabulary can be a preset adversarial perturbation value of the terminal device.

[0053] In another optional embodiment, the perturbation value of each vocabulary can be determined by the following steps:

[0054] The perturbation value of an important vocabulary in the target text is assigned as a first perturbation value, and the perturbation value of a non-important vocabulary in the target text is assigned as a second perturbation value; wherein the first perturbation value is greater than the second perturbation value.

[0055] In the embodiment of the present application, in order to make the model have better language generalization ability, the important vocabulary and the non-important vocabulary in the target text are identified, and the adversarial perturbation of the word embedding of those important vocabularies is increased.

[0056] In an optional embodiment, the identification of the important vocabulary of the target text specifically includes:

[0057] Based on the target text, the important vocabulary extraction module in the shared feature extraction network is called to perform mask processing on each vocabulary in the target text, to obtain a plurality of mask texts;

[0058] The plurality of mask texts and the target text are input into a language classification network to obtain the language prediction probability of the target text and the language prediction probability of each mask text, and to determine the maximum probability value in the language prediction probability of the target text and the maximum probability value in the language prediction probability of each mask text.

[0059] When the maximum value of the language prediction probability of the mask text is greater than the maximum value of the language prediction probability of the target text, the masked word in the corresponding mask text is obtained as the important vocabulary of the target text.

[0060] In the embodiment of the present application, the language information perturbation module in the learning model first screens out the important vocabulary of the target text based on the MLM strategy, and then superimposes a higher perturbation value on the important vocabulary.

[0061] For example, assuming that the input target text is "My name is xxx", the probability of outputting "English" in the input language classification network is 0.9, and then each word in the target text is masked in turn, that is, finally four mask texts are obtained:

[0062] (1) [MASK] name is xxx.

[0063] (2) My [MASK] is xxx.

[0064] (3) My name [MASK] xxx.

[0065] (4) My name is [MASK].

[0066] The probabilities of the mask texts (1), (2), (3), and (4) outputting "English" are 0.7, 0.3, 0.5, and 0.6, respectively. The maximum probability value of the language prediction probability corresponding to the original target text is subtracted from the maximum probability value of the language prediction probability corresponding to the mask text. If the subtraction result is greater than 0, it indicates that the masked word in the mask text is an important word. When the learning model processes such important words, more perturbation values are added.

[0067] It can be seen that, in the optional implementation, the learning model can avoid the language information deviation problem in multilingual text classification by identifying important words in the target text and performing regularization perturbation operation on the important words in an adversarial manner.

[0068] In an optional implementation, the pre-trained learning model is trained by the following steps:

[0069] A text training set is obtained, and the text training set is input into the learning model. The training text set includes training texts of multiple different languages. Each training text is labeled with a corresponding language label and a classification label.

[0070] In response to the input text training set, the learning model inputs the text training set into a shared feature extraction network in the learning model for training to obtain a sentence vector representation of each training text and corresponding language information.

[0071] In response to the obtained corresponding language information, the learning model inputs the sentence vector representation of each training text into a sub-task recognition network corresponding to the language in the learning model based on the corresponding language information to train the multiple sub-task recognition networks.

[0072] In response to the multiple sub-task recognition networks meeting a training completion condition, the training of the learning model is completed.

[0073] Specifically, the training process of the shared feature extraction network includes the training processes of the representation module, the language classification module, and the important word extraction module in the shared feature extraction network. The training process of the important word extraction module includes the following steps:

[0074] S1, a text training set is obtained. Each training text is labeled with a corresponding language label and a classification label.

[0075] S2, respectively, for each token in each training text, construct n mask texts with mask marks X' = [X\x1, X\x2, X\xi,..., X\xn], where X\xi = [x1, x2,..., [MASK],..., n] means that xi in the sentence X is replaced by [MASK]. Then, after introducing the language classification network to predict the probability distribution of the input sentence X and its corresponding n mask samples X', the probability difference between them is taken as the language information contained in the mask marked sentence. The goal of the language classification network M is to detect which language label the input sentence belongs to and output the probability of the label with the highest probability. Therefore, the language information Ixi of the token xi is defined as:

[0076] Ixi= log (My(xi) - My([MASK]))

[0077] where My() denotes the output prediction probability of the correct language label y by the language classification network M.

[0078] After obtaining the language information Ixi of each token xi, select the tokens with Ixi greater than 0 as important tokens to add to the important token set I.

[0079] Specifically, the sub-task recognition network adds an adversarial training process to the model through the language information perturbation module. The adversarial training process is as follows:

[0080] S1, after obtaining the sentence vector representation and important token set I of the training text by the general feature extraction network, superimpose the perturbation value of each token on the basis of the sentence vector representation

[0081]

[0082] g = wtL(V, Y, θ)

[0083] L(V, Y, θ) = log p(Y|V; θ)

[0084]

[0085] where x i represents the i-th token in the training text, wt represents the word embedding vector dimension of the training text, V represents the word embedding of the original training text X, and V = [v1, v2,..., t,..., n], where vt∈R d , d is the dimension of the sentence vector representation, n is the number of tokens in the training text, i.e., the token number, θ is the hyperparameter of the sub-task recognition network model, Y is the classification label corresponding to the training text; L(V, Y, θ) is the classifier; g is the original gradient for backpropagation, and α is a weight matrix representing the perturbation degree of the token.

[0086] S2, in the training process, the perturbed gradient is used for back propagation and parameter update, then the noise of the embedding layer is removed, the original gradient is restored, and the next epoch of iterative training is performed. In the same batch, the original loss L(θ) and the adversarial loss Ladv() of the sub-task recognition network model are respectively:

[0087]

[0088]

[0089] Therefore, the final loss of the sub-task recognition network model is:

[0090] L=L(θ)+Ladv(θ)

[0091] In order to better understand the technical scheme of the embodiment of the application, see Figure 2 , Figure 2 The framework adopted by the multilingual text classification method of the embodiment of the application is shown. On the one hand, the framework uses the LaBSE pre-training model as a base model, proposes an important extraction method based on the MLM strategy to filter out important words in the input text, and uses adversarial training to perform regularizing disturbance operation on important words of different languages. On the other hand, in view of the problem that the model cannot well identify and utilize the correlation between languages in multilingual multi-task learning, the embodiment proposes a language information self-learning method, uses the technology of Self-attention to make the language descriptor self-learn and rich semantic information, and then uses the language descriptor to integrate the related information between languages into the model.

[0092] Correspondingly, the embodiment of the application also provides a multilingual text classification device, comprising:

[0093] An acquisition module is configured to acquire a target text and a pre-trained learning model, wherein the learning model comprises a shared feature extraction network and a plurality of sub-task recognition networks; each of the sub-task recognition networks corresponds to one language; each of the sub-task recognition networks comprises a language self-learning module, a splicing module and a classifier, and the parameters of the language self-learning modules in the plurality of sub-task recognition networks are shared.

[0094] A language recognition module is configured to call a representation module in the shared feature extraction network to process the target text to obtain a sentence vector representation of the target text, and call a language classification module in the shared feature extraction network to process the target text to obtain a language recognition prediction result of the target text, and further configured to determine a language corresponding to the target text based on the language recognition prediction result.

[0095] a characteristic vector obtaining module, configured to invoke a language self-learning module in a sub-task recognition network of the corresponding language in the learning model to process the corresponding language based on the corresponding language, to obtain a language descriptor of the corresponding language, and to invoke a splicing module in the sub-task recognition network of the corresponding language to process the language descriptor of the corresponding language and the sentence vector representation based on the language descriptor of the corresponding language, to obtain a multi-dimensional characteristic vector of the target text;

[0096] a classification module, configured to invoke a classifier in the sub-task recognition network of the corresponding language to process the multi-dimensional characteristic vector based on the multi-dimensional characteristic vector, to obtain a classification result of the target text.

[0097] It should be noted that the multilingual text classification device provided by the embodiments of the present application is used to execute all steps and processes of the multilingual text classification method provided by the above-mentioned embodiments, and the working principles and beneficial effects of the two are one-to-one correspondence, which will not be described in more detail here.

[0098] In an optional implementation, the invocation of the language self-learning module in the sub-task recognition network of the corresponding language in the learning model to process the corresponding language based on the corresponding language to obtain the language descriptor of the corresponding language specifically comprises:

[0099] invoking the language self-learning module in the sub-task recognition network of the corresponding language in the learning model based on the corresponding language, so that the language self-learning module initializes a language description matrix, and inputs the language description matrix and a preset initial language descriptor of the corresponding language into a self-attention module to obtain the language descriptor of the corresponding language.

[0100] In an optional implementation, the sub-task recognition network further comprises a language information perturbation module, and the method further comprises:

[0101] invoking the language information perturbation module in the sub-task recognition network of the corresponding language in the learning model to process the corresponding language, to generate a perturbation value of each vocabulary in the target text, and to superimpose the perturbation value of each vocabulary on the basis of the sentence vector representation to generate an adversarial sentence vector representation; wherein the target text comprises a plurality of vocabularies.

[0102] Then, the invocation of the splicing module in the sub-task recognition network of the corresponding language to process the language descriptor of the corresponding language and the sentence vector representation comprises:

[0103] invoking the splicing module in the sub-task recognition network of the corresponding language to process the language descriptor of the corresponding language and the adversarial sentence vector representation.

[0104] In an optional implementation, the disturbance value of each of the words is determined by the following steps:

[0105] The disturbance value of an important word in the target text is assigned as a first disturbance value, and the disturbance value of a non-important word in the target text is assigned as a second disturbance value; wherein the first disturbance value is greater than the second disturbance value.

[0106] In an optional implementation, the shared feature extraction network further comprises an important word extraction module, and the method further comprises:

[0107] Based on the target text, the important word extraction module in the shared feature extraction network is called to perform mask processing on each word in the target text, to obtain a plurality of masked texts;

[0108] The plurality of masked texts and the target text are input into a language classification network to obtain language prediction probabilities of the target text and language prediction probabilities of each masked text, and to determine a maximum probability value in the language prediction probabilities of the target text and a maximum probability value in the language prediction probabilities of each masked text;

[0109] When the maximum language prediction probability of the masked text is greater than the maximum language prediction probability of the target text, the masked word in the corresponding masked text is obtained as an important word of the target text.

[0110] In an optional implementation, the pre-trained learning model is trained by the following steps:

[0111] A text training set is obtained, and the text training set is input into the learning model; the training text set comprises a plurality of training texts of different languages; each of the training texts is labeled with a corresponding language label and a classification label;

[0112] In response to the input text training set, the learning model inputs the text training set into a shared feature extraction network in the learning model for training, to obtain a sentence vector representation of each of the training texts and corresponding language information;

[0113] In response to the obtained corresponding language information, the learning model inputs the sentence vector representation of each training text into a sub-task recognition network of a corresponding language in the learning model based on the corresponding language information, to train the plurality of sub-task recognition networks;

[0114] In response to the plurality of sub-task recognition networks satisfying a training completion condition, the training of the learning model is completed.

[0115] Correspondingly, the embodiment of the present application further provides a terminal device, comprising a processor, a memory and a computer program stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to implement the multilingual text classification method provided by the above-mentioned embodiment, for example Figure 1 S11-S14.

[0116] Correspondingly, the embodiment of the present application further provides a storage medium, comprising a stored computer program, wherein the computer program controls a device where the storage medium is located to execute the multilingual text classification method provided by the above-mentioned embodiment when the computer program runs, for example Figure 1 S11-S14.

[0117] The above is the preferred embodiment of the present application, it should be pointed out that, for those skilled in the art, without departing from the principles of the present application, can make a number of improvements and refinements, these improvements and refinements also as the protection scope of the present application.

Claims

1. A multi-lingual text classification method, characterized by, The method comprises: acquiring target text and a pre-trained learning model, the learning model comprising a shared feature extraction network and a plurality of sub-task recognition networks; wherein each of the sub-task recognition networks corresponds to each language; wherein the sub-task recognition network comprises a language self-learning module, a splicing module and a classifier, and the parameters of the language self-learning modules in the plurality of sub-task recognition networks are shared; calling a representation module in the shared feature extraction network to process the target text to obtain a sentence vector representation of the target text; and calling a language classification module in the shared feature extraction network to process the target text to obtain a language recognition prediction result of the target text; determining the language corresponding to the target text based on the language recognition prediction result; based on the corresponding language, calling the language self-learning module in the sub-task recognition network of the corresponding language in the learning model to process the corresponding language to obtain a language descriptor of the corresponding language, and based on the language descriptor of the corresponding language, calling the splicing module in the sub-task recognition network of the corresponding language to process the language descriptor of the corresponding language and the sentence vector representation to obtain a multi-dimensional feature vector of the target text; based on the multi-dimensional feature vector, calling the classifier in the sub-task recognition network of the corresponding language to process the multi-dimensional feature vector to obtain a classification result of the target text.

2. The multilingual text classification method of claim 1, wherein, The method further comprises: calling the language information perturbation module in the sub-task recognition network of the corresponding language in the learning model to process the corresponding language to generate a perturbation value of each word in the target text, and to superimpose the perturbation value of each word on the basis of the sentence vector representation to generate an adversarial sentence vector representation; wherein the target text comprises a plurality of words.

3. The multilingual text classification method of claim 1, wherein, The method further comprises: calling the splicing module in the sub-task recognition network of the corresponding language to process the language descriptor of the corresponding language and the adversarial sentence vector representation. The perturbation value of each word is determined by the following steps: assigning a first perturbation value to the perturbation value of an important word in the target text and a second perturbation value to the perturbation value of a non-important word in the target text; wherein the first perturbation value is greater than the second perturbation value.

4. The multilingual text classification method of claim 3, wherein, ​ ​ 5. The multilingual text classification method of claim 4, wherein, The shared feature extraction network further comprises an important word extraction module, and the method further comprises: Based on the target text, calling the important word extraction module in the shared feature extraction network to perform mask processing on each word in the target text, to obtain a plurality of masked texts; Inputting the plurality of masked texts and the target text into a language classification network to obtain language prediction probabilities of the target text and each masked text, and determining the maximum probability value in the language prediction probabilities of the target text and the maximum probability value in the language prediction probabilities of each masked text; When the maximum value of the language prediction probabilities of the masked text is greater than the maximum value of the language prediction probabilities of the target text, obtaining the masked word in the corresponding masked text as an important word of the target text.

6. The multilingual text classification method according to any one of claims 1 to 5, wherein, The pre-trained learning model is trained by the following steps: Obtain a text training set and input the text training set into the learning model; the training text set includes a plurality of training texts of different languages; each training text is labeled with a corresponding language label and a classification label; In response to the input text training set, the learning model inputs the text training set into a shared feature extraction network in the learning model for training to obtain sentence vector representation and corresponding language information of each training text; In response to the obtained corresponding language information, the learning model inputs the sentence vector representation of each training text into a sub-task recognition network corresponding to the language information in the learning model respectively to train the plurality of sub-task recognition networks; In response to the plurality of sub-task recognition networks satisfying a training completion condition, the training of the learning model is completed.

7. A multi-lingual text classification apparatus characterized by comprising: Comprise: An acquisition module is configured to acquire a target text and a pre-trained learning model, the learning model comprising a shared feature extraction network and a plurality of sub-task recognition networks; each sub-task recognition network corresponds to one language; the sub-task recognition network comprises a language self-learning module, a splicing module and a classifier, and the parameters of the language self-learning modules in the plurality of sub-task recognition networks are shared; A language recognition module is configured to call a representation module in the shared feature extraction network to process the target text to obtain sentence vector representation of the target text, and call a language classification module in the shared feature extraction network to process the target text to obtain language recognition prediction result of the target text, and determine the corresponding language of the target text based on the language recognition prediction result; A characteristic vector acquisition module is configured to call a language self-learning module in a sub-task recognition network corresponding to a language to process the language based on the corresponding language, to obtain a language descriptor of the corresponding language, and call a splicing module in the sub-task recognition network corresponding to the language to process the language descriptor of the corresponding language and the sentence vector representation, to obtain a multi-dimensional feature vector of the target text. The classification module is configured to invoke a classifier in the sub-task recognition network of the corresponding language to process the multi-dimensional feature vector based on the multi-dimensional feature vector, and obtain a classification result of the target text.

8. A terminal device, comprising: The computer program is stored in the memory and configured to be executed by the processor, and the processor executes the computer program to implement the multilingual text classification method according to any one of claims 1 to 6.

9. A storage medium, characterized by The storage medium comprises a stored computer program, wherein the computer program controls a device where the storage medium is located to execute the multilingual text classification method according to any one of claims 1 to 6 when the computer program is running.

Images