Cross-language text classification method based on heterogeneous graph neural network based on topic correlation

Abstract

The present invention discloses a cross-language text classification method based on a heterogeneous graph neural network with topic relevance, belonging to the field of text classification technology, comprising: step 1. obtaining a training set; step 2. text preprocessing; step 3. constructing a topic-related word probability model; step 4. constructing a cross-language heterogeneous graph; step 5. encoding nodes; and step 6. performing classification prediction on a graph convolutional network. The present invention applies a heterogeneous graph convolutional network to cross-language text classification, utilizing a method combining the topic-related word probability model with a graph convolutional network to perform cross-language text classification. The present invention improves the translation probability of source documents, thereby improving the accuracy of text classification to a certain extent. The present invention only requires a small number of annotated documents to achieve a strong classification function and learn the embedding of words and document nodes. The present invention effectively solves the problem of insufficient training required for neural network machine translation.

Description

Technical Field

[0001] The present invention relates to a cross-language text classification method based on heterogeneous graph neural networks with topic relevance, and belongs to the technical field of text classification. Background Art

[0002] 1. Text classification and cross-lingual text classification

[0003] Text classification involves assigning a given text to a category within a defined classification system based on its characteristics. Text classification effectively organizes information resources, reducing the clutter and ensuring the speed and accuracy of information access. It assigns one or more categories, tags, or classes to documents or other textual entities. Text can be classified in various ways (e.g., based on subject matter, intent, type, and / or other attributes) and for various reasons (e.g., to organize, sort, or search text).

[0004] Due to the increasing popularity of the internet, the number of documents available in languages other than English continues to increase. Organizing these heterogeneous document collections significantly increases human effort. On the one hand, experts who understand different languages are required to organize these collections; on the other hand, there may be a large number of labeled documents in one language that share the same class structure as unlabeled documents in another language. Therefore, cross-lingual text classification, which aims to organize these heterogeneous document collections, is attracting increasing attention for leveraging existing labeled documents in a language to classify unlabeled documents in languages other than that language in multilingual scenarios. It aims to extend existing automatic text classification systems from one language to other languages without requiring additional human intervention.

[0005] The general process of cross-language text classification is not significantly different from that of standard single-language text classification. In fact, cross-language text classification draws on and utilizes many single-language text classification methods and strategies. Cross-language text classification methods that achieve good results can also be applied to single-language text classification. The most important difference is that cross-language text classification typically involves a translation process.

[0006] 2. Homogeneous and Heterogeneous Graphs

[0007] An isomorphic graph is one in which both the node and relationship types are of the same type. This is a simplified representation of real-world graph data, such as the hyperlinked World Wide Web. All information about this type of graph is contained in the adjacency matrix. Within a graph, an isomorphic graph is one in which both the node and edge types are of the same type.

[0008] In contrast to homogeneous graphs, heterogeneous graphs contain more than one node or relationship type. In real-world scenarios, the graph data objects we typically study are multi-type, and the interactions between objects are also diverse. Therefore, heterogeneous graphs better reflect reality. Within a graph, a heterogeneous graph is one with more than two node and edge types.

[0009] 3. Graph Convolutional Network and Topic-Related Word Probability Model

[0010] Graph convolutional networks (GCNs) are a type of neural network architecture that has become increasingly popular in recent years. Unlike traditional network models like LSTMs and CNNs, which can only be used on grid-structured data, GCNs are capable of processing data with generalized topological graph structures and deeply exploring their characteristics and patterns. GCNs generalize convolution operations from traditional data (images or grids) to graph data. They play a core role in building many other complex graph neural network models, including autoencoder-based models, generative models, and spatiotemporal networks.

[0011] The topic-related word probability model enables each document to be represented by a topic distribution, which helps to resolve translation ambiguity of a single word, can greatly improve the translation probability of the source document, and to a certain extent improve the accuracy of text classification.

[0012] 4. Transformer-based methods and BERT models

[0013] The Transformer is a pre-trained language model that effectively learns global semantic representations and significantly improves natural language processing (NLP) tasks, including text classification. It typically uses unsupervised methods to automatically mine semantic knowledge, then constructs a pre-training objective that enables machines to learn to understand semantics. The Transformer can perform parallel computations without considering sequential information, making it suitable for large datasets.

[0014] BERT (Bidirectional Encoder Representation from Transformers) is a pre-trained language representation model. It emphasizes that instead of using traditional unidirectional language models or shallow concatenation of two unidirectional language models for pre-training, it uses a new masked language model (MLM) to generate deep bidirectional language representations. The goal of the BERT model is to use large-scale unlabeled corpus training to obtain a representation of text that contains rich semantic information, namely the semantic representation of the text. The semantic representation of the text is then fine-tuned for a specific NLP task and ultimately applied to the NLP task. This model has the following main advantages:

[0015] (1) Use MLM to predict the bidirectional Transformer to generate deep bidirectional language representation.

[0016] (2) After pre-training, simply adding an additional output layer for fine-tuning can achieve state-of-the-art performance on a variety of downstream tasks. No task-specific structural modifications to BERT are required in this process.

[0017] 5. Generative Adversarial Neural Networks for Machine Translation

[0018] Current deep neural network-based machine translation primarily employs an encoder-decoder architecture, consisting of an encoder and a decoder, both of which utilize recurrent neural networks and long short-term memory (LSTM) networks. The translation process involves the following: First, the encoder converts the input source language sentence into a sequence of word vectors, which serves as the input to the recurrent neural network. The encoder then outputs a dense vector of fixed length, called a context vector. The decoder then uses the context vector as input and utilizes another recurrent neural network combined with a softmax classifier to output a sequence of word vectors in the target language. Finally, a dictionary is used to map each word vector to a word in the target language, completing the translation process. However, this technology also has certain drawbacks. Training relies heavily on large-scale, manually annotated bilingual parallel corpora, which makes manual annotation expensive. The lack of high-quality, large-scale bilingual parallel corpora leads to insufficient training data and poor performance for neural network-based machine translation models.

[0019] Generative adversarial neural network machine translation builds on the existing generative network by introducing a discriminant network that competes with the generative network. This network determines whether the target language translation comes from the training corpus or the generative network's machine translation results, completing the translation in a competitive environment. Generative adversarial neural network machine translation improves on neural network machine translation methods, enabling the neural network machine translation model to self-learn, effectively addressing the issue of insufficient training corpus for neural network machine translation. Summary of the Invention

[0020] In order to improve the accuracy of text classification and more effectively extract heterogeneous information from integrated multilingual corpora, the present invention has the following main objectives:

[0021] 1. To better resolve translation ambiguity for individual words, a topic-related word probability model was used. This model allows each document to resolve word ambiguity through topic distribution, derive translation probabilities, and weight each translation probability. This improves the translation probability of source document words based on their topic, and ultimately, improves text classification accuracy through translation probabilities.

[0022] 2. In order to integrate semantic and syntactic information within and across languages, a cross-lingual heterogeneous graph is introduced, which can effectively extract and integrate heterogeneous information in multilingual corpora. These heterogeneous relations can enhance existing models and facilitate cross-lingual tasks.

[0023] 3. A machine translation method based on generative adversarial networks was introduced. On the basis of the original machine translation generative network, a discriminative network that competes with the generative network was introduced, which enables the neural network machine translation model to have a self-learning ability, and effectively solves the problem of insufficient training requirements for neural network machine translation.

[0024] The present invention is based on a topic-related heterogeneous graph neural network cross-language text classification method, and the specific process is as follows:

[0025] 1. Get the training set

[0026] Data collection is the foundation of text mining, and data is primarily acquired through two methods: web crawling and page processing. First, web crawlers acquire raw web page data. Then, page processing removes excess page noise and converts web pages into clean, unified text and metadata formats.

[0027] 2. Text Preprocessing

[0028] After obtaining the training set, the initial text cannot be used directly because of its uncertain and inconsistent format and the presence of many special symbols. Therefore, the text needs to be processed in some way. Based on the actual needs of text processing, some pre-processing is performed, mainly including:

[0029] (1) Remove some special symbols and special marks to make the format unified.

[0030] (2) Remove stop words that are frequently used and have no practical meaning in the text, so as to reduce the interference of meaningless words on the text.

[0031] (3) Targeted preprocessing is also required for different languages. For example, English text needs to be stemmed, which means that some deformed words need to be returned to a unified state or their original state. After this restoration operation, the number of features extracted will be greatly reduced. For example, in Chinese sentences, all words are connected together, so for Chinese text, the words need to be separated through the process of word segmentation, and these basic units representing semantics need to be separated.

[0032] (4) Some other preprocessing operations.

[0033] Through preprocessing, a good foundation is laid for subsequent processing.

[0034] 3. Build a probability model for topic-related words

[0035]

[0036] Use E[count E (e)|F] records the expected number of word occurrences as features, and E is a translation of the source speech document (hereinafter referred to as the source document) F.

[0037] Under the proposed probabilistic model, it is believed that

[0038]

[0039] First, the model determines the probability distribution of potential topics given the input source document, i.e., the conditional probability p(z|F). Second, the model uses the conditional probability p(z|F) to determine the translation probability of each word in the source document, i.e., p(e j |f j , z).

[0040] where f j is the jth word in F, k is the number of words in the source document F. The random variable e j represents the translation of the jth word in F.

[0041] This model allows us to translate a word f j The entire source document F context is considered when generating the result. The generation process is as follows:

[0042] (1) For each source document, generate a c The class label c only considers binary classification tasks with class labels of “positive” or “negative”.

[0043] (2) For each source document, generate a probability π z|c Theme z.

[0044] (3) Given a topic z, independent of the class label and with probability Generate each word e in the target language document (hereinafter referred to as the target document).

[0045] (4) For each word e in the target language document, independent of the class label and with probability θ f|e Generate a word f in the source document.

[0046] Under this model, for a target document (e1, ..., e k ) and its corresponding source documents (f1, ..., f k ), joint probability p(z, c, e1, ..., e k ,f1,...,f k )for:

[0047]

[0048] Since the class labels c and the topic z are independent of f1, ..., f k Given documents e1,...,e in the target language k , so the parameter v c , π z|c 、 It can be learned using only training documents in the target language, given a set of classes D = {(E1, c1), ..., (E n , c n )} as training documents, use maximum likelihood to estimate the parameter v c :

[0049]

[0050] Among them 1 x (y) is an indicator function that is 1 if x=y and 0 otherwise.

[0051] For π z|c 、 Use the EM algorithm to derive the analytical solution,

[0052] E-step: Set the probability distribution to

[0053]

[0054] In M-step:

[0055]

[0056]

[0057] Select the translation probability with the highest probability under the current model Thus, the probability of observing the source document (without label) is maximized, given the source document D':=F1,...,F m , again using the EM algorithm,

[0058] The E-step corresponds to the setting for each source document i, the probability for:

[0059]

[0060] In the M-step, update θ f|e for

[0061]

[0062] 4. Build a cross-language heterogeneous graph

[0063] Based on the generative adversarial neural network machine translation and the topic-related word probability model constructed in step 3 above, representative words in the source document are translated, source documents in different languages are represented in the same form, and source documents from different languages and the selected feature words are represented as graph nodes to construct a heterogeneous graph. In order to extract more language information, the generative adversarial neural network machine translation method is used to translate documents from two directions, and two types of edges are defined in the graph.

[0064] (1) Document-word edge. Documents and words are connected by co-occurrence. Since different parts of speech play different roles in classification tasks, part-of-speech tags are added to word edges for different classification tasks, and different types of edges are created based on the part-of-speech of the words in the document.

[0065] (2) Document-document edges. In order to add more direct connections between documents, two types of document edges are defined: similarity edges and translation edges, in preparation for subsequent text classification.

[0066] Similarity edges. This approach, inspired by structural correspondence learning (SCL), uses word translation as the pivot feature. Using a pair of semantically similar words from two languages, this pivot is used to identify potential relationships between documents. The KNN model is then used to find multiple similar documents in the training set that are closest to each document. This is then used to improve text classification efficiency, linking each document to similar documents.

[0067] Translation edges. Connecting nodes created by GANMT with their original documents.

[0068] 5. Encode the node

[0069] After constructing the cross-language heterogeneous graph, the BERT model is used to encode all nodes, compress the sentences into sequences, and obtain the hidden state of the special classification embedding ([CLS]) of the first token in the sequence. This is used as the aggregated sequence representation of the classification task for subsequent classification tasks. At the same time, multi-label mapping is used to integrate the semantic information of the full text into each word / character in the output text, and the text is input into different subgraphs (such as noun subgraph, adjective subgraph, etc.) according to different parts of speech, similar documents, and translated documents.

[0070] 6. Classification prediction on graph convolutional networks

[0071] Perform graph convolution on each layer of subgraphs separated by different types of edges, represent all words and texts through the hidden layer, aggregate all text information through the hidden layer, and calculate the high-order representation and aggregate information of each node on the heterogeneous graph convolution network:

[0072]

[0073] is a submatrix of the symmetric normalized adjacency matrix containing only τ-shaped edges, is the feature matrix of each node’s τ-type neighboring nodes, is a trainable parameter and σ(·) represents a nonlinear activation function.

[0074] Finally, two graph convolutional layers are used to aggregate information within the second-order neighborhood, and then a linear transformation is applied to the document nodes to obtain predictions. The document classification is predicted using similarity and translation edges connected to the labeled documents.

[0075] Beneficial effects of the present invention:

[0076] This paper applies heterogeneous graph convolutional networks to cross-language text classification, combining a topic-related word probability model with a graph convolutional network. The key points of this paper are as follows:

[0077] 1. Build a cross-language heterogeneous graph using a topic-related word probability model. After text preprocessing, machine translation based on a generative adversarial neural network and a topic-related word probability model is used to translate representative words in the document. This eliminates ambiguity and selects appropriate translation words based on the document's related topics, improving the translation probability of the source document and, to a certain extent, the accuracy of text classification.

[0078] 2. Heterogeneous Graph Convolutional Network. It only requires a small number of annotated documents to achieve strong classification capabilities and learn embeddings for words and document nodes. After constructing a cross-lingual heterogeneous graph, the BERT model is used to encode all text. The encoded nodes are then passed to the heterogeneous graph convolutional network to capture high-order neighborhood information, transforming the text classification problem into a node classification problem. The high-order representation and aggregated information of each node are calculated. Each layer performs graph convolution on different subgraphs separated by different edge types, aggregates the information, and then applies a linear transformation to the document nodes to obtain node predictions.

[0079] 3. Generative Adversarial Neural Network Machine Translation. Based on a neural network machine translation model with an encoder-decoder structure, a discriminant network is introduced to compete with the original generative network. This network determines whether the target language translation originated from the training corpus or the generative network's machine translation results. This competitive mechanism improves the generative network's ability to generate the target language and discriminate the source of the translation, effectively addressing the problem of insufficient training requirements for neural network machine translation. When a balanced state is reached, the generated translation effectively deceives the discriminant network, and the translation is complete. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] Figure 1 Shown is a flowchart of the method of the present invention

[0081] Figure 2 Shown is a flow chart of the translation method of the present invention DETAILED DESCRIPTION

[0082] The present invention will be further described in detail below with reference to the accompanying drawings and examples. It will be understood that the specific embodiments described herein are intended only to illustrate the present invention and are not intended to limit the present invention. It should also be noted that, for ease of description, the accompanying drawings only illustrate portions relevant to the present invention, not all structures.

[0083] The present invention is a cross-language text classification method based on heterogeneous graph neural networks related to topics.

[0084] When translating, a multilayer perceptron feedforward neural network model is used to construct a discriminant network to achieve binary classification. The multilayer perceptron neural network includes an input layer X: {x1, x2, ..., x n}、One hidden layer H:{h1,h2,…,h m} and an output layer Y: {y1, y2}.

[0085] The hidden layer function h(x) can be formally expressed as:

[0086] h(x)=T(W1x+b1)

[0087] Among them, the model parameters W1 and b1 represent the weight matrix and implicit bias vector from the input layer to the hidden layer respectively; T(x) is the activation function of the hidden layer, which takes the form of a hyperbolic tangent function:

[0088]

[0089] The entire multilayer perceptron neural network model function f(x) can be formally expressed as:

[0090] f(x)=S(W2·h(x)+b2)=S(W2·T(W1x+b1)+b2)

[0091] Among them, the model parameters W2 and b2 represent the weight matrix from the hidden layer to the output layer and the output layer bias vector respectively. S(x) is the activation function of the hidden layer, which takes the form of a sigmoid function:

[0092]

[0093] When the multilayer perceptron neural network model performs binary classification, the input layer vector X is substituted into f(x) to calculate the two-dimensional output vector Y. The category represented by the dimension with the larger value in Y is selected as the classification result.

[0094] When performing cross-language translation, Figure 2 As shown in the figure, based on traditional neural network-based machine translation, an adversarial artificial neural network, called a discriminant network, is introduced. The original machine translation LSTM neural network is called a generative network. In the generative network machine translation model, the generative network uses a traditional neural network translation model based on the "encoder-decoder" principle. Its function is to generate the corresponding target language sentence based on the input source language sentence. The discriminant network adopts a multilayer perceptron feedforward neural network model to implement binary classification. Each node in the neural adversarial network is a perceptron. The discriminant network determines whether the target language translation is derived from the training corpus or the result of machine translation based on the recurrent neural network. The generative adversarial network introduces a competitive mechanism between the generative network and the discriminant network. Through adversarial training, it simultaneously improves the generative network's ability to generate the target language and the discriminant network's ability to determine the source of the translation. During the training process, the discriminant network's training goal is to determine whether the translation result is real data from the corpus or the result of machine translation. The training goal of the generative network is to generate translation results that can deceive the discriminant network, making it believe that the machine translation result is derived from the real corpus. When equilibrium is reached, the generated network model can be used as the output machine translation model.

[0095]

[0096] Use E[count E (e)|F] records the expected word occurrence count as a feature, and E is a translation of the source document F.

[0097] Under the proposed probabilistic model, it is believed that

[0098]

[0099] First, the model determines the probability distribution of potential topics given the input source document, i.e., the conditional probability p(z|F). Second, the model uses the conditional probability p(z|F) to determine the translation probability of each word in the source document, i.e., p(e j |f j , z).

[0100] Under this model, for a target document (e1, ..., e k ) and its corresponding source documents (f1, ..., f k ), joint probability p(z, c, e1, ..., e k ,f1,...,f k )for:

[0101]

[0102] to determine the translation of a word.

[0103] Generative adversarial neural network-based machine translation utilizes a probabilistic model of topic-related words to translate representative words from a document. Documents from different languages and selected characteristic words are represented as graph nodes to construct a heterogeneous graph. Because different parts of speech play different roles in classification tasks, when creating document-word edges, part-of-speech tags are added to each word for use in different classification tasks, and different edge types are created based on the word's part of speech in the document. Structural correspondence learning and the KNN model are used to connect similar documents. Furthermore, nodes created by generative adversarial neural network machine translation are connected to their original documents.

[0104] After constructing the cross-lingual heterogeneous graph, we use the BERT model to encode all nodes, directly input the text into the multi-label mapping, and obtain the hidden state of the first token in the sequence. We then apply a heterogeneous graph convolutional network to the graph to compute high-level representations and aggregate information for each node. Heterogeneous graph convolutional networks apply traditional graph convolutional networks to different subgraphs separated by different types of edges and aggregate information into an implicit common space.

[0105]

[0106] is a submatrix of the symmetric normalized adjacency matrix containing only τ-shaped edges, is the feature matrix of each node’s τ-type neighboring nodes, is a trainable parameter and σ(·) represents a nonlinear activation function.

[0107] Finally, two graph convolutional layers are used to aggregate information within the second-order neighborhood, and then a linear transformation is applied to the document node to obtain the prediction.

[0108] It should be noted that the embodiments mentioned above illustrate rather than limit the invention, and that those skilled in the art will be able to design many alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The use of the verb "comprise" and its conjugations does not exclude the presence of elements or steps other than those stated in the claims. The article "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In a device claim enumerating several means, several of these means may be embodied by one and the same item of hardware. The mere fact that certain measures are stated in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

[0109] If desired, the different functions discussed herein may be performed in different orders and / or concurrently with each other. In addition, if desired, one or more functions described above may be optional or may be combined.

[0110] If desired, each step discussed above is not limited to the execution order in each embodiment, and different steps can be performed in different orders and / or performed simultaneously with each other. In addition, in other embodiments, one or more steps described above can be optional or can be combined.

[0111] Although individual aspects of the invention are set out in independent claims, further aspects of the invention comprise features from the described embodiments and / or combinations of dependent claims with features of the independent claims, rather than just the combinations explicitly set out in the claims.

[0112] It should be noted here that, although the above describes example embodiments of the present invention, these descriptions should not be understood in a limiting sense. On the contrary, several changes and modifications are possible without departing from the scope of the present invention as defined in the appended claims.

[0113] Note that the above are only preferred embodiments of the present invention and the technical principles employed. Those skilled in the art will appreciate that the present invention is not limited to the specific embodiments herein, and that various obvious changes, readjustments, and substitutions are possible for those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments and may include many other equivalent embodiments without departing from the scope of the present invention. The scope of the present invention is determined by the scope of the appended claims.

Claims

1. A cross-language text classification method based on heterogeneous graph neural networks based on topic relevance, characterized by: The specific process of this method is as follows: Step 1. Get the training set Obtain the original web page data and convert the web page into a pure and unified text format and metadata format; Step 2. Text preprocessing Step 3. Build a probability model for topic-related words Use E[count E (e)|F] records the expected number of word occurrences as features, where E is a translation of the source speech document F; Step 4. Build a cross-language heterogeneous graph Generative adversarial neural network-based machine translation uses the topic-related word probability model built in step 3 above to translate representative words in the source document. Source documents in different languages are represented in the same form. Source documents from different languages and the selected characteristic words are represented as graph nodes to construct a heterogeneous graph. The document is translated from two directions using the generative adversarial neural network machine translation method, and two types of edges are defined in the graph. Step 5. Encode the node After constructing the cross-language heterogeneous graph, the BERT model is used to encode all nodes, compress the sentences into sequences, and obtain the hidden state of the special classification embedding of the first token in the sequence. This is used as the aggregated sequence representation for the classification task and used for subsequent classification tasks. At the same time, multi-label mapping is used to integrate the semantic information of the full text into each word / character in the output text, and the text is input into different subgraphs according to different parts of speech, similar documents, and translated documents. Step 6. Make classification predictions on the graph convolutional network Perform graph convolution on each layer of subgraphs separated by different types of edges, represent all words and texts through the hidden layer, aggregate all text information through the hidden layer, and calculate the high-order representation and aggregate information of each node on the heterogeneous graph convolution network: is a submatrix of the symmetric normalized adjacency matrix containing only τ-shaped edges, is the feature matrix of each node’s τ-type neighboring nodes, is a trainable parameter, σ(·) represents a nonlinear activation function; Finally, two graph convolutional layers are used to aggregate information within the second-order neighborhood, and then linear transformations are applied to the document nodes to obtain predictions; document classification is predicted through similarity edges and translation edges connected to the labeled documents.

2. The method for cross-language text classification based on heterogeneous graph neural networks based on topic relevance according to claim 1, characterized in that: Under the described probability model, we have: First, the model determines the probability distribution of potential topics based on the given input source document, that is, the conditional probability p(z|F); Second, the model uses the conditional probability p(z|F) to determine the translation probability of each word in the source document, i.e., p(e j |f j , z); where f j is the jth word in F, k is the number of words in the source document F; the random variable e j represents the translation of the jth word in F.

3. The method for cross-language text classification based on heterogeneous graph neural networks based on topic relevance according to claim 2, characterized in that: This probabilistic model allows translating a word f j Considering the context of the entire source document F, the specific generation process is as follows: (1) For each source document, generate a c The class label c only considers binary classification tasks with class labels of "positive" or "negative"; (2) For each source document, generate a probability π z|c Theme z; (3) Given a topic z, independent of the class label and with probability Generate each word e in the target language document; (4) For each word e in the target language document, independent of the class label and with probability θ f|e Generate a word f in the source document.

4. The method for cross-language text classification based on heterogeneous graph neural networks based on topic relevance according to claim 2, characterized in that: Under this probability model, for a target document (e1, ..., e k ) and its corresponding source documents (f1, ..., f k ), joint probability p(z, c, e1, ..., e k ,f1,...,f k )for: Since the class labels c and the topic z are independent of f1, ..., f k Given documents e1,...,e in the target language k , so the parameter v c , π z|c 、 It can be learned using only training documents in the target language, given a set of classes D = {(E1, c1), ..., (E n , c n )} as training documents, use maximum likelihood to estimate the parameter v c : Among them 1 x (y) is an indicator function that is 1 if x=y and 0 otherwise.

5. The method for cross-language text classification based on heterogeneous graph neural networks based on topic relevance according to claim 4 is characterized in that: The π z|c 、 Use the EM algorithm to derive the analytical solution, E-step: Set the probability distribution to In M-step: Select the translation probability with the highest probability under the current model So as to maximize the probability of observing the source document, given the source document D': = F1, ..., F m , again using the EM algorithm, The E-step corresponds to the setting for each source document i, the probability for: In M-step, update for 6. The method for cross-language text classification based on heterogeneous graph neural networks based on topic relevance according to claim 1, characterized in that: The two types of edges described in step 4 are defined as follows: (1) Document-word edge: Documents and words are connected by co-occurrence. Since different parts of speech play different roles in classification tasks, part-of-speech tags are added to word edges for different classification tasks, and different types of edges are created based on the part-of-speech of words in the document. (2) Document-document edges: In order to add more direct connections between documents, two types of document edges are defined: similarity edges and translation edges, in preparation for subsequent text classification.

7. The method for cross-language text classification based on heterogeneous graph neural networks based on topic relevance according to claim 6, characterized in that: The similar edge uses word translation as the axis feature, and uses a pair of words from two languages with similar semantics as the pivot. The axis feature is used to find the potential relationship between documents, and then the idea of the KNN model is used to find multiple similar documents in the training set that are closest to each document. The relationship is determined again to improve the efficiency of text classification and link each document with similar documents.

8. The method for cross-language text classification based on heterogeneous graph neural networks based on topic relevance according to claim 6, characterized in that: The translation edges connect the nodes created by generative adversarial neural network machine translation with their original documents.

Images