Chinese-Thai cross-language sensitive information recognition method integrating bilingual sensitive dictionary and heterogeneous graph

Abstract

The present invention relates to a Chinese-Thai cross-language sensitive information recognition method that integrates a bilingual sensitive dictionary and a heterogeneous graph to solve the problem that bilingual sensitive words are difficult to align in Chinese-Thai sensitive information recognition. The present invention first constructs a Chinese-Thai bilingual sensitive dictionary based on Wikipedia and social media sensitive data. Then, the document and the keywords and sensitive words contained therein are used as nodes, and the bilingual alignment, similarity relationship and different parts of speech are used as edges to construct a Chinese-Thai cross-language heterogeneous graph, enhance the sensitive features of the document and the alignment of the bilingual sensitive features, and use a multilingual pre-trained model to characterize the document nodes and word nodes. Finally, the input document is encoded by a multi-layer graph convolutional neural network, and the sensitive information classifier is used to classify and predict the document. Experimental results show that the proposed model performs better than the general cross-language text classification method in the Chinese-Thai cross-language sensitive information recognition task.

Description

Technical Field

[0001] The present invention relates to a Chinese-Thai cross-language sensitive information recognition method integrating a bilingual sensitive dictionary and a heterogeneous graph, and belongs to the field of natural language processing. Background Art

[0002] Cross-language sensitive information identification can be viewed as a domain-specific cross-language text classification task. However, in social media data, sensitive words have diverse representations and often appear as uncommon words or aliases, making general cross-language text classification methods less effective for cross-language sensitive information identification.

[0003] One of the core challenges in identifying sensitive information in social media text data is how to identify sensitive features within the data. Traditional cross-lingual classification methods typically rely on bilingual alignment resources, such as bilingual dictionaries (Balamurali et al., 2012; Barnes et al., 2018) or parallel corpora (Zhou et al., 2016; Xu et al., 2017). However, in low-resource languages, they often face challenges with limited labeled data and a lack of large-scale training datasets. Most commonly used cross-lingual text classification methods learn shared encoding representations across languages, including bilingual word embeddings (Ziser et al., 2018; Chen et al., 2018) and multilingual pre-trained language models (Dev et al., 2018; Conneau et al., 2019; Liu et al., 2019; Conneau et al., 2020). Most of these studies focus on common domains such as text classification, such as sentiment classification. However, in the cross-lingual sensitive information identification task, the diverse representations of sensitive words in Chinese and Thai social media data make bilingual sensitive words difficult to identify and align.

[0004] To address the problem of diverse representations of sensitive words in Chinese and Thai social media data, which makes bilingual sensitive word recognition and alignment difficult, a Chinese-Thai cross-lingual sensitive information recognition method is proposed that integrates a bilingual sensitive dictionary and a heterogeneous graph. This method utilizes the sensitive word alignment information from the bilingual sensitive dictionary to construct a cross-lingual heterogeneous graph, enhancing cross-lingual transfer learning capabilities. First, a Chinese-Thai bilingual sensitive dictionary is constructed based on Wikipedia and social media sensitive data. Then, a Chinese-Thai cross-lingual heterogeneous graph is constructed using documents, their keywords, and sensitive words as nodes, and bilingual alignment, similarity relationships, and different parts of speech as edges. This enhances document sensitivity features and bilingual sensitive feature alignment. A multilingual pre-trained model is then used to represent document nodes and word nodes. Finally, a multi-layer graph convolutional neural network is used to encode the input documents, and a sensitive information classifier is used to predict document classification. Summary of the Invention

[0005] The present invention provides a Chinese-Thai cross-language sensitive information recognition method that integrates a bilingual sensitive dictionary and a heterogeneous graph. This method is used to solve the problem that the bilingual sensitive word representations in Chinese-Thai cross-language sensitive information recognition are diverse, making bilingual sensitive words difficult to identify and align. This method improves the shortcomings of general methods and enhances the performance of Chinese-Thai cross-language sensitive information recognition.

[0006] The technical solution of the present invention is: a Chinese-Thai cross-language sensitive information identification method that integrates a bilingual sensitive dictionary and a heterogeneous graph. The method includes constructing a Chinese-Thai bilingual sensitive dictionary based on Wikipedia and social media sensitive data; then, using documents and the keywords and sensitive words they contain as nodes, and bilingual alignment, similarity relationships, and different parts of speech as edges to construct a Chinese-Thai cross-language heterogeneous graph, enhancing the sensitive features of the documents and aligning the bilingual sensitive features, and using a multilingual pre-trained model to represent document nodes and word nodes; finally, encoding the input documents through a multi-layer graph convolutional neural network, and using a sensitive information classifier to classify and predict the documents.

[0007] As a further solution of the present invention, the specific steps of the method are as follows:

[0008] Step 1: Use Python to write a web crawler program to collect and organize multilingual text data on the public network, perform data cleaning, and build a Chinese-Thai cross-language sensitive information dataset;

[0009] Step 2: Count the bilingual sensitive words in different sensitive categories from the processed data, group the bilingual sensitive words with similar meanings into phrases, and construct a Chinese-Thai bilingual sensitive dictionary.

[0010] Step 3: Using the sensitive words in the bilingual sensitive dictionary and the keywords in the document as word nodes, and the document itself as a document node, the alignment and similarity relationship between the documents, the different part-of-speech relationships between keywords and sensitive words, and the alignment relationship between the bilingual sensitive words based on the Chinese-Thai bilingual sensitive dictionary are used as edges to construct a Chinese-Thai cross-lingual heterogeneous graph.

[0011] Step 4: Use a multilingual pre-trained model to represent all nodes, obtain the global information and contextual information of the document, pass the representations of all nodes to the graph convolutional neural network, and finally perform graph convolution on different subgraphs constructed by different types of edges and aggregate features of different nodes; the obtained document node features are sent to the sensitive information classifier to finally obtain the sensitive information prediction results.

[0012] As a further solution of the present invention, the specific steps of Step 1 are:

[0013] Step 1.1: Using web crawler technology, we collected and organized a total of 158,638 multilingual text data from the public internet, including 15,798 Wikipedia data, 46,119 Weibo data, and 96,721 Twitter data. We then used language identification methods to remove non-Chinese and Thai text data. We used the emoji data package and regular expressions to remove emoticons, symbols, and hyperlinks from the text data. Data cleaning was then completed through manual data screening and organization. Finally, the Chinese and Thai text data were labeled with sensitive category labels based on their sensitive features, constructing a Chinese-Thai cross-language sensitive information recognition dataset with category labels.

[0014] As a further solution of the present invention, the specific steps of Step 2 are:

[0015] Step 2.1: Based on sensitive data from Wikipedia and social media, we manually identify sensitive words in the data with the help of machine translation methods, obtain statistics on bilingual sensitive words in different sensitive categories, and group bilingual sensitive words with similar meanings into phrases. We then establish an alignment relationship between Chinese and Thai bilingual sensitive words, thereby constructing a Chinese-Thai bilingual sensitive dictionary.

[0016] As a further solution of the present invention, Step 3 includes:

[0017] Step 3.1: Use the documents and document co-occurrence words in the Chinese-Thai cross-language sensitive information text dataset and the sensitive words in the bilingual sensitive dictionary as nodes to construct a Chinese-Thai cross-language heterogeneous graph structure. There are different types of relationships between documents, between documents and words, and between sensitive words, including translation and similarity relationships between documents, part-of-speech relationships between documents and words, and part-of-speech relationships between sensitive words.

[0018] Step 3.2: Document-to-document edges: In order to obtain the semantic information contained in the document and enable better cross-language transfer learning between Chinese and Thai documents, two types of document relationship edges are set. First, based on the relationship between the pseudo-parallel corpus of machine translation, corresponding translation edges are constructed between Chinese and Thai documents. Secondly, after obtaining the vector representation of the document through the multilingual pre-training model for the Chinese-Thai bilingual document, the document vector is used to calculate the similarity between the documents, such as document vector A = (x1, x2, ..., x l ) and document vector B=(y1,y2,…,y l ) The similarity S is obtained by calculating the cosine similarity, and the formula is shown in (1):

[0019]

[0020] For each document, select the k documents with the highest similarity S and construct similarity edges between document nodes, with the value of k being 3;

[0021] Step 3.3: Document-Word Edges: The most obvious relationship between documents and words is co-occurrence. Words have different parts of speech and contain important grammatical information. For sensitive information identification tasks, adjectives, nouns, and verbs may all contain sensitive information. Using a built bilingual sensitive dictionary to assist the word segmentation tool, we accurately segment the words in the document. Using POS-Tagger, we add part-of-speech tags and connect words of different parts of speech to co-occurring documents through part-of-speech relationships, thus constructing different types of edges.

[0022] Step 3.4: Words and Word Edges: For cross-language sensitive information identification, sensitive word pairs have a greater impact on prediction results than other words. However, sensitive words in social media texts can have a variety of parts of speech, and often only a few sensitive words appear as uncommon words. Based on the constructed bilingual sensitive word list, sensitive words segmented from the document are assigned to bilingual sensitive words with similar semantics as word nodes. A graph structure is established using edges between word nodes to increase the weight of sensitive information in the document. This also allows for cross-language word-level alignment and aggregation between Chinese and Thai.

[0023] As a further solution of the present invention, Step 4 includes the following steps:

[0024] Step 4.1: Construct a heterogeneous graph G based on the relationships contained in the Chinese-Thai cross-language sensitive information text dataset F F , consists of different types of subgraphs, each subgraph G = (V, E), where V represents defined nodes, including document nodes and word nodes; E represents different types of edges between nodes. Use a multilingual pre-trained model to encode all nodes and define the initialization X∈R n×m Is a matrix containing n nodes and their features, where m is the dimension of the feature vector, and each row X v =R m is the eigenvector of v. For each graph G we use its adjacency matrix A and degree matrix The element D in the degree matrix ii =∑ j1 A ij1 , i, j1 are the rows and columns of the adjacency matrix A respectively; a trainable weight matrix W is used (j) , for the first layer GCN of each subgraph, the m-dimensional node feature matrix H (1) =R n×m The calculation is as follows:

[0025]

[0026] Where σ() is an activation function, is an adjacency matrix of an undirected graph G with self-connection added, I N Is the identity matrix. For multi-layer GCN structures, higher-order neighborhood information can be obtained, as shown below:

[0027]

[0028] Among them, j represents the number of layers of graph convolution, H (0) =X, after the graph convolution operation is completed, all different types of sub-graph features are aggregated into a common implicit space, as shown below:

[0029]

[0030] Among them, τ represents different subgraphs, which are aggregated together to obtain the representation of the entire heterogeneous graph, and the information of word nodes is aggregated to document nodes;

[0031] Step 4.2: Then, the document feature h obtained by the GCN layer is passed through the activation function LeakyReLU and enters the fully connected layer to obtain the output. Finally, the normalized exponential function softmax function is used to predict the category of the document node to obtain the prediction values ​​corresponding to different categories. The category with the highest prediction value is the predicted classification result, as follows:

[0032]

[0033] q=Linear(p)

[0034]

[0035]

[0036] Among them, α is set to 0.01, W q and b are the weight and bias of q, respectively. q is the document feature output by the last fully connected layer of the model. Indicates the probability of the document corresponding to M categories, Represents the category result predicted by the model for the document.

[0037] The beneficial effects of the present invention are:

[0038] 1. We constructed a Chinese-Thai bilingual sensitive dictionary and a Chinese-Thai cross-language sensitive information recognition dataset. Based on the Chinese-Thai bilingual sensitive dictionary, we expanded the sensitive words in the input text and enhanced the sensitive features of the input text when representing it in the multilingual pre-training model.

[0039] 2. A Chinese-Thai cross-language heterogeneous graph was constructed, with documents, keywords, and sensitive words as nodes, and document alignment and similarity relationships, the different part-of-speech relationships between keywords and sensitive words, and the bilingual sensitive word alignment relationships based on the Chinese-Thai bilingual sensitive dictionary as edges. This enhances the relevance between Chinese and Thai input text and sensitive words, alleviating the problem of difficulty in aligning sensitive words in Chinese and Thai due to the large language differences between the two languages.

[0040] 3. A Chinese-Thai cross-language sensitive information recognition method that integrates bilingual sensitive dictionaries and heterogeneous graphs is proposed. A multi-layer graph convolutional neural network is used to aggregate Chinese-Thai cross-language information based on the different relationships contained in the Chinese-Thai cross-language heterogeneous graph, enhance the Chinese-Thai cross-language transfer learning ability, and improve the performance of Chinese-Thai cross-language sensitive information recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 This is a flow chart of the Chinese-Thai cross-language sensitive information recognition method proposed by the present invention that integrates a bilingual sensitive dictionary and a heterogeneous graph. DETAILED DESCRIPTION

[0042] Example 1: Figure 1 As shown in FIG, a Chinese-Thai cross-language sensitive information recognition method that integrates a bilingual sensitive dictionary and a heterogeneous graph is used to train a model using the constructed Chinese-Thai cross-language sensitive information dataset. The specific steps of the method are as follows:

[0043] Step 1: Use Python to write a web crawler program to collect and organize multilingual text data on the public network, perform data cleaning, and build a Chinese-Thai cross-language sensitive information dataset;

[0044] Step 2: Count the bilingual sensitive words in different sensitive categories from the processed data, group the bilingual sensitive words with similar meanings into phrases, and construct a Chinese-Thai bilingual sensitive dictionary.

[0045] Step 3: Using the sensitive words in the bilingual sensitive dictionary and the keywords in the document as word nodes, and the document itself as a document node, the alignment and similarity relationship between the documents, the different part-of-speech relationships between keywords and sensitive words, and the alignment relationship between the bilingual sensitive words based on the Chinese-Thai bilingual sensitive dictionary are used as edges to construct a Chinese-Thai cross-lingual heterogeneous graph.

[0046] Step 4: Use a multilingual pre-trained model to represent all nodes, obtain global information and contextual information of the document, pass the representations of all nodes to the graph convolutional neural network, and finally perform graph convolution on different subgraphs constructed by different types of edges, and perform feature aggregation on different nodes;

[0047] Step 5: Send the final document node features to the sensitive information classifier to finally obtain the sensitive information prediction results.

[0048] As a further solution of the present invention, the specific steps of Step 1 are:

[0049] Step 1.1: Using web crawler technology, we collected and organized a total of 158,638 multilingual text data from the public network, including 15,798 Wikipedia data, 46,119 Weibo data, and 96,721 Twitter data. Then, we used language recognition methods to remove non-Chinese and Thai text data, and used emoji data packages and regular expressions to remove emoticons, symbols, and hyperlinks from the text data. Then, we completed data cleaning through manual data screening and organization. Finally, we labeled the Chinese and Thai text data with sensitive category labels based on sensitive features, and constructed a Chinese and Thai cross-language sensitive information recognition dataset with category labels.

[0050] As a further solution of the present invention, the specific steps of Step 2 are:

[0051] Step 2.1: Based on sensitive data from Wikipedia and social media, we manually identify sensitive words in the data with the help of machine translation methods, obtain statistics on bilingual sensitive words in different sensitive categories, and group bilingual sensitive words with similar meanings into phrases. We then establish an alignment relationship between Chinese and Thai bilingual sensitive words, thereby constructing a Chinese-Thai bilingual sensitive dictionary.

[0052] As a further solution of the present invention, Step 3 includes:

[0053] Step 3.1: Use the documents and document co-occurrence words in the Chinese-Thai cross-language sensitive information text dataset and the sensitive words in the bilingual sensitive dictionary as nodes to construct a Chinese-Thai cross-language heterogeneous graph structure. There are different types of relationships between documents, between documents and words, and between sensitive words, including translation and similarity relationships between documents, part-of-speech relationships between documents and words, and part-of-speech relationships between sensitive words.

[0054] Step 3.2: Document-to-document edges: In order to obtain the semantic information contained in the document and enable better cross-language transfer learning between Chinese and Thai documents, two types of document relationship edges are set. First, based on the relationship between the pseudo-parallel corpus of machine translation, corresponding translation edges are constructed between Chinese and Thai documents. Secondly, after obtaining the vector representation of the document through the multilingual pre-training model for the Chinese-Thai bilingual document, the document vector is used to calculate the similarity between the documents, such as document vector A = (x1, x2, ..., x l ) and document vector B=(y1,y2,…,y l ) The similarity S is obtained by calculating the cosine similarity, and the formula is shown in (1):

[0055]

[0056] For each document, select the k documents with the highest similarity S and construct similarity edges between document nodes, with the value of k being 3.

[0057] Step 3.3: Document-Word Edges: The most obvious relationship between documents and words is co-occurrence. Words have different parts of speech and contain important grammatical information. For sensitive information identification tasks, adjectives, nouns, and verbs may all contain sensitive information. Using a built bilingual sensitive dictionary to assist the word segmentation tool, we accurately segment the words in the document. Using POS-Tagger, we add part-of-speech tags and connect words of different parts of speech to co-occurring documents through part-of-speech relationships, thus constructing different types of edges.

[0058] Step 3.4: Words and Word Edges: For cross-language sensitive information identification, sensitive word pairs have a greater impact on prediction results than other words. However, sensitive words in social media texts can have a variety of parts of speech, and often only a few sensitive words appear as uncommon words. Based on the constructed bilingual sensitive word list, sensitive words segmented from the document are assigned to bilingual sensitive words with similar semantics as word nodes. A graph structure is established using edges between word nodes to increase the weight of sensitive information in the document. This also allows for cross-language word-level alignment and aggregation between Chinese and Thai.

[0059] As a further solution of the present invention, Step 4 includes the following steps:

[0060] Step 4.1: Construct a heterogeneous graph G based on the relationships contained in the Chinese-Thai cross-language sensitive information text dataset F F , consists of different types of subgraphs, each subgraph G = (V, E), where V represents the defined nodes, including document nodes and word nodes; E represents the different types of edges between nodes. Use the multilingual pre-trained model to encode all nodes and define the initialization X∈R n×m Is a matrix containing n nodes and their features, where m is the dimension of the feature vector, and each row X v =R m is the eigenvector of v. For each graph G we use its adjacency matrix A and degree matrix The element D in the degree matrix ii =∑ j1 A ij1 , i, j1 are the rows and columns of the adjacency matrix A respectively; a trainable weight matrix W is used (j) , for the first layer GCN of each subgraph, the m-dimensional node feature matrix H (1) =Rn×m The calculation method is as follows:

[0061]

[0062] Where σ() is an activation function, is an adjacency matrix of an undirected graph G with self-connection added, I N Is the identity matrix. For multi-layer GCN structures, higher-order neighborhood information can be obtained, as shown below:

[0063]

[0064] Among them, j represents the number of layers of graph convolution, H (0) =X, after the graph convolution operation is completed, all different types of sub-graph features are aggregated into a common implicit space, as shown below:

[0065]

[0066] Among them, τ represents different subgraphs, which are aggregated together to obtain the representation of the entire heterogeneous graph, and the information of word nodes is aggregated to document nodes;

[0067] Step 4.2: Then, the document feature h obtained by the GCN layer is passed through the activation function LeakyReLU and enters the fully connected layer to obtain the output. Finally, the normalized exponential function softmax function is used to predict the category of the document node to obtain the prediction values ​​corresponding to different categories. The category with the highest prediction value is the predicted classification result, as follows:

[0068]

[0069] q=Linear(p)

[0070]

[0071]

[0072] Among them, α is set to 0.01, W q and b are the weight and bias of q, respectively. q is the document feature output by the last fully connected layer of the model. Indicates the probability of the document corresponding to M categories, Represents the category result predicted by the model for the document.

[0073] To illustrate the effectiveness of the present invention, three groups of comparative experiments were set up. The first group is the main experiment, using a cross-language sensitive information recognition dataset and a baseline model for comparison; the second group of experiments is a comparison of the effectiveness of the present invention using different multilingual pre-trained models; the third group of experiments compares the effectiveness of the present invention with different numbers of GCN layers;

[0074] (1) Main experimental results

[0075] First, we compared several state-of-the-art cross-language text classification models using a common cross-language classification method. The dataset used was a constructed Chinese-Thai cross-language sensitive information recognition dataset. The results in Table 1 demonstrate that our proposed method outperforms other state-of-the-art models and outperforms the baseline, CLHG, in the cross-language sensitive information recognition task. Experiments demonstrate that our proposed method can effectively identify sensitive features contained in social media text data and classify sensitive information.

[0076] Table 1: Results of different cross-lingual classification methods

[0077]

[0078] (2) Experimental results of different multilingual pre-training models

[0079] To explore the impact of different multilingual pre-trained models on model performance, we conducted comparative experiments using different multilingual pre-trained models. Table 2 shows the experimental results. When using different multilingual pre-trained models to represent nodes, the multilingual pre-trained model XLM-R performed best, while mBert and XLM performed poorly.

[0080] Table 2: Effects of different multilingual pre-training on classification results

[0081]

[0082] Table 3: Impact of the number of convolutional layers on classification results

[0083]

[0084] (3) Impact of GCN convolutional layer on model performance

[0085] To explore the impact of different numbers of GCN convolutional layers on model performance, ablation experiments were conducted with 2, 3, 4, and 5 GCN layers. The experimental results are shown in Table 3. When the number of convolutional layers is 3, the model achieves the best performance. When the number of convolutional layers is less than 3, the convolutional network's ability to aggregate information is insufficient, resulting in low model performance. When the number of convolutional layers exceeds 3, the overall performance decreases as the number of layers increases.

[0086] In summary, to address the problem of diverse representations of sensitive words in Chinese and Thai social media data, which makes bilingual sensitive word recognition and alignment difficult, we proposed a Chinese-Thai cross-lingual sensitive information recognition method that integrates a bilingual sensitive dictionary and a heterogeneous graph. This method utilizes the sensitive word alignment information from the bilingual sensitive dictionary to construct a cross-lingual heterogeneous graph, enhancing cross-lingual transfer learning capabilities. Extensive experiments have verified that the cross-lingual heterogeneous graph constructed based on the bilingual sensitive dictionary can accurately construct alignment relationships between bilingual sensitive words, enabling the target language to more effectively learn the sensitive features of the source language, thereby enhancing cross-lingual transfer learning capabilities.

[0087] The specific embodiments of the present invention are described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Various changes can be made within the knowledge of ordinary technicians in this field without departing from the scope of the present invention.

Claims

1. A Chinese-Thai cross-language sensitive information recognition method that integrates a bilingual sensitive dictionary and a heterogeneous graph is characterized by: The method includes firstly constructing a Chinese-Thai bilingual sensitive dictionary based on Wikipedia and social media sensitive data; Then, the document and its keywords and sensitive words are used as nodes, and bilingual alignment, similarity relationships, and different parts of speech are used as edges to construct a Chinese-Thai cross-lingual heterogeneous graph. This enhances the sensitive features of the document and aligns the bilingual sensitive features. A multilingual pre-trained model is used to represent document nodes and word nodes. Finally, a multi-layer graph convolutional neural network is used to encode the input document, and a sensitive information classifier is used to classify and predict the document. The construction of the Chinese-Thai cross-language heterogeneous graph includes: Step 3.1: Use the documents and document co-occurrence words in the Chinese-Thai cross-language sensitive information text dataset and the sensitive words in the bilingual sensitive dictionary as nodes to construct a Chinese-Thai cross-language heterogeneous graph structure. This graph structure contains different types of relationships between documents, between documents and words, and between sensitive words, including translation and similarity relationships between documents, part-of-speech relationships between documents and words, and part-of-speech relationships between sensitive words. Step 3.2: Document-to-Document Edges: To capture the semantic information contained in documents and facilitate cross-language transfer learning between Chinese and Thai documents, two types of document relationship edges are set. First, based on the relationship between the pseudo-parallel machine translation corpus, corresponding translation edges are constructed between Chinese and Thai documents. Second, after obtaining document vector representations for Chinese and Thai bilingual documents through a multilingual pre-trained model, the document vectors are used to calculate the similarity between the documents. Step 3.3: Document-Word Edges: Use the built bilingual sensitive dictionary to assist the word segmentation tool to accurately segment the words in the document. POS-Tagger is used to tag the words with part-of-speech tags. Words with different parts of speech are connected to co-occurring documents through part-of-speech relationships to construct different types of edges. Step 3.4: Words and word edges: Based on the constructed bilingual sensitive word list, for the sensitive words segmented from the document, bilingual sensitive words with similar semantics are used as word nodes, and a graph structure is established through the edges between word nodes to increase the weight of sensitive information in the document, as well as Chinese-Thai cross-language word-level alignment and aggregation.

2. The Chinese-Thai cross-language sensitive information identification method integrating a bilingual sensitive dictionary and a heterogeneous graph according to claim 1 is characterized by: The specific steps of the method are as follows: Step 1: Use Python to write a web crawler program to collect and organize multilingual text data on the public network, perform data cleaning, and build a Chinese-Thai cross-language sensitive information dataset; Step 2: Count the bilingual sensitive words in different sensitive categories from the processed data, group the bilingual sensitive words with similar meanings into phrases, and construct a Chinese-Thai bilingual sensitive dictionary. Step 3: Sensitive words in the Chinese-Thai bilingual sensitive dictionary and keywords in the document are used as word nodes, the document itself is used as a document node, and the alignment and similarity relationship between documents, the different part-of-speech relationships between keywords and sensitive words, and the alignment relationship between bilingual sensitive words based on the Chinese-Thai bilingual sensitive dictionary are used as edges to construct a Chinese-Thai cross-lingual heterogeneous graph. Step 4: Use a multilingual pre-trained model to represent all nodes, obtain the global information and contextual information of the document, pass the representations of all nodes to the graph convolutional neural network, and finally perform graph convolution on different subgraphs constructed by different types of edges and aggregate features of different nodes; the obtained document node features are sent to the sensitive information classifier to finally obtain the sensitive information prediction results.

3. The Chinese-Thai cross-language sensitive information identification method integrating a bilingual sensitive dictionary and a heterogeneous graph according to claim 2 is characterized by: The specific steps of Step 1 are: Step 1.1: Use web crawler technology to collect and organize multilingual text data from the public network. Then, use language recognition methods to remove non-Chinese and Thai text data. Use emoji data packages and regular expressions to remove emoticons, symbols, and hyperlinks from the text data. Then, complete the data cleaning through manual data screening and organization. Finally, label the Chinese and Thai text data with sensitive category labels based on sensitive features to construct a Chinese and Thai cross-language sensitive information recognition dataset with category labels.

4. The Chinese-Thai cross-language sensitive information identification method integrating a bilingual sensitive dictionary and a heterogeneous graph according to claim 2 is characterized by: The specific steps of Step 2 are: Step 2.1: Based on sensitive data from Wikipedia and social media, we manually identify sensitive words in the data with the help of machine translation methods, obtain statistics on bilingual sensitive words in different sensitive categories, and group bilingual sensitive words with similar meanings into phrases. We then establish an alignment relationship between Chinese and Thai bilingual sensitive words, thereby constructing a Chinese-Thai bilingual sensitive dictionary.

5. The Chinese-Thai cross-language sensitive information identification method integrating a bilingual sensitive dictionary and a heterogeneous graph according to claim 2 is characterized by: Step 4 includes the following steps: Step 4.1: Construct a heterogeneous graph G based on the relationships contained in the Chinese-Thai cross-language sensitive information text dataset F F , consists of different types of subgraphs, each subgraph G = (V, E), V represents the defined nodes, including document nodes and word nodes; E represents the different types of edges between nodes; use the multilingual pre-training model to encode all nodes and define the initialization X∈R n×m Is a matrix containing n nodes and their features, where m is the dimension of the feature vector, and each row X v ∈R m is the eigenvector of V; for each graph G, use its adjacency matrix A and degree matrix The element D in the degree matrix ii =∑ j1 A ij , i, j1 are the rows and columns of the adjacency matrix A respectively; a trainable weight matrix W is used (j) , for the first layer GCN of each subgraph, the m-dimensional node feature matrix H (1) ∈R n×m The calculation is as follows: Where σ() is an activation function, is an adjacency matrix of an undirected graph G with self-connection added, I N It is the unit matrix. For the multi-layer GCN structure, higher-order neighborhood information can be obtained, as shown below: Among them, j represents the number of layers of graph convolution, H (0) =X, after the graph convolution operation is completed, all different types of sub-graph features are aggregated into a common implicit space, as shown below: Among them, τ represents different subgraphs, which are aggregated together to obtain the representation of the entire heterogeneous graph, and the information of word nodes is aggregated to document nodes; Step 4.2: Then, the document feature h obtained by the GCN layer is passed through the activation function LeakyReLU and enters the fully connected layer to obtain the output. Finally, the normalized exponential function softmax function is used to predict the category of the document node to obtain the prediction values ​​corresponding to different categories. The category with the highest prediction value is the predicted classification result, as follows: q=Linear(p) Among them, α is set to 0.01, W q and b are the weight and bias of q, respectively. q is the document feature output by the last fully connected layer of the model. Indicates the probability of the document corresponding to M categories, Represents the category result predicted by the model for the document.

Images