A method for stance analysis of low-resource language texts based on mapped memory networks

Abstract

The present invention relates to the technical field of natural language understanding, and discloses a method for analyzing the stance of text in a low-resource language based on a mapping memory network. The method analyzes the stance of text sentences in a target language based on text sentences in a source language and stance labels of the text sentences in the source language. The method specifically includes: obtaining word features, keyword features, and hidden layer representations of each word in the text sentences in the target language; translating the keyword features in the target language into keyword features in the source language, and calculating the weights and memory slot features of the keyword features in the source language; calculating the output features corresponding to the word features of each word in the text sentences in the target language, and obtaining the stance analysis results of the text sentences in the target language after average pooling and classifier. The present invention makes full use of the knowledge information in the source language domain, so that the text stance analysis in the target language is more accurate, and can more accurately analyze the text stance in a cross-language environment.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language understanding, and in particular to a method for analyzing text stance in a low-resource language based on a mapping memory network. Background Art

[0002] Cross-language text stance analysis is an important research direction in the fields of natural language processing and computational linguistics, especially in social media analysis and text data mining. With the rapid development of social media platforms, text information on the Internet, such as opinion comments, has increased dramatically. On social media platforms in different countries, there are texts in multiple languages, and the number distribution is different. The number of opinion comments in mainstream languages ​​accounts for the majority. For example, on social media platforms in my country, the vast majority of opinion comments are in Chinese, and there are a small number of opinion comments in English, Japanese, Korean and other languages. Existing stance analysis methods can achieve better text stance analysis results when there are many samples (i.e. mainstream languages). However, there is still a large gap in text analysis for a few languages. Therefore, how to effectively detect and analyze users' stances in cross-language texts has become an urgent problem to be solved. Stance analysis can not only reflect the user's inherent values, but also has important practical significance in the field of security.

[0003] At present, most stance analysis research focuses on single language and multi-sample scenarios, using rich annotated corpora for model training. However, for low-resource languages, annotated data is usually very scarce, which makes the promotion and application of cross-language stance analysis face great challenges. To solve this problem, researchers have proposed cross-language stance analysis techniques that use data from high-resource languages ​​for model training and transfer their model knowledge to low-resource target languages, ignoring the contextual information of the target language and the alignment information between the target language and the source language. Some existing technologies rely on limited annotated data in the target language, but in practical applications, when the target language lacks annotated data, the performance of these methods is often unsatisfactory. In addition, cross-language stance analysis also faces the problem of target expression diversity. Traditional stance analysis tasks usually focus on specific people or events, and the targets usually appear in the form of keywords or phrases. However, in practical applications, the targets of stance analysis are usually expressed as sentence-level topics (such as questions or statements). Therefore, how to effectively represent and process these diverse topic expressions in cross-language scenarios has become a key issue in the field of stance analysis. Summary of the invention

[0004] To solve the above technical problems, the present invention provides a method for stance analysis of low-resource language texts based on a mapping memory network, which mainly includes the following advantages: (1) Accurate stance analysis: Through the mapping memory module, the keywords and their related information of the source language are effectively mapped to the target language, achieving high-precision stance analysis, and even if the target language lacks labeled data, good results can be achieved; (2) Full use of source language knowledge: The mapping memory module takes into account the contextual weight differences of keywords, and by calculating the mutual information between keywords and stances, ensures that the prior knowledge of the source language can be effectively transferred to the target language, significantly improving the detection accuracy; (3) Logical coherence and consistency: The present invention introduces precise keyword information to integrate prior information, ensuring that the detection results are logically coherent and consistent.

[0005] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0006] A method for analyzing text stance of a low-resource language based on a mapping memory network, which analyzes the stance of text sentences in a target language through text sentences in a source language and stance labels of the text sentences in the source language; wherein the number of text sentences in the target language is less than the number of text sentences in the source language; and specifically comprises the following steps:

[0007] Step 1: Feature extraction: Given a text sentence in the target language It consists of N words. for The i-th word in Encode each word in to get word features , express The word features of the i-th word in Perform keyword extraction to obtain K keywords in the target language. The features of the K keywords in the target language constitute the target language keyword feature set. ,in for The keyword features in the kth target language; according to the word features Is it , connect 0 or 1 after the word feature, encode the connected word features through the encoder, and get each word feature The corresponding hidden layer representation ;

[0008] Step 2: Map memory: Each keyword feature in the target language is translated into the keyword feature in the source language to obtain the source language keyword feature set , for The keyword features of the k-th source language in Each keyword feature in The importance of Weight ; Calculate the mutual information between the keyword features in each source language and each stance ,in The label representing the jth position; connect all mutual information to the weighted keyword features in the source language to obtain the memory slot features of the keywords in each source language ;

[0009] Step 3: Analysis of the standpoint of memory enhancement: ,if The i-th word in The word features belong to the keyword features in the target language, then Memory slot features of keywords in the corresponding source language As knowledge information is integrated into the hidden layer representation In, get The corresponding output features , is to make The subscript of ;if The i-th word in The word features do not belong to the keyword features in the target language. The corresponding output features are represented by the hidden layer Indicates that ; For all output features Perform average pooling and compare the average pooling result with the text sentence The overall characteristics of the sentence Connect and obtain the overall sentence representation r: Use a classifier to classify the overall sentence representation r and obtain the stance analysis result of the text sentence in the target language.

[0010] Further, in step 2, the Softmax function is used to Each keyword feature in The importance of Weight , specifically including:

[0011] k-th keyword feature Weight for:

[0012] ;

[0013] , is the Softmax function.

[0014] Furthermore, in step 2, the mutual information between the keyword features in each source language and each stance is calculated. , specifically including:

[0015] ;

[0016] in, Indicates all the source language contains In the text sentence of The number of samples; Indicates all the source language contains The number of sentences in the text.

[0017] Furthermore, in step 2, all mutual information is connected to the weighted keyword features in the source language to obtain the memory slot features of the keywords in each source language. , specifically including:

[0018] ;

[0019] in, Indicates a connection operation. Indicates the total number of stance labels.

[0020] Further, in step 1, the word feature Is it , connect 0 or 1 after the word feature, encode the connected word features through the encoder, and get each word feature The corresponding hidden layer representation , specifically including:

[0021] for Each word in Word features , define the function :

[0022] ;

[0023] At the same time, Add "[CLS]" to the beginning of the sentence, and the corresponding word feature is recorded as , Set to 1;

[0024] Will Connect to each word feature Finally, the encoder is used to encode the concatenated word features to obtain the hidden layer representation corresponding to each word. ;in It is the hidden layer feature of the “[CLS]” position, which is used to represent the overall characteristics of the sentence.

[0025] Furthermore, step three specifically includes:

[0026] calculate The corresponding output features :

[0027] ;

[0028] in, Indicates The subscript is used to find The corresponding keyword features, ; represents a linear layer, , W and b are learnable parameters, represents the linear layer input, represents the output of the linear layer;

[0029] Afterwards, for all Use average pooling operation, that is, sum and average, and then connect to the overall representation of the sentence On top, we get the overall sentence representation r:

[0030] ;

[0031] Indicates a connection operation;

[0032] Finally, a classifier is used to classify the overall sentence representation r to obtain the stance analysis result of the text sentence in the target language.

[0033] Compared with the prior art, the beneficial technical effects of the present invention are:

[0034] (1) The knowledge information in the source language domain is fully utilized to make the text stance analysis in the target language more accurate; (2) A mapping memory network is designed to fully integrate the weight information in the sentence to which it belongs and the sample prior information of each stance corresponding to the keywords in the source language to better utilize the knowledge information in the source language; (3) The text stance can be analyzed more accurately in a cross-language environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 The figure is a flow chart of a stance analysis method provided according to an embodiment of the present invention. DETAILED DESCRIPTION

[0036] A preferred embodiment of the present invention is described in detail below with reference to the accompanying drawings.

[0037] Cross-language text stance analysis technology is usually defined as: analyzing the stance of each sample (text sentence) in the target language domain when there are enough source language domain samples (text sentences) and corresponding labels. The number of samples in the target language domain is much smaller than the number of samples in the source language domain.

[0038] The process of the position analysis method proposed in the present invention is as follows: Figure 1 As shown in Figure 1, it includes the following three parts: (1) feature extraction, (2) mapping memory module, and (3) memory-enhanced stance analysis. These three parts are introduced in detail below.

[0039] 1. Feature extraction:

[0040] Given a text sentence consisting of N words in the target language ,right Encode each word in to get word features , Represents a text sentence The i-th word in Represents a text sentence The word features of the i-th word in .

[0041] First, we use keyword extraction technology to extract the target language text sentences. Extract K keywords, and the features of the K target language keywords form the target language keyword feature set . Use translation technology to translate the keyword features in each target language into the keyword features in the source language to obtain the source language keyword feature set .

[0042] Secondly, for the text sentences The word features of each word in , define the function :

[0043] ;

[0044] At the same time, in the text sentence Add "[CLS]" to the beginning of the sentence. Set the value of to 1.

[0045] Finally, Connect to each word feature Finally, the encoder is used to encode the concatenated word features to obtain the hidden layer representation of each word .in It is the hidden layer feature of the “[CLS]” position, which is used to represent the overall characteristics of the text sentence.

[0046] 2. Mapped memory module:

[0047] The source language keyword feature set The elements in are regarded as a set of keys. In different contexts, that is, in different sets, the weight of each keyword should be different. Therefore, the Softmax function is used to evaluate the importance of keywords. The weight of the kth keyword feature is for:

[0048] ;

[0049] , so the weighted keyword feature is .

[0050] At the same time, because there are more samples in the source language, the mutual information between each keyword feature and each stance is calculated. .

[0051] ;

[0052] in, Indicates all the source language contains In the text sentence of So, Indicates all the source language contains The number of sentences in the text. In essence, mutual information is a set of probabilities that record the probability of each keyword feature belonging to the position in the source language. Connect all mutual information to the weighted keyword features to obtain the memory slot feature of each keyword :

[0053] ;

[0054] in, Indicates a connection operation. Indicates the total number of positions. Memory slot features Integrated keywords Weighted features in the sentence it belongs to, and keywords in the source language The sample prior information corresponding to each position. The memory slot feature makes full use of the sample prior in the source language, also known as the keyword feature Knowledge information in the source language domain.

[0055] 3. Analysis of the standpoint of memory enhancement:

[0056] For each word ,if , then the word is a keyword, and there is knowledge information in the corresponding source language domain, and its knowledge information is integrated into its hidden layer representation The corresponding output features are obtained ;if , then the word It is not a keyword and has no corresponding knowledge information. Its output features are directly represented by its hidden layer. Indicates that The above content can be formalized as follows:

[0057] ;

[0058] in, Indicates The subscript of , used to find Corresponding keyword features to further utilize their knowledge information.

[0059] Afterwards, for all Use average pooling operation, that is, sum and average, and then connect to the overall features of the sentence On top, we get the overall sentence representation r:

[0060] .

[0061] Finally, a classifier is used to classify the overall sentence representation r to obtain the stance analysis result of the text sentence in the target language.

[0062] It is obvious to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential features of the present invention. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-limiting, and the scope of the present invention is defined by the appended claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention, and any reference numerals in the claims should not be regarded as limiting the claims involved.

[0063] In addition, it should be understood that although this specification is described in terms of implementation methods, not every implementation method contains only one independent technical solution. This narrative method of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation methods that can be understood by those skilled in the art.

Claims

1. A method for analyzing stance of low-resource language text based on mapped memory network, characterized in that: Analyzing the stance of the text sentences in the target language through the text sentences in the source language and the stance labels of the text sentences in the source language; wherein the number of the text sentences in the target language is less than the number of the text sentences in the source language; specifically comprising the following steps: Step 1: Feature extraction: Given a text sentence in the target language It consists of N words. for The i-th word in Encode each word in to get word features , express The word features of the i-th word in Perform keyword extraction to obtain K keywords in the target language. The features of the K keywords in the target language constitute the target language keyword feature set. ,in for The keyword features in the kth target language; according to the word features Is it , connect 0 or 1 after the word feature, encode the connected word features through the encoder, and get each word feature The corresponding hidden layer representation ; Step 2: Map memory: Each keyword feature in the target language is translated into the keyword feature in the source language to obtain the source language keyword feature set , for The keyword features of the k-th source language in Each keyword feature in The importance of Weight ; Calculate the mutual information between the keyword features in each source language and each stance ,in The label representing the jth position; connect all mutual information to the weighted keyword features in the source language to obtain the memory slot features of the keywords in each source language ; Step 3: Analysis of the standpoint of memory enhancement: ,if The i-th word in The word features belong to the keyword features in the target language, then Memory slot features of keywords in the corresponding source language As knowledge information is integrated into the hidden layer representation In, get The corresponding output features , is to make The subscript of ;if The i-th word in The word features do not belong to the keyword features in the target language. The corresponding output features are represented by the hidden layer Indicates that ; For all output features Perform average pooling and compare the average pooling result with the text sentence The overall characteristics of the sentence Connect and obtain the overall sentence representation r: Use a classifier to classify the overall sentence representation r and obtain the stance analysis result of the text sentence in the target language.

2. The method for analyzing low-resource language text stance based on mapped memory network according to claim 1, characterized in that: In step 2, the Softmax function is used to Each keyword feature in The importance of Weight , specifically including: k-th keyword feature Weight for: ； , is the Softmax function.

3. The method for analyzing low-resource language text stance based on mapped memory network according to claim 1, characterized in that: In step 2, the mutual information between the keyword features in each source language and each stance is calculated. , specifically including: ； in, Indicates all the source language contains In the text sentence of The number of samples; Indicates all the source language contains The number of sentences in the text; Indicates all the source language contains The index of the stance label type of the text sentence.

4. The method for analyzing stance of low-resource language text based on mapped memory network according to claim 1, characterized in that: In step 2, all mutual information is connected to the weighted keyword features in the source language to obtain the memory slot features of the keywords in each source language. , specifically including: ； in, Indicates a connection operation. Indicates the total number of stance labels.

5. The method for analyzing stance of low-resource language text based on mapped memory network according to claim 1, characterized in that: In step 1, the word features Is it , connect 0 or 1 after the word feature, encode the connected word features through the encoder, and get each word feature The corresponding hidden layer representation , specifically including: for Each word in Word features , define the function : ； At the same time, Add "[CLS]" to the beginning of the sentence, and the corresponding word feature is recorded as , Set to 1; Will Connect to each word feature Finally, the encoder is used to encode the concatenated word features to obtain the hidden layer representation corresponding to each word. ;in It is the hidden layer feature of the "[CLS]" position, which is used to represent the overall characteristics of the sentence.

6. The method for analyzing stance of low-resource language texts based on mapped memory networks according to claim 5, characterized in that: Step three specifically includes: calculate The corresponding output features : ； in, Indicates The subscript of The corresponding keyword features, ; represents a linear layer; Afterwards, for all Use average pooling operation, that is, sum and average, and then connect to the overall representation of the sentence On top, we get the overall sentence representation r: ； Indicates a connection operation; Finally, a classifier is used to classify the overall sentence representation r to obtain the stance analysis result of the text sentence in the target language.

Images