A Chinese metaphor interpretation method based on semantic matching and density clustering

Abstract

The invention discloses a Chinese metaphor interpretation method based on semantic matching and density clustering, comprising the following steps: for a metaphor sentence, we first determine the source domain and the target domain; construct a candidate word set for the target domain and the source domain; obtain word vector representations for the candidate words, the target domain and the source domain in the candidate word set, and then respectively calculate the semantic similarity between the candidate words and the target domain and the source domain, and respectively calculate the mutual information between the candidate words and the target domain and the source domain, and then count the frequency of the candidate words in the existing corpus; finally, obtain the final matching score using the semantic similarity, mutual information and frequency information of the candidate words, cluster the candidate words using the DBSCAN clustering method, and sort them according to the final matching score to obtain the best interpretation word list. The invention solves the problem of the current lack of data on specialized Chinese metaphor interpretations, and can use unsupervised learning methods to reasonably interpret metaphors appearing in Chinese texts.

Description

Technical Field

[0001] The present invention belongs to the technical field of natural language processing, and in particular relates to a Chinese metaphor interpretation method based on semantic matching and density clustering. Background Art

[0002] Metaphor is ubiquitous in language and thought. Metaphors often have ambiguous interpretations. For example, when we say that memory is a river, flowing and long may be considered reasonable interpretations. The intended meaning of a metaphor may be related to the source domain, the target domain, or both. For example, when it is mentioned that "a lawyer is a fox", it is often described as cunning or treacherous, which is more closely related to the target domain "fox" than the source domain "lawyer".

[0003] Researchers have found from corpus studies that metaphorical expressions appear in an average of every three sentences in people's daily conversations, which makes it necessary to accurately interpret metaphorical language for many natural language processing tasks and applications.

[0004] However, there is a lack of specialized data on Chinese metaphor interpretation. Therefore, it is necessary to use unsupervised methods to conduct relevant research on Chinese metaphor interpretation. Summary of the invention

[0005] The purpose of the present invention is to solve the problems existing in the prior art and provide a Chinese metaphor interpretation method based on semantic matching and density clustering, which can reasonably interpret the metaphors appearing in Chinese texts using unsupervised learning methods.

[0006] To achieve the above object, the present invention proposes the following technical solutions:

[0007] A Chinese metaphor interpretation method based on semantic matching and density clustering, comprising the steps of:

[0008] S1. Determine the source domain and target domain of the metaphor sentence to be interpreted;

[0009] S2, constructing candidate word sets for metaphor interpretation for the source domain and the target domain;

[0010] S3, for each candidate word in the candidate word set, respectively calculating its semantic similarity with the source domain and its semantic similarity with the target domain;

[0011] S4. For each candidate word in the candidate word set, calculate the mutual information between it and the source domain and the mutual information between it and the target domain respectively;

[0012] S5. Count the frequency of candidate words appearing in the metaphor interpretation corpus;

[0013] S6. For each candidate word c i, the semantic similarity with the source domain, the semantic similarity with the target domain, the mutual information with the source domain, the mutual information with the target domain, and the frequency of occurrence in the metaphor interpretation corpus are aggregated by weight to obtain the candidate word c i The final matching score of

[0014] S7. Cluster each candidate word in the candidate word set using the DBSCAN clustering method, extract the candidate word with the highest final matching score in each cluster obtained after clustering, and then sort the extracted candidate words according to their corresponding final matching scores to form a list of optimal interpretation words.

[0015] Preferably, in S1, the source domain and the target domain in the metaphorical sentence are determined by sentence dependency analysis.

[0016] Preferably, the candidate word set C in S2 is generated by word association.

[0017] Preferably, the semantic similarity in S3 is obtained by vectorizing the characters of the candidate words and then calculating the similarity between the candidate words and the source domain or the target domain by using cosine similarity.

[0018] Preferably, the candidate words are converted into real-valued vectors using a Word2vec pre-trained language model.

[0019] Preferably, the mutual information calculation method used in S4 is normalized mutual information.

[0020] Preferably, in S6, for any candidate word c i The final matching score calculation formula is:

[0021] score(c i )=λ1(sem(c i ,s))+λ2(sem(c i ,t))+λ3(npmi(c i ,s))+λ4(npmi(c i ,t))+λ5(freq(c i ))

[0022] In the formula, sem(c i ,s) represents the candidate word c i The cosine similarity score with the source domain s, sem(c i ,t) represents the candidate word c i The cosine similarity score with the target domain t, npmi(c i ,s) represents the candidate word candidate word c i The normalized mutual information score with the source domain s, npmi(c i ,t) represents the candidate word candidate word ci The normalized mutual information score with the target domain t, freq(c i ) represents the candidate word c i The frequency of occurrence in the metaphor interpretation corpus, λ1, λ2, λ3, λ4, and λ5 are the five weights.

[0023] Preferably, the five weights λ1, λ2, λ3, λ4, and λ5 are determined by optimizing within their respective value ranges, wherein the value range of λ5 is [1, 10], and the value ranges of the remaining four weights are [0.1, 1].

[0024] Preferably, for the candidate words, the characters are converted into real-valued vectors using a Word2vec pre-trained language model.

[0025] Compared with the prior art, the present invention solves the problem of the current lack of data on specialized Chinese metaphor interpretations, and can use unsupervised learning methods to reasonably interpret metaphors appearing in Chinese texts. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 A flowchart of the steps of the Chinese metaphor interpretation method based on semantic matching and density clustering;

[0027] Figure 2 The dependency analysis result of a sentence;

[0028] Figure 3 Dependency analysis results of another sentence for dependency analysis;

[0029] Figure 4 It is a schematic diagram of the result of DBSCAN clustering in an embodiment of the present invention. DETAILED DESCRIPTION

[0030] In order to make the above-mentioned purpose, features and advantages of the present invention more obvious and easy to understand, the specific implementation mode of the present invention is described in detail below in conjunction with the accompanying drawings. In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below. The technical features in each embodiment of the present invention can be combined accordingly without conflicting with each other.

[0031] In a preferred embodiment of the present invention, a Chinese metaphor interpretation method based on semantic matching and density clustering is provided, and the steps are as follows:

[0032] Step 1: Obtain metaphorical sentences that need to be interpreted metaphorically from Chinese texts, perform syntactic structure analysis on the metaphorical sentences, determine the source domain s and target domain t, and form the target word pair<s,t> ;

[0033] The source domain and target domain in a metaphorical sentence can be determined through sentence dependency analysis. Figure 2 and Figure 3 As shown in the figure, hanlp is used to perform dependency analysis on the sentences "The rising sun is a huge, dazzling ruby ​​in the plate" and "The night is as quiet as dead water", and obtains target word pairs such as <sun, ruby> and <night, dead water>.

[0034] Step 2: Construct a word association set, and obtain a candidate word set C for metaphor interpretation that matches the source domain and the target domain from the word association set.

[0035] (1) A word association set is a word pair composed of certain words that appear frequently in the same text in a large-scale text. More specifically, when people mention the first word, the second word will unconsciously come to their mind. For example, when people talk about "bank", they will unconsciously think of "money", and naturally "<bank, money>" will form a word pair; when talking about "school", they will mention "homework, university, elementary school, middle school, books, study, teacher, class, classmates, education" and so on. These can all form word pairs.

[0036] (2) In order to further expand the scope of candidate words, synonyms are added to the word association set. For example, "ridiculous, funny, spit out food, laugh out loud, make a fool of oneself, a lot of jokes, laugh out loud, ridiculous, hilarious", these words can be regarded as synonyms of each other.

[0037] (3) From the word association set, find word pairs in the source domain or target domain. For example, “the city is a jungle”, the matches with the source domain “city” are <city, work>, <city, subway>, <city, crowded>, <city, life>, etc., and the matches with the target domain “jungle” are <jungle, danger>, <jungle, crazy>, <jungle, crowded>, etc., so the candidate word set is {work, subway, crowded, life, danger, crazy, ...}.

[0038] Step 3. For each candidate word c in the candidate word set C i (c i ∈C), and calculate the semantic similarity sem(c i ,s) and the semantic similarity sem(c i ,t).

[0039] (1) For source domain s, target domain t and candidate word c i, use the word2vec pre-trained language model for vector representation and turn it into a computable real-valued vector.

[0040] (2) Based on the source domain s, target domain t and candidate word c i The vector representation of each domain is calculated using cosine similarity between the source domain s and the target domain t, and between the source domain s and the candidate word c. i The semantic similarity of i ,s) and sem(c i ,t).

[0041] Step 4. For each candidate word c in the candidate word set C i , respectively calculate the mutual information npmi(c i ,s) and the mutual information with the target domain npmi(c i ,t).

[0042] In this embodiment, the mutual information calculation method used is normalized mutual information. Therefore, for the candidate word c in the candidate word set i , the normalized mutual information npmi(c i ,s) and npmi(c i ,t), the normalized mutual information is similar to the mutual information, but its value will only be between [-1,1].

[0043]

[0044] In the formula, p(x) represents the frequency of x appearing in the metaphor interpretation corpus, and p(x,y) represents the frequency of x and y appearing together in the same sentence in the metaphor interpretation corpus. The metaphor interpretation corpus is a dataset containing a large amount of metaphor interpretation corpus.

[0045] Step 5. Count candidate words c i The frequency of occurrence in the metaphor interpretation corpus is obtained by freq(c i ).

[0046] Step 6. Aggregate the calculation results and statistical results obtained from steps 3 to 5 according to the following weighted formula (4) to obtain candidate word c i The final matching score score(c i ):

[0047] score(c i )=λ1(sem(c i ,s))+λ2(sem(c i ,t))+λ3(npmi(c i ,s))+λ4(npmi(ci ,t))+λ5(freq(c i )) (4)

[0048] In the formula, sem(c i ,s) represents the candidate word c i The cosine similarity score with the source domain s, sem(c i ,t) represents the candidate word c i The cosine similarity score with the target domain t, npmi(c i ,s) represents the candidate word candidate word c i The normalized mutual information score with the source domain s, npmi(c i ,t) represents the candidate word candidate word c i The normalized mutual information score with the target domain t, freq(c i ) represents the candidate word c i The frequency of occurrence in the metaphor interpretation corpus, λ1, λ2, λ3, λ4, and λ5 are the five weights.

[0049] It should be noted that the five weights λ1, λ2, λ3, λ4, and λ5 need to be determined in advance by optimizing within their respective value ranges, but when applied to the actual metaphor interpretation task, the five weights are fixed values. When optimizing the weights, all possible value combinations can be tried for each weight. i ) has a value of λ5 in the range of [1, 10], and the other four weights have values ​​in the range of [0.1, 1]. The final result is evaluated using the MRR indicator, as shown in formula (5). Thus, λ is adaptively adjusted on the metaphor interpretation corpus to obtain the best interpretation words for Chinese metaphor sentences.

[0050]

[0051] Among them, rank s and rank t Represents candidate word c i The source domain s and the target domain t are matched and ranked in the metaphor interpretation corpus respectively. The higher the matching degree, the smaller the ranking value and the larger the MRR index. Therefore, for all the combinations of the five weights, the best values ​​can be optimized to maximize the MRR index, and then used to calculate each candidate word c i The final matching score score(c i ).

[0052] Step 7. Use the DBSCAN clustering method to cluster each candidate word c in the candidate word set C. iClustering is performed, and the candidate words with the highest final matching score in each cluster obtained after clustering are extracted. The extracted candidate words are then sorted according to their corresponding final matching scores to form a list of best interpretation words. The specific number of interpretation words K displayed in the best interpretation word list can be adjusted according to actual needs, and the top K candidate words are returned to the user as the best interpretation words based on the final matching score.

[0053] The DBSCAN clustering method belongs to the prior art, and its clustering process is as follows:

[0054] (1) For each candidate word c i Draw a circle with ε∈[1,6] as the radius and centered at .

[0055] (2) Count the number of candidate words within the circle. This statistical value is the candidate word c i The density value of

[0056] (3) Then select a density threshold μ∈[1,12], if the density value of the candidate word is greater than this density value, it is a high density point, and if the density value of the candidate word is lower than this density value, it is a low density point;

[0057] (4) If a high-density point is within another high-density point, link the two candidate words;

[0058] (5) If there is a low-density point also within the high-density point, connect the candidate word with the candidate word of the nearest high-density point

[0059] (6) Finally, all the candidate words connected together form a cluster.

[0060] The DBSCAN clustering results in this embodiment are as follows: Figure 4 As shown in the figure, the circular points are high-density candidate words, the square points are low-density candidate words, the triangular points are independent points, i.e., noise, and the circular points and square points are connected to form a cluster. In each cluster obtained by clustering, when there are multiple candidate words, only the candidate word with the highest final matching score is retained, and the retained candidate words are then sorted and recommended according to their corresponding final matching scores.

[0061] It has been verified that the above Chinese metaphor interpretation method based on semantic matching and density clustering can reasonably interpret the metaphors appearing in Chinese texts. It can give an accurate list of the best interpretation words, thus solving the problem of the current lack of specialized Chinese metaphor interpretation data.

[0062] The above-described embodiment is only a preferred solution of the present invention, but it is not intended to limit the present invention. A person skilled in the relevant technical field may make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, any technical solution obtained by equivalent replacement or equivalent transformation falls within the protection scope of the present invention.

Claims

1. A Chinese metaphor interpretation method based on semantic matching and density clustering, characterized in that: Includes steps: S1. Determine the source domain and target domain of the metaphor sentence to be interpreted; S2, constructing a set of candidate words for metaphor interpretation for the source domain and the target domain; S3, for each candidate word in the candidate word set, respectively calculating its semantic similarity with the source domain and its semantic similarity with the target domain; S4. For each candidate word in the candidate word set, calculate the mutual information between it and the source domain and the mutual information between it and the target domain respectively; S5. Count the frequency of candidate words appearing in the metaphor interpretation corpus; S6. For each candidate word c i , the semantic similarity with the source domain, the semantic similarity with the target domain, the mutual information with the source domain, the mutual information with the target domain, and the frequency of occurrence in the metaphor interpretation corpus are aggregated by weight to obtain the candidate word c i The final matching score of S7. Cluster each candidate word in the candidate word set using the DBSCAN clustering method, extract the candidate word with the highest final matching score in each cluster obtained after clustering, and then sort the extracted candidate words according to their corresponding final matching scores to form a list of optimal interpretation words.

2. According to claim 1, a Chinese metaphor interpretation method based on semantic matching and density clustering is characterized in that: In S1, the source domain and the target domain in the metaphorical sentence are determined through sentence dependency analysis.

3. According to claim 1, a Chinese metaphor interpretation method based on semantic matching and density clustering is characterized in that: The candidate word set C in S2 is generated by word association.

4. The Chinese metaphor interpretation method based on semantic matching and density clustering according to claim 1 is characterized in that: The semantic similarity in S3 is obtained by vectorizing the candidate word characters and then calculating the similarity between the candidate word and the source domain or the target domain through cosine similarity.

5. The Chinese metaphor interpretation method based on semantic matching and density clustering according to claim 4 is characterized in that: The candidate words are converted into real-valued vectors using the Word2vec pre-trained language model.

6. The Chinese metaphor interpretation method based on semantic matching and density clustering according to claim 1 is characterized in that: The mutual information calculation method used in S4 is normalized mutual information.

7. The Chinese metaphor interpretation method based on semantic matching and density clustering according to claim 1 is characterized in that: In S6, for any candidate word c i The final matching score calculation formula is: score(c i )=λ1(sem(c i ,s))+λ2(sem(c i ,t))+λ3(npmi(c i ,s))+λ4(npmi(c i ,t))+λ5(freq(c i )) In the formula, sem(c i ,s) represents the candidate word c i The cosine similarity score with the source domain s, sem(c i ,t) represents the candidate word c i The cosine similarity score with the target domain t, npmi(c i ,s) represents the candidate word candidate word c i The normalized mutual information score with the source domain s, npmi(c i ,t) represents the candidate word candidate word c i The normalized mutual information score with the target domain t, freq(c i ) represents the candidate word c i The frequency of occurrence in the metaphor interpretation corpus, λ1, λ2, λ3, λ4, and λ5 are the five weights.

8. The Chinese metaphor interpretation method based on semantic matching and density clustering according to claim 7 is characterized in that: The five weights λ1, λ2, λ3, λ4, and λ5 are determined by optimizing within their respective value ranges, where the value range of λ5 is [1, 10], and the value ranges of the remaining four weights are [0.1, 1].

Images