Method for calculating sentence similarity, related device, and computer storage medium

Abstract

The present application discloses a method for calculating sentence similarity, a related device, and a computer storage medium, wherein the method includes: obtaining two sentences to be compared and a decay factor; vectorizing the phrases and negations in the two sentences to be compared respectively to obtain word vectors for each phrase and negation in the two sentences to be compared; determining the word vector similarity between the two sentences to be compared based on the word vectors for each phrase and negation in the two sentences to be compared; generating a dependency tree for each sentence to be compared according to a preset dependency syntactic rule; and determining the sentence similarity between the two sentences to be compared based on the decay factor, the dependency tree for each sentence to be compared, and the word vector similarity between the two sentences to be compared. Compared with the prior art method of calculating the similarity between sentences based only on the dependency tree, the embodiments of the present application are more accurate.

Description

Technical Field

[0001] The present application relates to the field of computer language processing, and in particular to a method for calculating sentence similarity, a related device, and a computer storage medium. Background Art

[0002] As the intelligence level of electronic products continues to improve, the requirements for text recognition accuracy are also getting higher and higher.

[0003] Existing techniques typically use sentence vector models to calculate text similarity, such as Onehot, Word2vector, BiLSTM, and Bert. However, if the sentence vectors of two sentences are very similar—for example, "I like apples" and "I don't like apples" differ only by the word "not"—the two sentences convey diametrically opposed meanings. Therefore, the sentence similarity calculated using these sentence vector models is inaccurate. Summary of the Invention

[0004] The embodiments of the present application provide a method for calculating sentence similarity, a related device, and a computer storage medium to solve the problem of inaccurate sentence similarity calculation in the prior art.

[0005] In a first aspect, an embodiment of the present application provides a method for calculating sentence similarity, the method comprising:

[0006] Obtain two sentences to be compared and a decay factor; wherein at least one of the sentences to be compared includes a negation word;

[0007] Vectorizing the phrases and negation words in the two sentences to be compared respectively to obtain word vectors for each phrase and negation word in the two sentences to be compared;

[0008] Determining the word vector similarity between the two sentences to be compared based on the word vector of each phrase and negation word in the two sentences to be compared;

[0009] Generate a dependency tree for each sentence to be compared according to preset dependency syntax rules;

[0010] Determine the sentence similarity between the two sentences to be compared based on the decay factor, the dependency tree of each sentence to be compared, and the word vector similarity between the two sentences to be compared.

[0011] In a second aspect, an embodiment of the present application provides a device for calculating sentence similarity, the device comprising:

[0012] An acquisition module, configured to acquire two sentences to be compared and a decay factor; wherein at least one of the sentences to be compared includes a negation word;

[0013] A word vector obtaining module is used to vectorize the phrases and negation words in the two sentences to be compared, respectively, to obtain a word vector for each phrase and negation word in the two sentences to be compared;

[0014] A word vector similarity determination module, configured to determine the word vector similarity between the two sentences to be compared based on the word vector of each phrase and negation word in the two sentences to be compared;

[0015] A generation module, configured to generate a dependency tree for each sentence to be compared according to a preset dependency syntax rule;

[0016] A sentence similarity determination module is used to determine the sentence similarity between the two sentences to be compared based on the decay factor, the dependency tree of each sentence to be compared, and the word vector similarity between the two sentences to be compared.

[0017] In a third aspect, an embodiment of the present application provides a computer storage medium, wherein the computer storage medium stores a plurality of instructions, wherein the instructions are suitable for being loaded by a processor and executing the above-mentioned method steps.

[0018] In a fourth aspect, an embodiment of the present application provides an electronic device, which may include: a processor and a memory;

[0019] The memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the above method steps.

[0020] The beneficial effects of the technical solutions provided by some embodiments of the present application include at least:

[0021] In an embodiment of the present application, the embodiment of the present application can obtain two sentences to be compared and a decay factor; respectively vectorize the phrases and negation words in the two sentences to be compared to obtain the word vectors of each phrase and negation word in the two sentences to be compared; determine the word vector similarity between the two sentences to be compared based on the word vectors of each phrase and negation word in the two sentences to be compared; generate a dependency tree for each sentence to be compared according to a preset dependency syntactic rule; determine the sentence similarity between the two sentences to be compared based on the decay factor, the dependency tree of each sentence to be compared, and the word vector similarity between the two sentences to be compared. Therefore, the present application further takes into account the similarity between the word vectors containing negation words on the basis of sentence structure, and more accurately grasps the semantic relationship between the various components of the sentence by combining the two, and does not lose the information contained in the sentence vector. In addition, the dependency tree in the embodiment of the present application can directly process the relationship between words in a sentence, and the number of nodes is greatly reduced, which is convenient for direct marking of parts of speech. It has the advantages of being concise and clear, especially in the automatic annotation of corpus text. It is more convenient to use than the syntactic structure tree, and is more accurate than the prior art that only calculates the similarity between sentences based on the dependency tree. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0023] Figure 1 A schematic diagram of a dependency syntactic relationship of a sentence to be detected provided in an embodiment of the present application;

[0024] Figure 2 A flowchart of a method for calculating sentence similarity provided in an embodiment of the present application;

[0025] Figure 3a A schematic diagram of a dependency syntactic relationship of a sentence to be detected provided in an embodiment of the present application;

[0026] Figure 3b for Figure 3a Schematic diagram of the dependency tree of the sentence to be detected;

[0027] Figure 4 A flowchart of another method for calculating sentence similarity provided in an embodiment of the present application;

[0028] Figure 5aA schematic diagram of a dependency tree of a sentence to be detected provided in an embodiment of the present application;

[0029] Figure 5b A schematic diagram of a dependency tree of another sentence to be detected provided in an embodiment of the present application;

[0030] Figure 6 A schematic diagram of the structure of a device for calculating sentence similarity provided in an embodiment of the present application;

[0031] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0032] When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Instead, they are merely examples of devices and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0033] In the description of this application, it should be understood that the terms "first", "second", etc. are used for descriptive purposes only and should not be understood as indicating or implying relative importance. For those of ordinary skill in the art, the specific meanings of the above terms in this application can be understood according to specific circumstances. In addition, in the description of this application, unless otherwise specified, "multiple" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the previous and subsequent associated objects are in an "or" relationship.

[0034] In natural language, sentences can be divided into subject, predicate, and object representations. The way people speak (even drunken gibberish) has structures and rules. The goal of grammatical analysis in linguistics is to isolate these grammatical structures. The relationship between words is more than a simple sequence; it must describe how the words are connected. A comprehensible approach is to construct a tree structure for each sentence.

[0035] Syntactic parsing is a key technology in Natural Language Processing (NLP) that can help analyze the constituent structure of a sentence and the relationships between words. Syntactic analysis trees are mainly divided into two categories: the constituent tree and the dependency tree. Specifically, the constituent tree is used to express the syntactic structure of a sentence. Only its leaf nodes are associated with words in the input sentence, and the other intermediate nodes are all labeled phrase components. The dependency tree is used to express the dependency relationship between words in a sentence and is used to analyze and identify grammatical components such as "subject, predicate, object", "attributive, adverbial, complement" in a sentence. Each node is a word. The dependency tree is used to express the tree structure of the constituent words in a sentence and the dependency relationship between words. Compared with the grammar of the constituent tree, the dependency grammar does not have the phrase level. Each node corresponds to a word in the sentence.

[0036] It's understandable that dependency syntax considers the verb in the "predicate" to be the center of a sentence, with other components directly or indirectly connected to the verb. In dependency syntax theory, "dependency" refers to the dominant-subordinate relationship between words. This relationship is not reciprocal but rather directional. Specifically, the dominant component is called the dominant, while the subordinate component is called the subordinate. Dependency grammar itself does not require a categorization of dependency relationships. However, to enrich the syntactic information conveyed by dependency structures, in practical applications, different labels are generally assigned to the edges of dependency trees. In the syntax related to dependency trees, the common fundamental assumption is that syntactic structure essentially consists of relationships between words and word pairs. This relationship is called a dependency relationship. A dependency relationship connects two words: one is a core word, and the other is a modifier.

[0037] See also Figure 1 The following diagram shows the dependency syntax for the sentence "Xiao Wang buys a computer." The diagram shows a dependency relationship HED (core relationship) between Root (core word) and buy, indicating that "buy" is the core word in this sentence. Furthermore, a dependency relationship SBV (subject-verb relationship) exists between Xiao Wang (dependent word) and buy (core word), and a dependency relationship VOB (verb-object relationship) exists between buy (core word) and computer (dependent word).

[0038] Next, combine Figure 1 The method for calculating sentence similarity provided in an embodiment of the present application is introduced using a schematic diagram of a dependency tree in the related art.

[0039] S201, obtaining two sentences to be compared and a decay factor.

[0040] Specifically, in the embodiment of the present application, at least one sentence to be compared may include a negative word.

[0041] The decay factor is used to represent a preset coefficient, which is between 0 and 1. Specifically, the decay factor can be set according to user needs, for example, it can be determined according to the length of the sentence or the depth of the dependency tree.

[0042] S202 , respectively vectorizing the phrases and negation words in the two sentences to be compared to obtain a word vector for each phrase and negation word in the two sentences to be compared.

[0043] Among them, word vectors are used to represent vectors in which words or phrases from the vocabulary are mapped to real numbers.

[0044] Possibly, the embodiments of the present application can obtain word vectors through a long short-term memory model (Long Short-Term Memory, LSTM), a bidirectional long short-term memory model (Bi-directional Long Short-Term Memory, BiLSTM), etc.

[0045] Specifically, the phrases in the sentence are replaced with word vectors according to the second dimension (the index of each word) before entering the LSTM, so the input of the LSTM can be a 3-dimensional vector (number of samples x sentence length x word vector dimension).

[0046] Specifically, the BiLSTM model can be composed of four parts: input layer: used to receive input sentences, which for Chinese refers to the words that have been divided into sentences; embedding layer: maps each word in the sentence into a vector of fixed length; long short-term memory layer: uses a bidirectional long short-term memory network (Long Short-Term Memory, LSTM) to calculate the vectors in the embedding layer. In fact, the bidirectional LSTM calculates the word vectors to obtain higher-level sentence vectors; output layer: used to output the word vector results corresponding to the words.

[0047] S203 : Determine the word vector similarity between the two sentences to be compared based on the word vector of each phrase and negation word in the two sentences to be compared.

[0048] It can be understood that this application can construct a Euclidean space by converting text into word vectors. Comparing the word vector similarity between two sentences is converted into measuring the distance between word vectors. There are many indicators to choose from, such as Euclidean distance, Pearson coefficient, and cosine distance.

[0049] Specifically, the vectors of the two words are A and B, and the word vector similarity between them is This is the cosine of the angle between two vectors, and its value range is [-1, 1]. When a pair of texts differ significantly in length but have similar content, their Euclidean distance in feature space is typically large if word frequency or word vectors are used as features. However, if cosine similarity is used, the angle between them may be small, resulting in high similarity. It is understandable that cosine distance often has high feature dimensionality in objects such as text. Cosine similarity maintains the property of "1 for identical, 0 for orthogonal, and -1 for opposite" in high dimensions. However, the Euclidean distance is affected by dimensionality, has an unfixed range, and is more ambiguous in meaning.

[0050] Possibly, the similarity VSP between the BiLSTM word vector "1011" of the two sentences to be compared, "I eat" and the BiLSTM word vector "1111" of "I don't eat", can be 1011·1111=3.

[0051] S204: Generate a dependency tree for each sentence to be compared according to preset dependency syntax rules.

[0052] Among them, the dependency syntax rules preset in the embodiment of the present application are used to represent the rules for generating the dependency tree of the sentences to be compared.

[0053] Possibly, embodiments of the present application can use a deep learning-based method to establish a dependency tree for the sentences to be compared. Specifically, deep learning is to vectorize atomic features (words, parts of speech, category labels) and then use a multi-layer neural network to extract features. Among them, the deep learning method is based on large-scale manually annotated data and is based on massive data training to make text matching more accurate.

[0054] Specifically, embodiments of the present application can mine sentences based on graph-relationship triples, i.e., through dependency grammar analysis, a large number of triple relationships, such as subject, predicate, and object, can be constructed. For example, a graph-based dependency parser aims to find a maximum spanning tree to obtain a globally optimal solution for the overall dependency structure of a sentence.

[0055] In addition, the embodiment of the present application can also perform operations such as word segmentation on Chinese sentences, filtering stop words, and removing meaningless words based on the word segmentation model in the Language Technology Platform (LTP). Based on the word segmentation results, the part-of-speech tagging model in the LTP is used to tag the words after word segmentation. Based on the results of the part-of-speech tagging, the dependency syntax analysis model in the LTP is used to perform dependency syntax analysis on the sentences after the part-of-speech tagging to obtain a semantic dependency tree.

[0056] Possibly, each node in the semantic dependency tree in the embodiments of the present application may include: the content in the sentence to be compared corresponding to the node, the node type, and the node attributes. Specifically, the node type may include: the core word, the first dependent word, and the second dependent word; where the first dependent word is used to represent the dependent word of the core word, and the second dependent word is used to represent the dependent word of the dependent word; the node attributes may include: the nominal subject, the word to be supplemented and explained, and the direct object.

[0057] See Figure 3a the syntactic dependency relationship of the sentence "I don't like picking apples" in Figure 3a it can be observed that: there is a dependency relationship HED (core relationship) between ROOT (core word) and "like", there is a dependency relationship SBV (subject-predicate relationship) between "I" (dependent word) and "like" (core word), there is a dependency relationship ATT (modifier-head relationship) between "not" (first dependent word) and "like" (core word), there is a dependency relationship CMP (verb-complement relationship) between "like" (core word) and "pick" (first dependent word), and there is a relationship VOB (verb-object relationship) between "pick" (first dependent word) and "apples" (second dependent word).

[0058] See Figure 3b the dependency tree of the sentence "I don't like picking apples" in Figure 3b it can be observed that: a dependency tree with "like" as the (root) core word. Among them, the content in the sentence corresponding to the ROOT node is the whole sentence: "I don't like picking apples", the node type is the root node, and the node attribute is the clause. The node content corresponding to the first node below the ROOT node is "I", the node type is the first dependent word node, and the node attribute is the nominal subject nsubj. The node content corresponding to the second node below the ROOT node is "not", the node type is the modifier-head relationship, and the node attribute is the negation word neg. The node content corresponding to the third node below the ROOT node is "like", the node type is the core word node, and the node attribute is the verb. The node content corresponding to the fourth node below the ROOT node is "pick", the node type is the first dependent word node, and the node attribute is the main component ccomp of the complement clause. The node content corresponding to the fifth node below the ROOT node is "apples", the node type is the second dependent word node, and the node attribute is the direct object dobj.

[0059] S205. Determine the sentence similarity between two sentences to be compared based on the decay factor, the dependency tree of each sentence to be compared, and the word vector similarity between the two sentences to be compared.

[0060] It can be understood that the embodiments of the present application can use the decay factor to process each node in the dependency tree of each sentence to be compared, so as to reduce the redundancy phenomenon caused by repeated calculation of nodes due to too deep dependency trees.

[0061] Specifically, the embodiment of the present application can determine the sentence similarity between the two sentences to be compared by combining the similarity between the dependency tree nodes of the two sentences to be compared with the similarity of the word vectors between the two sentences to be compared.

[0062] Alternatively, the embodiment of the present application may calculate the similarity E between two sentences to be compared using the following formula:

[0063]

[0064] Among them, K(T1, T2) represents the comparison value between the two sentences to be compared, which may include the similarity between the dependency tree nodes of the two sentences to be compared and the similarity of the word vectors between the two sentences to be compared; K(T1, T1) represents the comparison value of the first sentence to be compared itself, which may include the similarity between the dependency tree nodes of the first sentence to be compared itself and the similarity of the word vectors of the dependency tree nodes of the first sentence to be compared itself; K(T2, T2) represents the comparison value of the second sentence to be compared itself, which may include the similarity between the dependency tree nodes of the second sentence to be compared itself and the similarity of the word vectors of the dependency tree nodes of the second sentence to be compared itself.

[0065] Thus, the embodiment of the present application can obtain two sentences to be compared and a decay factor; respectively vectorize the phrases and negation words in the two sentences to be compared to obtain the word vectors of each phrase and negation word in the two sentences to be compared; determine the word vector similarity between the two sentences to be compared based on the word vectors of each phrase and negation word in the two sentences to be compared; generate a dependency tree for each sentence to be compared according to a preset dependency syntactic rule; determine the sentence similarity between the two sentences to be compared based on the decay factor, the dependency tree of each sentence to be compared, and the word vector similarity between the two sentences to be compared. Thus, the present application further considers the similarity between the word vectors containing negation words on the basis of sentence structure, and more accurately grasps the semantic relationship between the various components of the sentence by combining the two, without losing the information contained in the sentence vector. In addition, the dependency tree in the embodiment of the present application can directly process the relationship between words in a sentence, and the number of nodes is greatly reduced, which is convenient for direct marking of parts of speech. It has the advantages of being concise and clear, especially in the automatic annotation of corpus text. It is more convenient to use than the syntactic structure tree, and is more accurate than the prior art that only calculates the similarity between sentences based on the dependency tree.

[0066] In some embodiments, Figure 4The following is a flow chart showing a method for calculating sentence similarity provided by an embodiment of the present application. Figure 4 As shown, the method for calculating sentence similarity may include at least the following steps:

[0067] S401: Obtain two sentences to be compared and a decay factor.

[0068] Specifically, S401 is consistent with S201 and will not be repeated here.

[0069] S402 , respectively vectorizing the phrases and negation words in the two sentences to be compared to obtain a word vector for each phrase and negation word in the two sentences to be compared.

[0070] Specifically, S402 is consistent with S202 and will not be repeated here.

[0071] S403 : Determine the word vector similarity between the two sentences to be compared based on the word vector of each phrase and negation word in the two sentences to be compared.

[0072] Specifically, S403 is consistent with S203 and will not be repeated here.

[0073] S404: Generate a dependency tree for each sentence to be compared according to preset dependency syntax rules.

[0074] Specifically, S404 is consistent with S204 and will not be repeated here.

[0075] S405, traverse the dependency trees of the two sentences to be compared. If the dependency trees of the two sentences to be compared are generated in the same way, execute S406; if the dependency trees of the two sentences to be compared are generated in different ways, execute S407-S410.

[0076] It can be understood that traversal means starting from the root node and following a search route, visiting each node in the tree in turn to determine the node content, node type, and node attributes of each node.

[0077] Specifically, the embodiment of the present application can traverse the dependency tree of two sentences to be compared from top to bottom.

[0078] S406 , when the dependency trees of the two sentences to be compared are generated in the same manner, determining the sentence similarity between the two sentences to be compared based on the decay factor, the dependency tree of each sentence to be compared, and the word vector similarity between the two sentences to be compared.

[0079] It can be understood that the same generation method in the embodiment of the present application means that the node types and node attributes of each level in the dependency tree are the same.

[0080] See also Figure 3b The dependency tree for "I don't like picking apples" is shown below and Figure 5a The dependency tree of "I don't like eating apples" shown in Figure 3b and Figure 5a The content of the nodes is different, but what is certain is that Figure 3b and Figure 5a The dependency tree in is generated in the same way, that is, the node attributes and node types are consistent.

[0081] Furthermore, embodiments of the present application can determine the comparison value of each sentence to be compared based on the decay factor, the dependency tree of each sentence to be compared, and the word vector similarity of each sentence to be compared. The comparison value between the two sentences to be compared is determined based on the number of nodes at each level in the dependency tree of the two sentences to be compared, the content similarity of nodes of the same node type and node attribute, and the word vector similarity between the two sentences to be compared. The sentence similarity between the two sentences to be compared is determined based on the comparison value of each sentence to be compared and the comparison value between the two sentences to be compared.

[0082] Specifically, the embodiment of the present application can determine the comparison value of each sentence to be compared based on the decay factor, the number of node types at each level in the dependency tree of each sentence to be compared, and the word vector similarity of each sentence to be compared.

[0083] Possibly, the embodiment of the present application may use the following formula to calculate the comparison value of each sentence to be compared:

[0084]

[0085] Among them, T1 represents the dependency tree of the first sentence to be compared, N1 represents the node set of the dependency tree of the first sentence to be compared, n1 represents the node in the dependency tree of the first sentence to be compared, #ch(n1) represents the number of first dependency word nodes with second dependency word nodes, ch(n1,k) is the kth first dependency word node with a second dependency word node, VSP is the similarity between the word vector obtained by the BiLSTM model of ch(n1,k) and the word vector obtained by the BiLSTM model of ch(n1,k), and λ(0<λ<1) is the decay factor.

[0086] Possibly, an embodiment of the present application can determine the content similarity of nodes in each level based on the word vectors and decay factors corresponding to the content of the nodes in each level; determine the comparison value between the two sentences to be compared based on the number of nodes in each level in the dependency tree of the two sentences to be compared, the content similarity of nodes with the same node type and node attributes, and the word vector similarity between the two sentences to be compared.

[0087] Possibly, the embodiment of the present application may use the following formula to calculate the comparison value between two sentences to be compared:

[0088]

[0089] Wherein, T1 represents the dependency tree of the first sentence to be compared, T2 represents the dependency tree of the second sentence to be compared, N1 represents the node set of the dependency tree of the first sentence to be compared, N2 represents the node set of the dependency tree of the second sentence to be compared, n1 represents the node in the dependency tree of the first sentence to be compared, n2 represents the node in the dependency tree of the second sentence to be compared, #ch(n) represents the number of first dependency word nodes and second dependency word nodes with the same connection relationship in the two trees, ch(n1,k) is the kth first dependency word node with the second dependency word node in the dependency tree of the first sentence to be compared, ch(n2,k) is the kth first dependency word node with the second dependency word node in the dependency tree of the second sentence to be compared, VSP is the similarity between the word vector obtained by the BiLSTM model of ch(n1,k) and the word vector obtained by the BiLSTM model of ch(n2,k), and λ(0<λ<1) is the decay factor.

[0090] Specifically, the content similarity of the nodes at each level of the dependency tree of the two sentences to be compared in the embodiment of the present application can be obtained by calculating λVS, where VS is the similarity of the BiLSTM word vectors of the nodes.

[0091] For a specific example, see Figure 3b and Figure 5aAssuming the decay factor λ = 0.5, the first sentence to be compared is "I don't like picking apples", and the second sentence to be compared is "I don't like eating apples". First, according to the above formula, the comparison value of each sentence to be compared can be calculated separately. Starting from the root node of the dependency tree, traversing the entire dependency tree, we get: 1 root node (ROOT), 1 core word node (root), 3 first dependency nodes (nsubj, neg, ccomp), 1 second dependency word node (dobj), and a total of 6 nodes; traversing the node in the second level (ccomp), we get: 1 first dependency node (ccomp), 1 second dependency word node (dobj), and a total of 2 nodes. Further, assuming that the first sentence to be compared is "I don't like picking apples" and its word vector is 11101, and the second sentence to be compared is "I don't like eating apples" and its word vector is 11111, then the similarity of the leaf nodes at each level in the syntactic binary tree of the first sentence to be compared can be obtained according to λVS: 0.5*(1+1+1+1)=2, and the word vector similarity VSP of the first sentence to be compared is VSP=1+1+1+1=4; the similarity of the leaf nodes at each level in the second syntactic binary tree can be obtained according to λVS: 0.5*(1+1+1+1+1)=2.5, and the word vector similarity VSP of the second sentence to be compared is VSP=1+1+1+1+1=5; the similarity of the leaf nodes at each level in the two syntactic binary trees can be obtained according to λVS: 0.5*(1+1+1+1)=2; the word vector similarity VSP of the two sentences to be compared is VSP=1+1+1+1=4. Thus, the comparison value K(T1, T1) of the first sentence to be compared is 0.5*(6+2+2+4)=7; the comparison value K(T2, T2) of the second sentence to be compared is 0.5*(6+2+2.5+5)=7.75; the comparison value K(T1, T2) between the two sentences to be compared is [0.5*(6+2+2+4)] 2 =7 2 =49; further, the similarity between the two sentences to be compared can be determined

[0092] Therefore, the present application can compare the node similarity between the components in the two sentences based on the dependency tree, and further determine the similarity between the two sentences to be compared based on the similarity between similar words. In this way, the information contained in the sentence vector is not lost and the relationship between the dependency structures is compared, making the similarity between the sentences more accurate.

[0093] S407 : When the dependency trees of the two sentences to be compared are generated in different ways, determine a first dependent word and a second dependent word in the dependency trees of the two sentences to be compared that have the same node type and node attributes and have a mutual dependency relationship.

[0094] It is understandable that the different generation methods in the embodiments of the present application refer to the differences in the node types and / or node attributes at each level in the dependency tree.

[0095] See Figure 5a the dependency tree of the sentence "I don't like to eat apples" to be compared shown in Figure 5b and Figure 5a and Figure 5b the dependency tree of the sentence "I like to eat apples" to be compared shown in. It can be observed from Figure 5a and Figure 5b that the third node types and node attributes from top to bottom are different. Therefore, the generation methods of the dependency trees of these two sentences to be compared are different. And from Figure 5a and Figure 5b it can be observed that the first dependent word and the second dependent word with the same node types and node attributes and having a mutual dependency relationship in the dependency trees of the two sentences to be compared are the two nodes "eat" and "apple".

[0096] S408. Determine the comparison value of each sentence to be compared based on the decay factor, the number of node types at each level in the dependency tree of each sentence to be compared, and the word vector similarity of each sentence to be compared itself.

[0097] Possibly, the embodiments of the present application can calculate the comparison value of each sentence to be compared using the following formula:

[0098]

[0099]

[0100] where, T1 represents the dependency tree of the first sentence to be compared, N1 represents the node set of the dependency tree of the first sentence to be compared, n1 represents a node in the dependency tree of the first sentence to be compared, #ch(n1) represents the number of first dependent word nodes with second dependent word nodes, ch(n1,k) is the kth first dependent word node with a second dependent word node, VSP is the similarity between the word vector obtained by the BiLSTM model of ch(n1,k) and the word vector obtained by the BiLSTM model of ch(n1,k), and λ(0 < λ < 1) is the decay factor.

[0101] S409. Determine the comparison value between the two sentences to be compared based on the number of the first dependent word and the second dependent word with the same node attributes and node types and having a mutual dependency relationship, the content similarity of each node, the word vector similarity between the two sentences to be compared, and the number of the same nodes in the dependency trees of the two sentences to be compared.

[0102] Specifically, embodiments of the present application can determine the node content similarity between two sentences to be compared based on the number of the first dependent word and the second dependent word having the same node attributes and having a mutual dependence relationship, the content similarity of each node, and the word vector similarity between the two sentences to be compared; determine the number of identical nodes in the dependence relationship trees of the two sentences to be compared after removing the negative words in the two sentences to be compared; and determine the comparison value between the two sentences to be compared based on the difference between the node content similarity and the number of identical nodes, and the word vector similarity between the two sentences to be compared.

[0103] Possibly, embodiments of the present application can calculate the comparison value between two sentences to be compared using the following formula:

[0104]

[0105] where, T1 represents the dependence relationship tree of the first sentence to be compared, T2 represents the dependence relationship tree of the second sentence to be compared, N1 represents the node set of the dependence relationship tree of the first sentence to be compared, N2 represents the node set of the dependence relationship tree of the second sentence to be compared, n1 represents a node in the dependence relationship tree of the first sentence to be compared, n2 represents a node in the dependence relationship tree of the second sentence to be compared, #ch(n) represents the number of the first dependent word node and the second dependent word node having the same connection relationship in the two trees, ch(n1,k) is the k-th first dependent word node having a second dependent word node in the dependence relationship tree of the first sentence to be compared, ch(n2,k) is the k-th first dependent word node having a second dependent word node in the dependence relationship tree of the second sentence to be compared, VSP is the similarity between the word vector obtained by the BiLSTM model of ch(n1,k) and the word vector obtained by the BiLSTM model of ch(n2,k), and λ(0 < λ < 1) is a decay factor.

[0106] Possibly, the content similarity of each node in the dependence relationship trees of the two sentences to be compared in embodiments of the present application can be obtained by calculating λVS. Where, VS is the similarity of the BiLSTM word vectors of the nodes.

[0107] It can be understood that after removing Figure 5a the negative word "not" in the sentence to be compared "I don't like to eat apples" shown, Figure 5b the same sentence to be compared "I like to eat apples" can be obtained, that is, the number of identical nodes is 5.

[0108] S410. Determine the sentence similarity between the two sentences to be compared based on the comparison value of each sentence to be compared itself and the comparison value between the two sentences to be compared.

[0109] For a specific example, see Figure 5a and Figure 5bAssuming the decay factor λ = 0.5, the first sentence to be compared is "I like to eat apples", and the second sentence to be compared is "I don't like to eat apples". First, according to the above formula, the comparison value of each sentence to be compared can be calculated separately. Starting from the dependency tree of the first sentence to be compared, traversing the entire dependency tree, we get: 1 root node (ROOT), 1 core word node (root), 2 first dependency nodes (subj, ccomp), 1 second dependency node (dobj), and a total of 5 nodes; traversing the nodes of the second level: 2 first dependency nodes (ccomp), 1 second dependency node (dobj), and a total of 2 nodes. Starting from the dependency tree of the second sentence to be compared, traversing the entire dependency tree, we get: 1 root node (ROOT), 1 core word node (root), 3 first dependency nodes (subj, neg, ccomp), 1 second dependency node (dobj), and a total of 6 nodes; traversing the nodes of the second level: 1 first dependency node (ccomp), 1 second dependency node (dobj), and a total of 2 nodes. Furthermore, assuming that the first sentence to be compared is "I like to eat apples" with a word vector of 11011, and the second sentence to be compared is "I don't like to eat apples" with a word vector of 11111, the similarity of each node in the dependency tree of the first sentence to be compared can be obtained according to λVS: 0.5*(1+1+1+1)=2, and the word vector similarity of the first sentence to be compared VSP=1+1+1+1=4; the similarity of each node in the second dependency tree can be obtained according to λVS: 0.5*(1+1+1+1+1)=2.5, and the word vector similarity of the second sentence to be compared VSP=1+1+1+1=4. The word vector similarity VSP = 1+1+1+1+1 = 5. Since the second sentence to be compared removes the negation word "not", the corresponding identical node contents are "I", "like", "eat", and "apple". That is, traversing the entire dependency tree can obtain 5 nodes: 1 root node (ROOT), 1 core word node (root), 2 first dependency nodes (subj, ccomp), and 1 second dependency node (dobj); traversing the second level can obtain 2 nodes: 1 first dependency node (ccomp) and 1 second dependency node (dobj). Therefore, the similarity of each node in the two dependency trees can be obtained according to λVS: 0.5*(1+1+1+1)=2, and the word vector similarity VSP of the two sentences to be compared is VSP=1+1+1+1=4. In the dependency trees of the two sentences to be compared, the first dependent word and the second dependent word with the same node type and node attributes and mutual dependence are two nodes, "eat" and "apple", and the number of the same nodes in the dependency trees of the two sentences to be compared is 5.Therefore, the comparison value K(T1, T1) of the first sentence to be compared is 0.5*(5+2+4)=5.5; the comparison value K(T2, T2) of the second sentence to be compared is 0.5*(6+2+2.5+5)=7.75; the comparison value K(T1, T2) between the two sentences to be compared is [0.5*(2+2+4-5)]. 2 =1.5 2 =2.25; further, the similarity between the two sentences to be compared can be determined (5.5*7.75)=0.05.

[0110] Thus, the embodiment of the present application can determine the similarity between two sentences to be compared by determining the same node attributes and node types in two sentences to be compared and the presence of the first dependency word and the second dependency word node of the mutual dependence relationship and removing the negation word in the negation sentence. In this way, not only the similarity of the same nodes is compared, but also the word vectors of the two sentences to be compared are compared, and it can be observed that when the generation mode of the dependency tree is different and there is a negation word, the similarity between the two sentences to be compared obviously decreases. Therefore, it can be determined that the embodiment of the present application can significantly improve the accuracy of the similarity between the comparative sentences.

[0111] Figure 6 This is a schematic diagram of a device for calculating sentence similarity provided by an exemplary embodiment of the present application. The device for calculating sentence similarity can be set in a device such as a server to execute the method for calculating sentence similarity of any of the above embodiments of the present application. Figure 6 As shown, the device for calculating sentence similarity may include:

[0112] An acquisition module 61 is configured to acquire two sentences to be compared and a decay factor; wherein at least one of the sentences to be compared includes a negation word;

[0113] A word vector obtaining module 62 is used to vectorize the phrases and negation words in the two sentences to be compared, respectively, to obtain a word vector for each phrase and negation word in the two sentences to be compared;

[0114] A word vector similarity determination module 63 is configured to determine the word vector similarity between the two sentences to be compared based on the word vector of each phrase and negation word in the two sentences to be compared;

[0115] A generating module 64, configured to generate a dependency tree for each sentence to be compared according to a preset dependency syntax rule;

[0116] The sentence similarity determination module 65 is configured to determine the sentence similarity between the two sentences to be compared based on the decay factor, the dependency tree of each sentence to be compared, and the word vector similarity between the two sentences to be compared.

[0117] Thus, the embodiment of the present application can obtain two sentences to be compared and a decay factor; respectively vectorize the phrases and negation words in the two sentences to be compared to obtain the word vectors of each phrase and negation word in the two sentences to be compared; determine the word vector similarity between the two sentences to be compared based on the word vectors of each phrase and negation word in the two sentences to be compared; generate a dependency tree for each sentence to be compared according to a preset dependency syntactic rule; determine the sentence similarity between the two sentences to be compared based on the decay factor, the dependency tree of each sentence to be compared, and the word vector similarity between the two sentences to be compared. Thus, the present application further considers the similarity between the word vectors containing negation words on the basis of sentence structure, and more accurately grasps the semantic relationship between the various components of the sentence by combining the two, without losing the information contained in the sentence vector. In addition, the dependency tree in the embodiment of the present application can directly process the relationship between words in a sentence, and the number of nodes is greatly reduced, which is convenient for direct marking of parts of speech. It has the advantages of being concise and clear, especially in the automatic annotation of corpus text. It is more convenient to use than the syntactic structure tree, and is more accurate than the prior art that only calculates the similarity between sentences based on the dependency tree.

[0118] In some embodiments, each node in the dependency tree includes: the content of the sentence to be compared corresponding to the node, the node type, and the node attributes; wherein,

[0119] The node type includes: a core word, a first dependent word, and a second dependent word; wherein the first dependent word is used to represent a dependent word of the core word, and the second dependent word is used to represent a dependent word of the dependent word;

[0120] The node attributes include: a noun subject, a supplementary word, and a direct object.

[0121] In some implementations, the sentence similarity determination module includes:

[0122] A first traversal unit, configured to traverse the dependency tree of the two sentences to be compared;

[0123] The first determination unit is used to determine the sentence similarity between the two sentences to be compared based on the decay factor, the dependency tree of each sentence to be compared, and the word vector similarity between the two sentences to be compared, when the dependency trees of the two sentences to be compared are generated in the same way.

[0124] In some embodiments, the first determining unit includes:

[0125] A first determining subunit, configured to determine a comparison value of each sentence to be compared based on the decay factor, the dependency tree of each sentence to be compared, and the word vector similarity of each sentence to be compared;

[0126] a second determining subunit, configured to determine a comparison value between the two sentences to be compared based on content similarity of nodes of the same node type and node attributes in the dependency trees of the two sentences to be compared, and word vector similarity between the two sentences to be compared;

[0127] The third determining subunit is configured to determine the sentence similarity between the two sentences to be compared based on the comparison value of each sentence to be compared and the comparison value between the two sentences to be compared.

[0128] In some embodiments, the second determination subunit is specifically used to: determine the content similarity of the nodes in each level based on the word vectors and decay factors corresponding to the contents of the nodes in each level; determine the comparison value between the two sentences to be compared based on the number of nodes in each level in the dependency tree of the two sentences to be compared, the content similarity of nodes with the same node type and node attributes, and the word vector similarity between the two sentences to be compared.

[0129] In some implementations, the sentence similarity determination module includes:

[0130] A second traversal unit, configured to traverse the dependency tree of the two sentences to be compared;

[0131] a fourth determining subunit, configured to determine, when the dependency trees of the two to-be-compared sentences are generated in different ways, a first dependent word and a second dependent word that have the same node attributes and node types and have a mutual dependency relationship in the dependency trees of the two to-be-compared sentences;

[0132] a fifth determining subunit, configured to determine a comparison value of each sentence to be compared based on the decay factor, the number of node types at each level in the dependency tree of each sentence to be compared, and the word vector similarity of each sentence to be compared;

[0133] a sixth determining subunit, configured to determine a comparison value between the two sentences to be compared based on the number of the first dependent words and the second dependent words that have the same node attributes and node types and are in a mutual dependency relationship, the content similarity of the nodes, the word vector similarity between the two sentences to be compared, and the number of identical nodes in the dependency trees of the two sentences to be compared;

[0134] The seventh determining subunit is configured to determine the sentence similarity between the two sentences to be compared based on the comparison value of each sentence to be compared and the comparison value between the two sentences to be compared.

[0135] In some implementations, the sixth determining subunit is specifically configured to:

[0136] Based on the word vector of each sentence to be compared and the decay factor, the content similarity of each node is determined; based on the number of first dependent words and second dependent words that have the same node attributes and are mutually dependent, the content similarity of each node, and the word vector similarity between the two sentences to be compared, the node content similarity between the two sentences to be compared is determined; after removing the negative words in the two sentences to be compared, the number of identical nodes in the dependency tree of the two sentences to be compared is determined; based on the difference between the node content similarity and the number of identical nodes, and the word vector similarity between the two sentences to be compared, the comparison value between the two sentences to be compared is determined.

[0137] In some embodiments, the first determining subunit or the fifth determining subunit is specifically configured to:

[0138] The comparison value of each sentence to be compared is determined based on the decay factor, the number of node types in the dependency tree of each sentence to be compared, and the word vector similarity of each sentence to be compared.

[0139] It should be noted that the apparatus for calculating sentence similarity provided in the above embodiment only uses the division of the above functional modules as an example when executing the method for calculating sentence similarity. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus for calculating sentence similarity provided in the above embodiment and the embodiment of the method for calculating sentence similarity belong to the same concept. The implementation process thereof is detailed in the method embodiment and will not be repeated here.

[0140] The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0141] See Figure 7 , is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 7 As shown, the electronic device 70 may include: at least one processor 71 , at least one network interface 74 , a user interface 73 , a memory 75 , and at least one communication bus 72 .

[0142] The communication bus 72 is used to realize the connection and communication between these components.

[0143] The user interface 73 may include a display screen (Display) and a camera (Camera). Optionally, the user interface 73 may also include a standard wired interface and a wireless interface.

[0144] The network interface 74 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0145] The processor 71 may include one or more processing cores. The processor 71 utilizes various interfaces and circuits to connect various components within the electronic device 70. It executes instructions, programs, code sets, or instruction sets stored in the memory 75, and accesses data stored in the memory 75 to perform various functions and process data within the electronic device 70. Optionally, the processor 71 may be implemented using at least one of the following hardware forms: a digital signal processing (DSP), a field-programmable gate array (FPGA), or a programmable logic array (PLA). The processor 71 may integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. The CPU primarily processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing the content displayed on the display; and the modem handles wireless communications. It is understood that the modem may not be integrated into the processor 71 but implemented as a separate chip.

[0146] Among them, the memory 75 may include a random access memory (RAM) or a read-only memory (Read-Only Memory). Optionally, the memory 75 includes a non-transitory computer-readable storage medium. The memory 75 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 75 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 75 may also be optionally at least one storage device located away from the aforementioned processor 71. As Figure 7 As shown, the memory 85 as a computer storage medium may include an operating system, a network communication module, a user interface module, and an application program for calculating sentence similarity.

[0147] exist Figure 7 In the electronic device 70 shown, the user interface 73 is mainly used to provide an input interface for the user and obtain data input by the user; and the processor 71 can be used to call the sentence similarity calculation application stored in the memory 75 and specifically perform the following operations:

[0148] Obtain two sentences to be compared and a decay factor; wherein at least one of the sentences to be compared includes a negation word;

[0149] Vectorizing the phrases and negation words in the two sentences to be compared respectively to obtain word vectors for each phrase and negation word in the two sentences to be compared;

[0150] Determining the word vector similarity between the two sentences to be compared based on the word vector of each phrase and negation word in the two sentences to be compared;

[0151] Generate a dependency tree for each sentence to be compared according to preset dependency syntax rules;

[0152] Determine the sentence similarity between the two sentences to be compared based on the decay factor, the dependency tree of each sentence to be compared, and the word vector similarity between the two sentences to be compared.

[0153] In some embodiments, each node in the dependency tree includes: the content of the sentence to be compared corresponding to the node, the node type, and the node attributes; wherein,

[0154] The node type includes: a core word, a first dependent word, and a second dependent word; wherein the first dependent word is used to represent a dependent word of the core word, and the second dependent word is used to represent a dependent word of the dependent word;

[0155] The node attributes include: a noun subject, a supplementary word, and a direct object.

[0156] In some implementations, when the processor 71 determines the sentence similarity between the two sentences to be compared based on the decay factor, the dependency tree of each sentence to be compared, and the word vector similarity between the two sentences to be compared, specifically performs:

[0157] Traversing the dependency tree of the two sentences to be compared;

[0158] When the dependency trees of the two sentences to be compared are generated in the same manner, the sentence similarity between the two sentences to be compared is determined based on the decay factor, the dependency tree of each sentence to be compared, and the word vector similarity between the two sentences to be compared.

[0159] In some implementations, when the processor 71 determines the sentence similarity between the two sentences to be compared based on the decay factor, the dependency tree of each sentence to be compared, and the word vector similarity between the two sentences to be compared when the dependency trees of the two sentences to be compared are generated in the same manner, the processor 71 specifically performs:

[0160] Determining a comparison value of each sentence to be compared based on the decay factor, the dependency tree of each sentence to be compared, and the word vector similarity of each sentence to be compared;

[0161] Determining a comparison value between the two sentences to be compared based on content similarity of nodes of the same node type and node attributes in the dependency trees of the two sentences to be compared, and word vector similarity between the two sentences to be compared;

[0162] The sentence similarity between the two sentences to be compared is determined based on the comparison value of each sentence to be compared and the comparison value between the two sentences to be compared.

[0163] In some embodiments, when the processor 71 determines the comparison value between the two sentences to be compared based on the content similarity of nodes of the same node type and node attributes in the dependency tree of the two sentences to be compared and the word vector similarity between the two sentences to be compared, the processor 71 specifically performs:

[0164] Determining the content similarity of the nodes in each level based on the word vectors and decay factors corresponding to the content of the nodes in each level;

[0165] The comparison value between the two sentences to be compared is determined based on the number of nodes at each level in the dependency tree of the two sentences to be compared, the content similarity of nodes with the same node type and node attributes, and the word vector similarity between the two sentences to be compared.

[0166] In some implementations, when the processor 71 determines the sentence similarity between the two sentences to be compared based on the dependency tree of each sentence to be compared and the word vector similarity between the two sentences to be compared, the processor 71 specifically performs:

[0167] Traversing the dependency tree of the two sentences to be compared;

[0168] When the dependency trees of the two sentences to be compared are generated in different ways, determining a first dependent word and a second dependent word in the dependency trees of the two sentences to be compared that have the same node attributes and node types and have a mutual dependency relationship;

[0169] Determining the comparison value of each sentence to be compared based on the decay factor, the number of node types at each level in the dependency tree of each sentence to be compared, and the word vector similarity of each sentence to be compared;

[0170] Determining a comparison value between the two sentences to be compared based on the number of the first dependent words and the second dependent words that have the same node attributes and node types and are in a mutual dependency relationship, the content similarity of the nodes, the word vector similarity between the two sentences to be compared, and the number of the same nodes in the dependency relationship trees of the two sentences to be compared;

[0171] The sentence similarity between the two sentences to be compared is determined based on the comparison value of each sentence to be compared and the comparison value between the two sentences to be compared.

[0172] In some embodiments, when determining the comparison value between the two sentences to be compared based on the number of the first dependent words and the second dependent words that have the same node attributes and node types and are mutually dependent, the content similarity of each node, the word vector similarity between the two sentences to be compared, and the number of identical nodes in the dependency trees of the two sentences to be compared, the processor 71 specifically performs:

[0173] Determining the content similarity of each node based on the word vector of each sentence to be compared and the decay factor;

[0174] Determine the node content similarity between the two sentences to be compared based on the number of the first dependent words and the second dependent words that have the same node attributes and are in a mutual dependence relationship, the content similarity of each node, and the word vector similarity between the two sentences to be compared;

[0175] Determining the number of identical nodes in the dependency trees of the two sentences to be compared after removing negation words from the two sentences to be compared;

[0176] Based on the difference between the node content similarity and the number of the identical nodes, and the word vector similarity between the two sentences to be compared, a comparison value between the two sentences to be compared is determined.

[0177] In some implementations, when the processor 71 determines the comparison value of each sentence to be compared based on the decay factor, the dependency tree of each sentence to be compared, and the word vector similarity of each sentence to be compared, the processor 71 specifically performs:

[0178] The comparison value of each sentence to be compared is determined based on the decay factor, the number of node types in the dependency tree of each sentence to be compared, and the word vector similarity of each sentence to be compared.

[0179] The present application also provides a computer-readable storage medium containing instructions that, when executed on a computer or processor, cause the computer or processor to perform one or more steps of the above-described embodiments. If the components of the apparatus for calculating sentence similarity are implemented as software functional units and sold or used as independent products, they may be stored in the computer-readable storage medium.

[0180] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When software is used for implementation, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted by the computer-readable storage medium. The computer instructions can be transmitted from a website, computer, server or data center to another website, computer, server or data center by wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrations. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a digital versatile disc (DVD)), or a semiconductor medium (eg, a solid state disk (SSD)).

[0181] Those skilled in the art will appreciate that all or part of the processes in the above-described embodiments can be implemented by instructing the relevant hardware through a computer program. The program can be stored in a computer-readable storage medium. When executed, the program can include the processes of the above-described embodiments. The aforementioned storage medium includes various media capable of storing program code, such as read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks. Unless there is a conflict, the technical features of this embodiment and the implementation scheme can be combined in any manner.

[0182] The embodiments described above are merely preferred embodiments of the present application and are not intended to limit the scope of the present application. Without departing from the design spirit of the present application, various modifications and improvements made to the technical solutions of the present application by ordinary technicians in this field should fall within the scope of protection determined by the claims of the present application.

Claims

1. A method for calculating sentence similarity, characterized in that: The method comprises: Obtain two sentences to be compared and a decay factor; wherein at least one of the sentences to be compared includes a negation word; Vectorizing the phrases and negation words in the two sentences to be compared respectively to obtain word vectors for each phrase and negation word in the two sentences to be compared; Determining the word vector similarity between the two sentences to be compared based on the word vector of each phrase and negation word in the two sentences to be compared; Generate a dependency tree for each sentence to be compared according to preset dependency syntax rules; Traversing the dependency tree of the two sentences to be compared; When the dependency trees of the two sentences to be compared are generated in the same manner, determining a comparison value of each sentence to be compared based on the decay factor, the dependency tree of each sentence to be compared, and the word vector similarity of each sentence to be compared; Determining a comparison value between the two sentences to be compared based on content similarity of nodes of the same node type and node attributes in the dependency trees of the two sentences to be compared, and word vector similarity between the two sentences to be compared; The sentence similarity between the two sentences to be compared is determined based on the comparison value of each sentence to be compared and the comparison value between the two sentences to be compared.

2. The method according to claim 1, characterized in that Each node in the dependency tree includes: the content of the sentence to be compared corresponding to the node, the node type, and the node attribute; wherein, The node type includes: a core word, a first dependent word, and a second dependent word; wherein the first dependent word is used to represent a dependent word of the core word, and the second dependent word is used to represent a dependent word of the dependent word; The node attributes include: a noun subject, a supplementary word, and a direct object.

3. The method according to claim 1, characterized in that Determining a comparison value between the two sentences to be compared based on content similarity of nodes of the same node type and node attributes in the dependency trees of the two sentences to be compared and word vector similarity between the two sentences to be compared includes: Determining the content similarity of the nodes in each level based on the word vectors and decay factors corresponding to the content of the nodes in each level; The comparison value between the two sentences to be compared is determined based on the number of nodes at each level in the dependency tree of the two sentences to be compared, the content similarity of nodes with the same node type and node attributes, and the word vector similarity between the two sentences to be compared.

4. The method according to claim 1, wherein The method further comprises: When the dependency trees of the two sentences to be compared are generated in different ways, determining a first dependent word and a second dependent word in the dependency trees of the two sentences to be compared that have the same node attributes and node types and have a mutual dependency relationship; Determining the comparison value of each sentence to be compared based on the decay factor, the number of node types at each level in the dependency tree of each sentence to be compared, and the word vector similarity of each sentence to be compared; Determining a comparison value between the two sentences to be compared based on the number of the first dependent words and the second dependent words that have the same node attributes and node types and are in a mutual dependency relationship, the content similarity of the nodes, the word vector similarity between the two sentences to be compared, and the number of the same nodes in the dependency relationship trees of the two sentences to be compared; The sentence similarity between the two sentences to be compared is determined based on the comparison value of each sentence to be compared and the comparison value between the two sentences to be compared.

5. The method according to claim 4, characterized in that The determining of a comparison value between the two sentences to be compared based on the number of the first dependent words and the second dependent words having the same node attributes and node types and having a mutual dependency relationship, the content similarity of each node, the word vector similarity between the two sentences to be compared, and the number of the same nodes in the dependency relationship trees of the two sentences to be compared includes: Determining the content similarity of each node based on the word vector of each sentence to be compared and the decay factor; Determine the node content similarity between the two sentences to be compared based on the number of the first dependent words and the second dependent words that have the same node attributes and are in a mutual dependence relationship, the content similarity of each node, and the word vector similarity between the two sentences to be compared; Determining the number of identical nodes in the dependency trees of the two sentences to be compared after removing negation words from the two sentences to be compared; Based on the difference between the node content similarity and the number of the identical nodes, and the word vector similarity between the two sentences to be compared, a comparison value between the two sentences to be compared is determined.

6. The method according to claim 1, characterized in that Determining the comparison value of each sentence to be compared based on the decay factor, the dependency tree of each sentence to be compared, and the word vector similarity of each sentence to be compared includes: The comparison value of each sentence to be compared is determined based on the decay factor, the number of node types in the dependency tree of each sentence to be compared, and the word vector similarity of each sentence to be compared.

7. A device for calculating sentence similarity, characterized in that: The device comprises: An acquisition module, configured to acquire two sentences to be compared and a decay factor; wherein at least one of the sentences to be compared includes a negation word; A word vector obtaining module is used to vectorize the phrases and negation words in the two sentences to be compared, respectively, to obtain a word vector for each phrase and negation word in the two sentences to be compared; A word vector similarity determination module, configured to determine the word vector similarity between the two sentences to be compared based on the word vector of each phrase and negation word in the two sentences to be compared; A generation module, configured to generate a dependency tree for each sentence to be compared according to a preset dependency syntax rule; A sentence similarity determination module is used to traverse the dependency trees of the two sentences to be compared; when the dependency trees of the two sentences to be compared are generated in the same manner, determine the comparison value of each sentence to be compared based on the decay factor, the dependency tree of each sentence to be compared, and the word vector similarity of each sentence to be compared; determine the comparison value between the two sentences to be compared based on the content similarity of nodes of the same node type and node attribute in the dependency trees of the two sentences to be compared, and the word vector similarity between the two sentences to be compared; and determine the sentence similarity between the two sentences to be compared based on the comparison value of each sentence to be compared and the comparison value between the two sentences to be compared.

8. A computer storage medium, characterized in that The computer storage medium stores a plurality of instructions, which are suitable for being loaded by a processor and executing the method steps according to any one of claims 1 to 6.

9. An electronic device, characterized in that: include: A processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the method steps according to any one of claims 1 to 6.

Images