Method for calculating sentence similarity, related device, and computer storage medium

Abstract

The present application discloses a method for calculating sentence similarity, a related device, and a computer storage medium, wherein the method includes: obtaining two sentences to be compared and a decay factor; vectorizing the phrases and negations in the two sentences to be compared respectively to obtain a word vector for each phrase and negation in the two sentences to be compared; determining the word vector similarity between the two sentences to be compared based on the word vector of each phrase and negation in the two sentences to be compared; generating a syntactic structure binary tree of each sentence to be compared according to a preset sentence structure division rule; and determining the sentence similarity between the two sentences to be compared based on the decay factor, the syntactic structure binary tree of each sentence to be compared, and the word vector similarity between the two sentences to be compared. Compared with the prior art method of calculating the similarity between sentences based only on the syntactic structure binary tree, the present application is more accurate.

Description

Technical Field

[0001] The present application relates to the field of computer language processing, and in particular to a method for calculating sentence similarity, a related device, and a computer storage medium. Background Art

[0002] As electronic products become increasingly intelligent, the demand for text recognition accuracy is also increasing. Existing technologies typically use sentence vector models to calculate text similarity, such as Onehot, Word2vector, BiLSTM, and Bert. The sentences to be compared are input into the model, and the output is sentence similarity. When the sentence vectors of the input sentences to be compared are very similar, such as "I like apples" and "I don't like eating apples," the output sentence similarity is also correspondingly high. However, in reality, the semantics of these two sentences are diametrically opposed. Summary of the Invention

[0003] The embodiments of the present application provide a method for calculating sentence similarity, a related device, and a computer storage medium to solve the problem of inaccurate sentence similarity calculation in the prior art.

[0004] In a first aspect, an embodiment of the present application provides a method for calculating sentence similarity, the method comprising:

[0005] Obtain two sentences to be compared and a decay factor; wherein at least one of the sentences to be compared includes a negation word;

[0006] Vectorizing the phrases and negation words in the two sentences to be compared respectively to obtain word vectors for each phrase and negation word in the two sentences to be compared;

[0007] Determining the word vector similarity between the two sentences to be compared based on the word vector of each phrase and negation word in the two sentences to be compared;

[0008] Generate a syntactic structure binary tree of each sentence to be compared according to a preset sentence structure division rule;

[0009] The sentence similarity between the two sentences to be compared is determined based on the decay factor, the syntactic structure binary tree of each sentence to be compared, and the word vector similarity between the two sentences to be compared.

[0010] In a second aspect, an embodiment of the present application provides a device for calculating sentence similarity, the device comprising:

[0011] An acquisition module, configured to acquire two sentences to be compared and a decay factor; wherein at least one of the sentences to be compared includes a negation word;

[0012] A word vector obtaining module is used to vectorize the phrases and negation words in the two sentences to be compared, respectively, to obtain a word vector for each phrase and negation word in the two sentences to be compared;

[0013] A word vector similarity determination module, configured to determine the word vector similarity between the two sentences to be compared based on the word vector of each phrase and negation word in the two sentences to be compared;

[0014] A generation module, configured to generate a binary tree of the syntactic structure of each sentence to be compared according to a preset sentence structure division rule;

[0015] A sentence similarity determination module is used to determine the sentence similarity between the two sentences to be compared based on the decay factor, the syntactic structure binary tree of each sentence to be compared, and the word vector similarity between the two sentences to be compared.

[0016] In a third aspect, an embodiment of the present application provides a computer storage medium, wherein the computer storage medium stores a plurality of instructions, wherein the instructions are suitable for being loaded by a processor and executing the above-mentioned method steps.

[0017] In a fourth aspect, an embodiment of the present application provides an electronic device, which may include: a processor and a memory;

[0018] The memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the above method steps.

[0019] The beneficial effects of the technical solutions provided by some embodiments of the present application include at least:

[0020] The embodiment of the present application can obtain two sentences to be compared and a decay factor; respectively vectorize the phrases and negation words in the two sentences to be compared to obtain the word vectors of each phrase and negation word in the two sentences to be compared; determine the word vector similarity between the two sentences to be compared based on the word vectors of each phrase and negation word in the two sentences to be compared; generate a syntactic structure binary tree of each sentence to be compared according to a preset sentence structure division rule; based on the decay factor, the syntactic structure binary tree of each sentence to be compared, and the word vector similarity between the two sentences to be compared, determine the sentence similarity between the two sentences to be compared. Therefore, the present application further takes into account the similarity between the word vectors containing negation words on the basis of sentence structure, and more accurately grasps the semantic relationship between the various components of the sentence by combining the two, and does not lose the information contained in the sentence vector. Therefore, the present application is more accurate than the prior art method of calculating the similarity between sentences based only on the syntactic structure binary tree. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0022] Figure 1 A schematic diagram of a binary tree of the syntactic structure of a sentence to be detected provided in an embodiment of the present application;

[0023] Figure 2 A flowchart of a method for calculating sentence similarity provided in an embodiment of the present application;

[0024] Figure 3 A flowchart of another method for calculating sentence similarity provided in an embodiment of the present application;

[0025] Figure 4a A schematic diagram of a binary tree of the syntactic structure of a sentence to be detected provided in an embodiment of the present application;

[0026] Figure 4b A schematic diagram of a binary tree of the syntactic structure of another sentence to be detected provided in an embodiment of the present application;

[0027] Figure 5a A schematic diagram of a binary tree of the syntactic structure of a sentence to be detected provided in an embodiment of the present application;

[0028] Figure 5b A schematic diagram of a binary tree of the syntactic structure of another sentence to be detected provided in an embodiment of the present application;

[0029] Figure 6 A schematic diagram of the structure of an information push device provided in an embodiment of the present application;

[0030] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0031] When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Instead, they are merely examples of devices and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0032] In the description of this application, it should be understood that the terms "first", "second", etc. are used for descriptive purposes only and should not be understood as indicating or implying relative importance. For those of ordinary skill in the art, the specific meanings of the above terms in this application can be understood according to specific circumstances. In addition, in the description of this application, unless otherwise specified, "multiple" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the previous and subsequent associated objects are in an "or" relationship.

[0033] In natural language, sentences can be divided into subject, predicate, and object representations. The way people speak (even drunken gibberish) has structures and rules. The goal of grammatical analysis in linguistics is to isolate these grammatical structures. The relationship between words is more than a simple sequence; it must describe how the words are connected. A comprehensible approach is to construct a tree structure for each sentence.

[0034] Syntactic parsing is a key technology in Natural Language Processing (NLP) that can help analyze the component structure of a sentence and the relationship between words. Syntactic analysis trees are mainly divided into two categories: the constituent tree and the dependency tree. Specifically, the constituent tree is used to express the syntactic structure of a sentence. Only its leaf nodes are associated with the words in the input sentence, and the other intermediate nodes are all marked phrase components. The dependency tree is used to express the dependency relationship between words in a sentence, and is used to analyze and identify grammatical components such as "subject, predicate, object", "attributive, adverbial, complement" in a sentence. Each node is a word.

[0035] See also Figure 1 An important feature of the syntax tree is that it has only one root node. Each node in the syntax tree represents an independent syntactic component, and the top root node represents the largest syntactic component in the entire sentence, that is, the sentence itself. For example, Figure 1 The top node is the root node, representing the sentence to be processed: "I eat." It can be understood that this is also a simple sentence IP. The other nodes in the sentence represent the syntactic components: the noun phrase NP (I, rice), the verb phrase VP (eat), and the verb VV (eat).

[0036] Next, combine Figure 1 The method for calculating sentence similarity provided in the embodiment of the present application is introduced with the help of a schematic diagram of a sentence structure tree in the related art.

[0037] In one embodiment, Figure 1 As shown in FIG, a flow chart of a method for calculating sentence similarity is provided. Figure 2 As shown, the method for calculating sentence similarity may include the following steps:

[0038] S201, obtaining two sentences to be compared and a decay factor.

[0039] At least one of the sentences to be compared includes a negative word. For example, the two sentences to be compared may be: "I eat" and "I don't eat", wherein "I don't eat" includes the negative word "no".

[0040] Specifically, the decay factor is used to represent a preset coefficient, which is between 0 and 1. The decay factor can be set according to user needs, for example, it can be determined according to the length of the sentence or the depth of the syntactic structure binary tree.

[0041] S202 , respectively vectorizing the phrases and negation words in the two sentences to be compared to obtain a word vector for each phrase and negation word in the two sentences to be compared.

[0042] Among them, word vectors are used to represent vectors in which words or phrases from the vocabulary are mapped to real numbers.

[0043] Possibly, the embodiments of the present application can obtain word vectors through a long short-term memory model (Long Short-Term Memory, LSTM), a bidirectional long short-term memory model (Bi-directional Long Short-Term Memory, BiLSTM), etc.

[0044] Specifically, the phrases in the sentence are replaced with word vectors according to the second dimension (the index of each word) before entering the LSTM, so the input of the LSTM can be a 3-dimensional vector (number of samples x sentence length x word vector dimension).

[0045] Specifically, the BiLSTM model can be composed of four parts: input layer: used to receive input sentences, which for Chinese refers to the words that have been divided into sentences; embedding layer: maps each word in the sentence into a vector of fixed length; long short-term memory layer: uses a bidirectional long short-term memory network (Long Short-Term Memory, LSTM) to calculate the vectors in the embedding layer. In fact, the bidirectional LSTM calculates the word vectors to obtain higher-level sentence vectors; output layer: used to output the word vector results corresponding to the words.

[0046] Possibly, inputting the two sentences to be compared, "I eat" and "I don't eat", into the BiLSTM model word vector may output word vectors: "1011" and "1111" respectively.

[0047] S203 : Determine the word vector similarity between the two sentences to be compared based on the word vector of each phrase and negation word in the two sentences to be compared.

[0048] It can be understood that this application can construct a Euclidean space by converting text into word vectors. Comparing the word vector similarity between two sentences is converted into measuring the distance between word vectors. There are many indicators to choose from, such as Euclidean distance, Pearson coefficient, and cosine distance.

[0049] Specifically, the vectors of the two words are A and B, and the word vector similarity between them is This is the cosine of the angle between two vectors, and its value range is [-1, 1]. When a pair of texts differ significantly in length but have similar content, their Euclidean distance in feature space is typically large if word frequency or word vectors are used as features. However, if cosine similarity is used, the angle between them may be small, resulting in high similarity. It is understandable that cosine distance often has high feature dimensionality in objects such as text. Cosine similarity maintains the property of "1 for identical, 0 for orthogonal, and -1 for opposite" in high dimensions. However, the Euclidean distance is affected by dimensionality, has an unfixed range, and is more ambiguous in meaning.

[0050] Possibly, the similarity between the BiLSTM word vector "1011" of the two sentences to be compared, "I eat" and the BiLSTM word vector "1111" of "I don't eat", is 1011·1111=3.

[0051] S204: Generate a syntactic structure binary tree of each sentence to be compared according to a preset sentence structure division rule.

[0052] Among them, the sentence structure division rules preset in the embodiment of the present application are used to represent the rules for generating a binary tree of the syntactic structure of the sentences to be compared.

[0053] It is understandable that due to the ambiguity in the grammatical parsing of a sentence, a method is needed to find the most likely tree from multiple possible grammatical trees. The preset sentence structure partitioning rule for generating the syntactic structure binary tree in this application can be Probabilistic Context-Free Grammar (PCFG).

[0054] Possibly, the embodiment of the present application can construct a probabilistic context-free grammar model by inputting the sentences to be compared into the model to output a binary tree of the syntactic structure of the sentences to be compared. Specifically, the embodiment of the present application can use the model to parse the sentences to be compared to obtain a parsing result, which is represented in the form of a binary tree. During the specific parsing, a dynamic programming method is used to find a grammar set that the sentences to be compared satisfy, and the grammars in the grammar set are reduced to obtain the reduced grammar. According to the grammar reduction path, multiple candidate grammar trees for phrases in the sentences to be compared are obtained, and the grammar tree with the highest probability is selected as the parsing result. Among them, the probability of the grammar tree is the product of the probabilities of occurrence of all grammars in the grammar tree. In this way, the ambiguity problem existing in formula parsing can be solved, and each grammar is assigned a probability. The probability can be obtained by training a large amount of text data containing the sentences to be compared. In this way, when the sentences to be compared are semantically parsed, they can be parsed according to the principle of maximum probability.

[0055] Furthermore, the probabilistic context-free grammar model can be obtained by pre-collecting a large amount of text data containing mathematical formulas using an offline method, and trained according to the context of each mathematical formula and the grammar set of the mathematical formula.

[0056] In embodiments of the present application, each node in the binary syntactic tree may include the content of the sentence to be compared, the node type, and node attributes. Specifically, node types may include root nodes, leaf nodes, and child nodes. Node attributes may include clauses, phrases, and words.

[0057] See also Figure 1 The sentence corresponding to the IP node at the top level contains the entire sentence: I eat breakfast. The node attribute of the IP node is a clause. The node type of the NP node at the middle level is a leaf node, and the node type of the VP node is a child node. The node attribute of the NP node is a noun phrase, and the node attribute of the VP node is a verb phrase. The node type of the V node and the NP node at the bottom level is a leaf node. The node attribute of the V node is a verb, and the node attribute of the NP node is a noun phrase.

[0058] S205 , determining the sentence similarity between the two sentences to be compared based on the decay factor, the syntactic structure binary tree of each sentence to be compared, and the word vector similarity between the two sentences to be compared.

[0059] It is understandable that the embodiment of the present application can use the decay factor to process the leaf nodes in the syntactic structure binary tree of each sentence to be compared, so as to reduce the redundancy caused by repeated calculation of leaf nodes due to the syntactic structure binary tree being too deep.

[0060] Specifically, the embodiment of the present application can determine the sentence similarity between the two sentences to be compared by combining the similarity between the syntactic structure binary tree nodes of the two sentences to be compared with the similarity of the word vectors between the two sentences to be compared.

[0061] Alternatively, the embodiment of the present application may calculate the similarity E between two sentences to be compared using the following formula:

[0062]

[0063] Among them, K(T1, T2) represents the comparison value between the two sentences to be compared, which may include the similarity between the syntactic structure binary tree nodes of the two sentences to be compared and the similarity of the word vectors between the two sentences to be compared; K(T1, T1) represents the comparison value of the first sentence to be compared itself, which may include the similarity between the syntactic structure binary tree nodes of the first sentence to be compared itself and the similarity of the word vectors of the syntactic structure binary tree nodes of the first sentence to be compared itself; K(T2, T2) represents the comparison value of the second sentence to be compared itself, which may include the similarity between the syntactic structure binary tree nodes of the second sentence to be compared itself and the similarity of the word vectors of the syntactic structure binary tree nodes of the second sentence to be compared itself.

[0064] Thus, the embodiment of the present application can obtain two sentences to be compared and a decay factor; respectively vectorize the phrases and negation words in the two sentences to be compared to obtain the word vectors of each phrase and negation word in the two sentences to be compared; determine the word vector similarity between the two sentences to be compared based on the word vectors of each phrase and negation word in the two sentences to be compared; generate a syntactic structure binary tree of each sentence to be compared according to a preset sentence structure division rule; based on the decay factor, the syntactic structure binary tree of each sentence to be compared, and the word vector similarity between the two sentences to be compared, determine the sentence similarity between the two sentences to be compared. Thus, the present application further considers the similarity between the word vectors containing negation words on the basis of sentence structure, and more accurately grasps the semantic relationship between the various components of the sentence by combining the two, and does not lose the information contained in the sentence vector. Therefore, the present application is more accurate than the prior art that only calculates the similarity between sentences based on the syntactic structure binary tree.

[0065] In some embodiments, Figure 3 The following is a flow chart showing a method for calculating sentence similarity provided by an embodiment of the present application. Figure 3 As shown, the method for calculating sentence similarity may include at least the following steps:

[0066] S301: Obtain two sentences to be compared and a decay factor.

[0067] Specifically, S301 is consistent with S201 and will not be repeated here.

[0068] S302 , respectively vectorizing the phrases in the two sentences to be compared to obtain a word vector for each phrase in the two sentences to be compared.

[0069] Specifically, S302 is consistent with S202 and will not be repeated here.

[0070] S303 : Determine the word vector similarity between the two sentences to be compared based on the word vector of each phrase in the two sentences to be compared.

[0071] Specifically, S303 is consistent with S203 and will not be repeated here.

[0072] S304: Generate a syntactic structure binary tree of each sentence to be compared according to a preset sentence structure division rule.

[0073] Specifically, S304 is consistent with S204 and will not be repeated here.

[0074] S305, traverse the syntactic structure binary trees of the two sentences to be compared. If the generation methods of the syntactic structure binary trees of the two sentences to be compared are the same, execute S305; if the generation methods of the syntactic structure binary trees of the two sentences to be compared are different, execute S306-S309.

[0075] It can be understood that traversal means starting from the root node and following a search route, visiting each node in the tree in turn to determine the node content, node type, and node attributes of each node.

[0076] Specifically, the embodiment of the present application can traverse the syntactic structure binary tree of the two sentences to be compared from top to bottom. Possibly, the embodiment of the present application can adopt a pre-order traversal method, first visiting the root node, then traversing the left subtree, and finally traversing the right subtree.

[0077] S306, when the generation method of the syntactic structure binary trees of the two sentences to be compared is the same, determine the sentence similarity between the two sentences to be compared based on the decay factor, the syntactic structure binary tree of each sentence to be compared, and the word vector similarity between the two sentences to be compared.

[0078] It can be understood that the same generation method in the embodiment of the present application means that the node types and node attributes of each level in the syntactic structure binary tree are the same.

[0079] See also Figure 4aThe syntactic structure binary tree of "I don't like picking apples" is shown below and Figure 4b The syntactic structure binary tree of "I don't like eating apples" is shown. Figure 4a and Figure 4b The node contents at each level are different, but what is certain is that Figure 4a and Figure 4b The syntactic structure binary tree in is generated in the same way, that is, the node attributes and node types are consistent.

[0080] Furthermore, the embodiment of the present application can determine the comparison value of each sentence to be compared based on the decay factor, the syntactic structure binary tree of each sentence to be compared, and the word vector similarity of each sentence to be compared. The comparison value between the two sentences to be compared is determined based on the content similarity of the leaf nodes in each level of the syntactic structure binary tree of the two sentences to be compared, the number of the root node and the child nodes in each level, and the word vector similarity between the two sentences to be compared. The sentence similarity between the two sentences to be compared is determined based on the comparison value of each sentence to be compared and the comparison value between the two sentences to be compared.

[0081] Specifically, the embodiment of the present application can determine the comparison value of each sentence to be compared based on the decay factor, the number of node types at each level in the syntactic structure binary tree of each sentence to be compared, and the word vector similarity of each sentence to be compared.

[0082] Possibly, the embodiment of the present application may use the following formula to calculate the comparison value of each sentence to be compared:

[0083]

[0084] Among them, T1 represents the syntactic binary tree of the first sentence to be compared, N1 represents the node set of the syntactic binary tree of the first sentence to be compared, n1 represents the node in the syntactic binary tree of the first sentence to be compared, #ch(n1) represents the number of child nodes in the i-th subtree, ch(n1,k) is the k-th child node, VSP is the similarity between the word vector obtained by the BiLSTM model of ch(n1,k) and the word vector obtained by the BiLSTM model of ch(n1,k), and λ(0<λ<1) is the decay factor.

[0085] Possibly, an embodiment of the present application can determine the content similarity of leaf nodes in each level based on the word vectors and decay factors corresponding to the content of the leaf nodes in each level; determine the comparison value between the two sentences to be compared based on the content similarity of the leaf nodes in each level of the syntactic structure binary tree of the two sentences to be compared, the number of nodes in each level, and the word vector similarity between the two sentences to be compared.

[0086] Possibly, the embodiment of the present application may use the following formula to calculate the comparison value between two sentences to be compared:

[0087]

[0088] Among them, T1 represents the syntactic binary tree of the first sentence to be compared, T2 represents the syntactic binary tree of the second sentence to be compared, N1 represents the node set of the syntactic binary tree of the first sentence to be compared, N2 represents the node set of the syntactic binary tree of the second sentence to be compared, n1 represents a node in the syntactic binary tree of the first sentence to be compared, n2 represents a node in the syntactic binary tree of the second sentence to be compared, #ch(n) represents the number of child nodes in the i-th subtree that are the same in the two trees, ch(n1,k) is the k-th child node in the syntactic binary tree of the first sentence to be compared, ch(n2,k) is the k-th child node in the syntactic binary tree of the second sentence to be compared, VSP is the similarity between the word vector obtained by the BiLSTM model of ch(n1,k) and the word vector obtained by the BiLSTM model of ch(n2,k), and λ(0<λ<1) is the decay factor.

[0089] Specifically, the content similarity of the leaf nodes in each level of the binary tree of the syntactic structure of the two sentences to be compared in the embodiment of the present application can be obtained by calculating λVS, where VS is the similarity of the BiLSTM word vectors of the leaf nodes.

[0090] For a specific example, see Figure 4a and Figure 4bAssuming the decay factor λ = 0.5, the first sentence to be compared is "I don't like picking apples", and the second sentence to be compared is "I don't like eating apples". First, according to the above formula, the comparison value of each sentence to be compared can be calculated separately. Starting from the root node of the syntactic binary tree, traversing the entire syntactic binary tree, we get: 1 root node (IP), 3 child nodes (VP, VP, IP), 5 leaf nodes (NP, ADVP, VV, VV, NP), and a total of 9 nodes; traversing the subtree corresponding to the second-level child node (VP) we get: 3 child nodes (VP, VP, IP), 4 leaf nodes (ADVP, VV, VV, NP), and a total of 7 nodes; traversing the subtree corresponding to the third-level child node (VP) we get: 2 child nodes (IP), 3 leaf nodes (VV, VV, NP), and a total of 5 nodes. Among them, assuming that the first sentence to be compared is "I don't like picking apples" and its word vector is 11101, and the second sentence to be compared is "I don't like eating apples" and is 11111, then the similarity of the leaf nodes at each level in the syntactic binary tree of the first sentence to be compared can be obtained according to λVS: 0.5*(1+1+1+1)=2, and the word vector similarity VSP of the first sentence to be compared is VSP=1+1+1+1=4; the similarity of the leaf nodes at each level in the second syntactic binary tree can be obtained according to λVS: 0.5*(1+1+1+1+1)=2.5, and the word vector similarity VSP of the second sentence to be compared is VSP=1+1+1+1+1=5; the similarity of the leaf nodes at each level in the two syntactic binary trees can be obtained according to λVS: 0.5*(1+1+1+1+1)=2.5. Thus, the comparison value K(T1, T1) of the first sentence to be compared is 0.5*(9+7+5+3+2+4)=15.5; the comparison value K(T2, T2) of the second sentence to be compared is 0.5*(9+7+5+3+2.5+5)=15.75; the comparison value K(T1, T2) between the two sentences to be compared is [0.5*(9+7+5+3+2.5+4)] 2 =15.25 2 =232.56; further, the similarity between the two sentences to be compared can be determined

[0091] Therefore, the present application can compare the node similarity between the components in two sentences based on the syntactic binary tree, and then further determine the similarity between the two sentences to be compared based on the similarity between similar words. In this way, the information contained in the sentence vector is not lost and the relationship between the sentence binary tree structure is compared, making the similarity between sentences more accurate.

[0092] S307. When the generation methods of the syntactic structure binary trees of two sentences to be compared are different, remove the negation words in at least one of the sentences to be compared, and determine the subtrees with the same node types and node attributes in the syntactic structure binary trees of the two sentences to be compared.

[0093] It can be understood that the different generation methods in the embodiments of the present application mean that there are differences in the node types and / or node attributes at each level in the syntactic structure binary tree.

[0094] See Figure 5a the syntactic structure binary tree of the sentence to be compared "I like to eat apples" shown in Figure 5b and Figure 5a and Figure 5b the syntactic structure binary tree of the sentence to be compared "I don't like to eat apples" shown in. It can be observed from

[0095] that the node types and node attributes in the third level from top to bottom are different. Therefore, the generation methods of the syntactic structure binary trees of these two sentences to be compared are different.

[0095] Furthermore, the embodiments of the present application can determine the same subtrees in the syntactic structure binary trees of two sentences to be compared by removing the negation words. For example, after removing the ADVP leaf node corresponding to the negation word "not" in Figure 5b , Figure 5a and Figure 5b the node content corresponding to the same subtrees in is "like to eat apples".

[0096] S308. Based on the decay factor, the syntactic structure binary tree of each sentence to be compared, and the word vector similarity of each sentence to be compared, determine the comparison value of each sentence to be compared itself.

[0097] Possibly, the embodiments of the present application can calculate the comparison value of each sentence to be compared itself using the following formula:

[0098] [[ID=−32]]

[0099] where, T1 represents the syntactic binary tree of the first sentence to be compared, N1 represents the set of nodes of the syntactic binary tree of the first sentence to be compared, n1 represents the node in the syntactic binary tree of the first sentence to be compared, #ch(n1) represents the number of child nodes in the i-th subtree, ch(n1,k) is the k-th child node, VSP is the similarity between the word vector obtained by the BiLSTM model of ch(n1,k) and the word vector obtained by the BiLSTM model of ch(n1,k), and λ(0 < λ < 1) is the decay factor.

[0100] S309, determining the comparison value between the two sentences to be compared based on the content similarity of the leaf nodes in the subtree with the same node type and node attributes, the number of nodes at each level in the subtree, and the word vector similarity between the two sentences to be compared.

[0101] Specifically, the embodiments of the present application can determine the content similarity of leaf nodes in subtrees of each level based on the word vectors and decay factors corresponding to the content of leaf nodes in subtrees with the same node type and node attributes; determine the comparison value between two sentences to be compared based on the content similarity of leaf nodes in the subtree excluding the top level, the number of nodes at each level in the subtree, and the word vector similarity between the two sentences to be compared.

[0102] Possibly, the embodiment of the present application may use the following formula to calculate the comparison value between two sentences to be compared:

[0103]

[0104] Among them, T1 represents the syntactic binary tree of the first sentence to be compared, T2 represents the syntactic binary tree of the second sentence to be compared, N1 represents the node set of the syntactic binary tree of the first sentence to be compared, N2 represents the node set of the syntactic binary tree of the second sentence to be compared, n1 represents a node in the syntactic binary tree of the first sentence to be compared, n2 represents a node in the syntactic binary tree of the second sentence to be compared, #ch(n) represents the number of child nodes in the i-th subtree that are the same in the two trees, ch(n1,k) is the k-th child node in the syntactic binary tree of the first sentence to be compared, ch(n2,k) is the k-th child node in the syntactic binary tree of the second sentence to be compared, VSP is the similarity between the word vector obtained by the BiLSTM model of ch(n1,k) and the word vector obtained by the BiLSTM model of ch(n2,k), and λ(0<λ<1) is the decay factor.

[0105] Possibly, the content similarity of leaf nodes in each level of the same subtree in the binary syntactic structure tree of the two sentences to be compared in the embodiment of the present application can be obtained by calculating λVS, where VS is the similarity of the BiLSTM word vectors of the leaf nodes.

[0106] S310 , determining the sentence similarity between the two sentences to be compared based on the comparison value of each sentence to be compared and the comparison value between the two sentences to be compared.

[0107] For a specific example, see Figure 5a and Figure 5bAssuming that the decay factor λ = 0.5, the first sentence to be compared is "I like to eat apples" and the second sentence to be compared is "I don't like to eat apples". First, according to the above formula, the comparison value of each sentence to be compared can be calculated separately. Starting from the syntactic binary tree of the first sentence to be compared, traversing the entire syntactic binary tree, we get: 1 root node (IP), 2 child nodes (VP ​​and IP), 4 leaf nodes (NP, VV, VV, NP), and a total of 7 nodes; traversing the subtree corresponding to the second-level child node (VP) we get: 2 child nodes (VP ​​and IP), 3 leaf nodes (VV, VV, NP), and a total of 5 nodes; traversing the subtree corresponding to the third-level child node (IP) we get: 1 child node (IP), 2 leaf nodes (VV, NP), and a total of 3 nodes. Starting from the syntactic binary tree of the second sentence to be compared, traversing the entire syntactic binary tree, we obtain: 1 root node (IP), 3 child nodes (VP, VP, IP), 5 leaf nodes (NP, ADVP, VV, VV, NP), and a total of 9 nodes; traversing the subtree corresponding to the second-level child node (VP), we obtain: 3 child nodes (VP, VP, IP), 4 leaf nodes (ADVP, VV, VV, NP), and a total of 7 nodes; traversing the subtree corresponding to the third-level child node (VP), we obtain: 2 child nodes (VP ​​and IP), 3 leaf nodes (VV, VV, NP), and a total of 5 nodes; traversing the subtree corresponding to the fourth-level child node (IP), we obtain: 1 child node (IP), 2 leaf nodes (VV, NP), and a total of 3 nodes. Furthermore, assuming that the first sentence to be compared is "I like to eat apples" with a word vector of 11011, and the second sentence to be compared is "I don't like to eat apples" with a word vector of 11111, then the similarity of the leaf nodes at each level in the syntactic binary tree of the first sentence to be compared can be obtained according to λVS: 0.5*(1+1+1+1)=2, and the word vector similarity of the first sentence to be compared VSP=1+1+1+1=4; the similarity of the leaf nodes at each level in the second syntactic binary tree can be obtained according to λVS: 0.5*(1+1+1+1)=2 +1)=2.5, the word vector similarity of the second sentence to be compared is VSP=1+1+1+1+1=5; since the node content corresponding to the same subtree in the syntactic binary trees of the two sentences to be compared is "like to eat apples", that is, two child nodes (VP ​​and IP) and three leaf nodes (VV, VV, NP), the similarity of the leaf nodes at each level in the two syntactic binary trees can be obtained according to λVS: 0.5*(1+1+1+1+1)=2.5, the word vector similarity of the two sentences to be compared is VSP=1+1+1+1=4.Therefore, the comparison value K(T1, T1) of the first sentence to be compared is 0.5*(7+5+3+2+4)=10.5; the comparison value K(T2, T2) of the second sentence to be compared is 0.5*(9+7+5+3+2.5+5)=15.75; the comparison value K(T1, T2) between the two sentences to be compared is [0.5*(5+3+2.5+4-5)]. 2 =4.75 2 =22.56; further, the similarity between the two sentences to be compared can be determined

[0108] Thus, the embodiment of the present application can determine the similarity between the two sentences to be compared by removing the negation words in the negation sentence and determining the same subtree in the two sentences to be compared. In this way, while removing the same nodes, the embodiment of the present application compares the similarity between the nodes and the similarity of the word vectors between the sentences when calculating the similarity, and it can be observed compared with the embodiment with the same syntactic structure binary tree generation method that when the syntactic structure binary tree is generated in a different manner and negation words appear, the similarity between the two sentences to be compared significantly decreases. Therefore, it can be determined that the embodiment of the present application can significantly improve the accuracy of the similarity between the compared sentences.

[0109] Figure 6 This is a schematic diagram of a device for calculating sentence similarity provided by an exemplary embodiment of the present application. The device for calculating sentence similarity can be set in a device such as a server to execute the method for calculating sentence similarity of any of the above embodiments of the present application. Figure 6 As shown, the device for calculating sentence similarity may include:

[0110] An acquisition module 61 is configured to acquire two sentences to be compared and a decay factor; wherein at least one of the sentences to be compared includes a negation word;

[0111] A word vector obtaining module 62 is used to vectorize the phrases and negation words in the two sentences to be compared, respectively, to obtain a word vector for each phrase and negation word in the two sentences to be compared;

[0112] A word vector similarity determination module 63 is configured to determine the word vector similarity between the two sentences to be compared based on the word vector of each phrase and negation word in the two sentences to be compared;

[0113] A generating module 64 is used to generate a syntactic structure binary tree of each sentence to be compared according to a preset sentence structure division rule;

[0114] The sentence similarity determination module 65 is configured to determine the sentence similarity between the two sentences to be compared based on the decay factor, the syntactic structure binary tree of each sentence to be compared, and the word vector similarity between the two sentences to be compared.

[0115] Thus, the embodiment of the present application can obtain two sentences to be compared and a decay factor; respectively vectorize the phrases and negation words in the two sentences to be compared to obtain the word vectors of each phrase and negation word in the two sentences to be compared; determine the word vector similarity between the two sentences to be compared based on the word vectors of each phrase and negation word in the two sentences to be compared; generate a syntactic structure binary tree of each sentence to be compared according to a preset sentence structure division rule; based on the decay factor, the syntactic structure binary tree of each sentence to be compared, and the word vector similarity between the two sentences to be compared, determine the sentence similarity between the two sentences to be compared. Thus, the present application further considers the similarity between the word vectors containing negation words on the basis of sentence structure, and more accurately grasps the semantic relationship between the various components of the sentence by combining the two, and does not lose the information contained in the sentence vector. Therefore, the present application is more accurate than the prior art that only calculates the similarity between sentences based on the syntactic structure binary tree.

[0116] In some embodiments, each node in the syntactic structure binary tree includes: the content of the sentence to be compared corresponding to the node, the node type, and the node attribute; wherein,

[0117] The node types include: root node, leaf node, and child node;

[0118] The node attributes include: clause, phrase, and word.

[0119] In some implementations, the sentence similarity determination module includes:

[0120] A first traversal unit, configured to traverse the syntactic structure binary tree of the two sentences to be compared;

[0121] The first determination unit is used to determine the sentence similarity between the two sentences to be compared based on the decay factor, the syntactic structure binary tree of each sentence to be compared, and the word vector similarity between the two sentences to be compared, when the generation method of the syntactic structure binary trees of the two sentences to be compared is the same.

[0122] In some embodiments, the first determining unit includes: a first determining subunit, configured to determine the comparison value of each sentence to be compared based on the decay factor, the binary tree of the syntactic structure of each sentence to be compared, and the word vector similarity of each sentence to be compared;

[0123] a second determining subunit, configured to determine a comparison value between the two sentences to be compared based on content similarity of leaf nodes at each level of the syntactic structure binary tree of the two sentences to be compared, the number of root nodes and child nodes at each level, and word vector similarity between the two sentences to be compared;

[0124] The third determining subunit is configured to determine the sentence similarity between the two sentences to be compared based on the comparison value of each sentence to be compared and the comparison value between the two sentences to be compared.

[0125] In some embodiments, the second determination subunit is specifically used to: determine the content similarity of the leaf nodes in each level based on the word vectors corresponding to the contents of the leaf nodes in each level and the decay factor; determine the comparison value between the two sentences to be compared based on the content similarity of the leaf nodes in each level of the syntactic structure binary tree of the two sentences to be compared, the number of root nodes and child nodes in each level, and the word vector similarity between the two sentences to be compared.

[0126] In some implementations, the sentence similarity determination module includes:

[0127] A second traversal unit, configured to traverse the syntactic structure binary trees of the two sentences to be compared;

[0128] a fourth determining subunit, configured to, when the syntactic structure binary trees of the two sentences to be compared are generated in different ways, remove negation words from at least one of the sentences to be compared, and determine subtrees having the same node type and node attributes in the syntactic structure binary trees of the two sentences to be compared;

[0129] a fifth determining subunit, configured to determine a comparison value of each sentence to be compared based on the decay factor, the binary tree of the syntactic structure of each sentence to be compared, and the word vector similarity of each sentence to be compared;

[0130] a sixth determining subunit, configured to determine a comparison value between the two sentences to be compared based on the content similarity of the leaf nodes in the subtrees having the same node type and node attributes, the number of nodes at each level in the subtrees, and the word vector similarity between the two sentences to be compared;

[0131] The seventh determining subunit is configured to determine the sentence similarity between the two sentences to be compared based on the comparison value of each sentence to be compared and the comparison value between the two sentences to be compared.

[0132] In some implementations, the sixth determining subunit is specifically configured to:

[0133] Based on the word vectors corresponding to the content of the leaf nodes in the subtree with the same node type and node attributes and the decay factor, the content similarity of the leaf nodes in the subtrees of each level is determined; based on the content similarity of the leaf nodes in the subtree without the top level, the number of nodes at each level in the subtree, and the word vector similarity between the two sentences to be compared, the comparison value between the two sentences to be compared is determined.

[0134] In some embodiments, the first determining subunit or the fifth determining subunit is specifically configured to:

[0135] Based on the decay factor, the number of node types at each level in the binary tree of the syntactic structure of each sentence to be compared, and the word vector similarity of each sentence to be compared, the comparison value of each sentence to be compared is determined.

[0136] It should be noted that the apparatus for calculating sentence similarity provided in the above embodiment only uses the division of the above functional modules as an example when executing the method for calculating sentence similarity. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus for calculating sentence similarity provided in the above embodiment and the embodiment of the method for calculating sentence similarity belong to the same concept. The implementation process thereof is detailed in the method embodiment and will not be repeated here.

[0137] The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0138] See Figure 7 , is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 7 As shown, the electronic device 70 may include: at least one processor 71 , at least one network interface 74 , a user interface 73 , a memory 75 , and at least one communication bus 72 .

[0139] The communication bus 72 is used to realize the connection and communication between these components.

[0140] The user interface 73 may include a display screen (Display) and a camera (Camera). Optionally, the user interface 73 may also include a standard wired interface and a wireless interface.

[0141] The network interface 74 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0142] The processor 71 may include one or more processing cores. The processor 71 utilizes various interfaces and circuits to connect various components within the electronic device 70. It executes instructions, programs, code sets, or instruction sets stored in the memory 75, and accesses data stored in the memory 75 to perform various functions and process data within the electronic device 70. Optionally, the processor 71 may be implemented using at least one of the following hardware forms: a digital signal processing (DSP), a field-programmable gate array (FPGA), or a programmable logic array (PLA). The processor 71 may integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. The CPU primarily processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing the content displayed on the display; and the modem handles wireless communications. It is understood that the modem may not be integrated into the processor 71 but implemented as a separate chip.

[0143] Among them, the memory 75 may include a random access memory (RAM) or a read-only memory (Read-Only Memory). Optionally, the memory 75 includes a non-transitory computer-readable storage medium. The memory 75 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 75 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 75 may also be optionally at least one storage device located away from the aforementioned processor 71. As Figure 7 As shown, the memory 85 as a computer storage medium may include an operating system, a network communication module, a user interface module, and an application program for calculating sentence similarity.

[0144] exist Figure 7 In the electronic device 70 shown, the user interface 73 is mainly used to provide an input interface for the user and obtain data input by the user; and the processor 71 can be used to call the sentence similarity calculation application stored in the memory 75 and specifically perform the following operations:

[0145] Obtain two sentences to be compared and a decay factor; wherein at least one of the sentences to be compared includes a negation word;

[0146] Vectorizing the phrases and negation words in the two sentences to be compared respectively to obtain word vectors for each phrase and negation word in the two sentences to be compared;

[0147] Determining the word vector similarity between the two sentences to be compared based on the word vector of each phrase and negation word in the two sentences to be compared;

[0148] Generate a syntactic structure binary tree of each sentence to be compared according to a preset sentence structure division rule;

[0149] The sentence similarity between the two sentences to be compared is determined based on the decay factor, the syntactic structure binary tree of each sentence to be compared, and the word vector similarity between the two sentences to be compared.

[0150] In some embodiments, each node in the syntactic structure binary tree includes: the content of the sentence to be compared corresponding to the node, the node type, and the node attribute; wherein,

[0151] The node types include: root node, leaf node, and child node;

[0152] The node attributes include: clause, phrase, and word.

[0153] In some implementations, when determining the sentence similarity between the two sentences to be compared based on the decay factor, the binary tree of the syntactic structure of each sentence to be compared, and the word vector similarity between the two sentences to be compared, the processor 71 specifically performs:

[0154] Traversing the syntactic structure binary tree of the two sentences to be compared;

[0155] When the syntactic structure binary trees of the two sentences to be compared are generated in the same way, the sentence similarity between the two sentences to be compared is determined based on the decay factor, the syntactic structure binary tree of each sentence to be compared, and the word vector similarity between the two sentences to be compared.

[0156] In some embodiments, when the processor 71 determines the sentence similarity between the two sentences to be compared based on the decay factor, the syntactic structure binary tree of each sentence to be compared, and the word vector similarity between the two sentences to be compared, when the syntactic structure binary trees of the two sentences to be compared are generated in the same manner, the processor 71 specifically performs:

[0157] Determining a comparison value of each sentence to be compared based on the decay factor, the syntactic structure binary tree of each sentence to be compared, and the word vector similarity of each sentence to be compared;

[0158] Determining a comparison value between the two sentences to be compared based on the content similarity of the leaf nodes in each level of the syntactic structure binary tree of the two sentences to be compared, the number of the root node and the child nodes in each level, and the word vector similarity between the two sentences to be compared;

[0159] The sentence similarity between the two sentences to be compared is determined based on the comparison value of each sentence to be compared and the comparison value between the two sentences to be compared.

[0160] In some embodiments, when the processor 71 determines the comparison value between the two sentences to be compared based on the content similarity of the leaf nodes in each level of the binary tree of the syntactic structure of the two sentences to be compared, the number of the root node and the child nodes in each level, and the word vector similarity between the two sentences to be compared, the processor 71 specifically performs:

[0161] Determining the content similarity of the leaf nodes in each level based on the word vectors corresponding to the content of the leaf nodes in each level and the decay factor;

[0162] The comparison value between the two sentences to be compared is determined based on the content similarity of the leaf nodes in each level of the syntactic structure binary tree of the two sentences to be compared, the number of nodes in each level, and the word vector similarity between the two sentences to be compared.

[0163] In some implementations, when the processor 71 determines the sentence similarity between the two sentences to be compared based on the syntactic structure binary tree of each sentence to be compared and the word vector similarity between the two sentences to be compared, specifically performs:

[0164] Traversing the syntactic structure binary tree of the two sentences to be compared;

[0165] In a case where the syntactic structure binary trees of the two sentences to be compared are generated in different ways, removing negation words in at least one of the sentences to be compared, and determining subtrees having the same node type and node attributes in the syntactic structure binary trees of the two sentences to be compared;

[0166] Determining a comparison value of each sentence to be compared based on the decay factor, the syntactic structure binary tree of each sentence to be compared, and the word vector similarity of each sentence to be compared;

[0167] Determine a comparison value between the two sentences to be compared based on the content similarity of the leaf nodes in the subtrees having the same node type and node attributes, the number of nodes at each level in the subtrees, and the word vector similarity between the two sentences to be compared;

[0168] The sentence similarity between the two sentences to be compared is determined based on the comparison value of each sentence to be compared and the comparison value between the two sentences to be compared.

[0169] In some embodiments, when the processor 71 determines the comparison value between the two sentences to be compared based on the content similarity of the leaf nodes in the subtrees having the same node type and node attributes, the number of child nodes in the subtrees, and the word vector similarity between the two sentences to be compared, the processor 71 specifically performs:

[0170] Determining the content similarity of leaf nodes in subtrees of each level based on the word vectors corresponding to the content of leaf nodes in the subtrees having the same node type and node attributes and the decay factor;

[0171] The comparison value between the two sentences to be compared is determined based on the content similarity of the leaf nodes in the subtree excluding the top layer, the number of nodes at each level in the subtree, and the word vector similarity between the two sentences to be compared.

[0172] In some implementations, when the processor 71 determines the comparison value of each sentence to be compared based on the decay factor, the binary tree of the syntactic structure of each sentence to be compared, and the word vector similarity of each sentence to be compared, the processor 71 specifically performs:

[0173] The comparison value of each sentence to be compared is determined based on the decay factor, the number of node types at each level in the syntactic structure binary tree of each sentence to be compared, and the word vector similarity of each sentence to be compared.

[0174] The embodiment of the present application also provides a computer-readable storage medium, which stores instructions, which, when executed on a computer or processor, enable the computer or processor to execute the above-mentioned Figure 2-Figure 3 One or more steps in the embodiment shown. If the components of the apparatus for calculating sentence similarity are implemented in the form of software functional units and sold or used as independent products, they can be stored in the computer-readable storage medium.

[0175] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When software is used for implementation, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted by the computer-readable storage medium. The computer instructions can be transmitted from a website, computer, server or data center to another website, computer, server or data center by wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrations. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a digital versatile disc (DVD)), or a semiconductor medium (eg, a solid state disk (SSD)).

[0176] Those skilled in the art will appreciate that all or part of the processes in the above-described embodiments can be implemented by instructing the relevant hardware through a computer program. The program can be stored in a computer-readable storage medium. When executed, the program can include the processes of the above-described embodiments. The aforementioned storage medium includes various media capable of storing program code, such as read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks. Unless there is a conflict, the technical features of this embodiment and the implementation scheme can be combined in any manner.

[0177] The embodiments described above are merely preferred embodiments of the present application and are not intended to limit the scope of the present application. Without departing from the design spirit of the present application, various modifications and improvements made to the technical solutions of the present application by ordinary technicians in this field should fall within the scope of protection determined by the claims of the present application.

Claims

1. A method for calculating sentence similarity, characterized in that: The method comprises: Obtain two sentences to be compared and a decay factor; wherein at least one of the sentences to be compared includes a negation word; Vectorizing the phrases and negation words in the two sentences to be compared respectively to obtain word vectors for each phrase and negation word in the two sentences to be compared; Determining the word vector similarity between the two sentences to be compared based on the word vector of each phrase and negation word in the two sentences to be compared; Generate a syntactic structure binary tree of each sentence to be compared according to a preset sentence structure division rule; Determining sentence similarity between the two sentences to be compared based on the decay factor, the syntactic structure binary tree of each sentence to be compared, and the word vector similarity between the two sentences to be compared; The determining of the sentence similarity between the two sentences to be compared based on the decay factor, the syntactic structure binary tree of each sentence to be compared, and the word vector similarity between the two sentences to be compared includes: Traversing the syntactic structure binary tree of the two sentences to be compared; In a case where the syntactic structure binary trees of the two sentences to be compared are generated in different ways, removing negation words in at least one of the sentences to be compared, and determining subtrees having the same node type and node attributes in the syntactic structure binary trees of the two sentences to be compared; Determining a comparison value of each sentence to be compared based on the decay factor, the syntactic structure binary tree of each sentence to be compared, and the word vector similarity of each sentence to be compared; Determine a comparison value between the two sentences to be compared based on the content similarity of the leaf nodes in the subtrees having the same node type and node attributes, the number of nodes at each level in the subtrees, and the word vector similarity between the two sentences to be compared; The sentence similarity between the two sentences to be compared is determined based on the comparison value of each sentence to be compared and the comparison value between the two sentences to be compared.

2. The method according to claim 1, characterized in that Each node in the syntactic structure binary tree includes: the content of the sentence to be compared corresponding to the node, the node type, and the node attribute; wherein, The node types include: root node, leaf node, and child node; The node attributes include: clause, phrase, and word.

3. The method according to claim 2, characterized in that Determining the sentence similarity between the two sentences to be compared based on the decay factor, the syntactic structure binary tree of each sentence to be compared, and the word vector similarity between the two sentences to be compared includes: Traversing the syntactic structure binary tree of the two sentences to be compared; When the syntactic structure binary trees of the two sentences to be compared are generated in the same way, the sentence similarity between the two sentences to be compared is determined based on the decay factor, the syntactic structure binary tree of each sentence to be compared, and the word vector similarity between the two sentences to be compared.

4. The method according to claim 3, characterized in that The method of determining the sentence similarity between the two sentences to be compared based on the decay factor, the syntactic structure binary tree of each sentence to be compared, and the word vector similarity between the two sentences to be compared, when the syntactic structure binary trees of the two sentences to be compared are generated in the same manner, includes: Determining a comparison value of each sentence to be compared based on the decay factor, the syntactic structure binary tree of each sentence to be compared, and the word vector similarity of each sentence to be compared; Determining a comparison value between the two sentences to be compared based on the content similarity of the leaf nodes in each level of the syntactic structure binary tree of the two sentences to be compared, the number of the root node and the child nodes in each level, and the word vector similarity between the two sentences to be compared; The sentence similarity between the two sentences to be compared is determined based on the comparison value of each sentence to be compared and the comparison value between the two sentences to be compared.

5. The method according to claim 4, characterized in that Determining the comparison value between the two sentences to be compared based on the content similarity of the leaf nodes in each level of the syntactic structure binary tree of the two sentences to be compared, the number of the root node and the child nodes in each level, and the word vector similarity between the two sentences to be compared includes: Determining the content similarity of the leaf nodes in each level based on the word vectors corresponding to the content of the leaf nodes in each level and the decay factor; The comparison value between the two sentences to be compared is determined based on the content similarity of the leaf nodes in each level of the syntactic structure binary tree of the two sentences to be compared, the number of nodes in each level, and the word vector similarity between the two sentences to be compared.

6. The method according to claim 1, characterized in that Determining the comparison value between the two sentences to be compared based on the content similarity of the leaf nodes in the subtree having the same node type and node attributes, the number of nodes at each level in the subtree, and the word vector similarity between the two sentences to be compared includes: Determining the content similarity of leaf nodes in subtrees of each level based on the word vectors corresponding to the content of leaf nodes in the subtrees having the same node type and node attributes and the decay factor; The comparison value between the two sentences to be compared is determined based on the content similarity of the leaf nodes in the subtree excluding the top layer, the number of nodes at each level in the subtree, and the word vector similarity between the two sentences to be compared.

7. The method according to claim 1 or 4, characterized in that Determining the comparison value of each sentence to be compared based on the decay factor, the binary tree of the syntactic structure of each sentence to be compared, and the word vector similarity of each sentence to be compared includes: Based on the decay factor, the number of node types at each level in the binary tree of the syntactic structure of each sentence to be compared, and the word vector similarity of each sentence to be compared, the comparison value of each sentence to be compared is determined.

8. A device for calculating sentence similarity, characterized in that: The device comprises: An acquisition module, configured to acquire two sentences to be compared and a decay factor; wherein at least one of the sentences to be compared includes a negation word; A word vector obtaining module is used to vectorize the phrases and negation words in the two sentences to be compared, respectively, to obtain a word vector for each phrase and negation word in the two sentences to be compared; A word vector similarity determination module, configured to determine the word vector similarity between the two sentences to be compared based on the word vector of each phrase and negation word in the two sentences to be compared; A generation module, configured to generate a binary tree of the syntactic structure of each sentence to be compared according to a preset sentence structure division rule; a sentence similarity determination module, configured to determine the sentence similarity between the two sentences to be compared based on the decay factor, the syntactic structure binary tree of each sentence to be compared, and the word vector similarity between the two sentences to be compared; Among them, the sentence similarity determination module is specifically used to: traverse the syntactic structure binary trees of the two sentences to be compared; when the generation methods of the syntactic structure binary trees of the two sentences to be compared are different, remove the negative words in at least one of the sentences to be compared, and determine the subtrees with the same node type and node attributes in the syntactic structure binary trees of the two sentences to be compared; determine the comparison value of each sentence to be compared based on the decay factor, the syntactic structure binary tree of each sentence to be compared, and the word vector similarity of each sentence to be compared; determine the comparison value between the two sentences to be compared based on the content similarity of the leaf nodes in the subtrees with the same node type and node attributes, the number of nodes at each level in the subtrees, and the word vector similarity between the two sentences to be compared; determine the sentence similarity between the two sentences to be compared based on the comparison value of each sentence to be compared and the comparison value between the two sentences to be compared.

9. A computer storage medium, characterized in that The computer storage medium stores a plurality of instructions, which are suitable for being loaded by a processor and executing the method according to any one of claims 1 to 7.

10. An electronic device, characterized in that: include: A processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the method according to any one of claims 1 to 7.

Images