Chinese sentence semantic similarity calculation method based on recursive convolutional neural network

Abstract

The application discloses a Chinese sentence semantic similarity calculation method based on a recursive convolutional neural network. It belongs to the field of data cleaning in data mining. The method is calculated based on a semantic similarity network model, comprising: obtaining a sentence pair data set, extracting the sentence pair in the data set, and obtaining the text sentence pair after word segmentation; extracting the preliminary semantic features and deep semantic features of the sentence pair; using an Attention layer to generate the weight of the sentence pair feature vector according to the deep semantic features of the sentence pair; inputting the weight of the sentence pair feature vector into a full connection layer to generate the similarity of the sentence pair. The application enriches the output connotation of the word vector, better reflects the original sentence pair input by using an adaptive pooling layer, improves the model performance, designs the Attention layer to assign corresponding weights to the sentence pair features, makes the model focus on the key information in the data, extracts more direct semantic dependency relationships in the sentence, and improves the accuracy of the sentence pair semantic similarity judgment.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of data cleaning in data mining, and particularly relates to a Chinese sentence semantic similarity calculation method based on a recursive convolutional neural network. BACKGROUND

[0002] With the rapid development of Internet technology, massive information is constantly emerging in the Internet, and text is the most important carrier of information. Research on the deep mining of text information is of great significance for people to quickly and accurately obtain the required content. Semantic text similarity calculation is the link between text representation and upper-layer application. At present, in the field of information retrieval, semantic text similarity calculation plays an extremely important role in text classification, text clustering, entity disambiguation and other tasks. In the field of artificial intelligence, question and answer systems and intelligent retrieval tasks also need semantic text similarity algorithm as support. In addition, semantic text similarity calculation is also widely used in plagiarism detection, text summarization, machine translation and other natural language processing tasks.

[0003] Given two texts A and B, semantic text similarity calculation aims to measure the degree of similarity between the two texts in semantics. Generally, the smaller the semantic similarity value of the text, the greater the semantic difference between the two texts, that is, the less similar they are in the semantic level; on the contrary, the greater the value, the more similar the semantics expressed by the two texts. Because human language expression is very complex, the text contains many synonyms, abbreviations, negations and other words, and the variable syntax structure also increases the difficulty of semantic text similarity calculation. In order to solve these problems, the academic and industrial circles have carried out a lot of research and practice, and have put forward a series of models and methods for the problem of semantic text similarity calculation.

[0004] The current research on semantic text similarity mainly includes string-based, statistical, knowledge-based and deep learning-based methods. The string-based method directly compares the original text, mainly including edit distance, longest common subsequence, N-Gram and Jaccard similarity, etc. The string-based method is simple in principle, easy to implement, and directly compares the original text, which is mainly used for fast fuzzy matching of text. Its main shortcomings are that it does not consider the meaning of words and the relationship between words, and it cannot handle synonyms, polysemous words and other problems. At present, the string-based method is rarely used alone to calculate text similarity. The statistical method is based on a distribution assumption that similar contexts have similar semantics. This kind of calculation method first converts the text into a vector through a certain strategy, then converts the vector space, and finally calculates the distance between the vectors representing the text, and measures the similarity between texts in the vector space. The mainstream statistical method includes vector space model and topic model. This method ignores the relationship between words, but in many cases, there is a relationship between words, so it is unreasonable to simply consider that words are independent of each other. With the rapid development of deep learning methods in recent years, the research on string-based methods has gradually declined, but the idea is still present in deep learning methods. The knowledge-based semantic text similarity calculation method can be divided into two categories according to the type of knowledge base. One is the ontology-based method, which uses structured semantic dictionaries for calculation. The basic idea is to use the concept information and hierarchical relationships between concepts contained in these semantic dictionaries to calculate semantic text similarity. The other is the network knowledge-based method, which mainly uses large network knowledge base resources such as Wikipedia and Baidu Encyclopedia, and calculates similarity through web content and hyperlinks between web pages. The method based on structured semantic dictionary does consider the semantic information between words to some extent, but the cost is high and experts are needed to participate in the establishment of semantic dictionary relationships, and the semantic dictionary in a specific field is prone to heterogeneous problems, making the algorithm not universal.

[0005] Since the distributed word vector was invented in 2013, the method based on deep learning has made great progress in the field of semantic text similarity. At present, the best several models are all based on deep learning. Therefore, this kind of method has very important significance in the field of similarity calculation. At present, the semantic text similarity calculation method based on deep learning can be divided into two categories: unsupervised method and supervised method. Whether it is an unsupervised method or a supervised method, it needs to be based on a distributed word vector. The unsupervised learning method can calculate the semantic similarity between texts without a labeled data set. This method is more general and is widely used in some resource-poor specific fields. The basic idea of this method is to perform self-supervised training through the information carried by the data set itself or to obtain a sentence vector by weighting and summing the word vectors in the sentence through mathematical analysis, and finally to represent the semantic text similarity by calculating the distance between vectors. The unsupervised method does not need a labeled training set to train the model, which reduces the cost of manual labeling and is widely used in some resource-poor specific fields. However, due to the inability to integrate labeled information and some prior knowledge, the calculation accuracy is lower than that of the supervised method.

[0006] The above methods have their advantages, but their respective shortcomings are also very obvious. For the problem of low calculation accuracy of traditional methods and unsupervised methods in deep learning, we now use more supervised methods to calculate the text similarity. The widely used BERT pre-training language model can well understand the context information of the sentence and vectorize the sentence. However, the output of the word vector is not comprehensive and cannot fully represent the connotation of the word vector. In the recursive convolutional neural network, the commonly used maximum pooling and average pooling methods cannot always obtain effective information, which affects the allocation of feature vector weights. Therefore, these problems will ultimately affect the training effect of the neural network and have a negative impact on the performance of the model. SUMMARY

[0007] In order to solve the problem of uneven allocation of feature vector weights in the prior art, the present application proposes a Chinese sentence semantic similarity calculation method based on a recursive convolutional neural network. The present application can effectively obtain the vector representation of the sentence semantics by capturing the semantics of multiple words, and by assigning different importance weight indicators to different positions of the vector, it can better reflect the original sentence pair input and further improve the accuracy of sentence pair semantic similarity judgment.

[0008] Technical scheme: A Chinese sentence semantic similarity calculation method based on a recursive convolutional neural network, which is calculated based on a semantic similarity network model, the semantic similarity network model includes a DeBERTa model, a recursive convolutional neural network model, an Attention layer, and a full connection layer; the maximum pooling layer in the recursive convolutional neural network model is replaced by an adaptive pooling layer; the full connection layer adopts a Softmax function, and the method comprises the following steps:

[0009] Step one, obtain a sentence pair data set, extract the sentence pairs in the data set, preprocess the sentence pairs to obtain the segmented text sentence pairs;

[0010] Step two, extract the preliminary semantic features and deep semantic features of the sentence pairs in the DeBERTa model and the recursive convolutional neural network model;

[0011] Step three, use the Attention layer to generate the weight of the sentence pair feature vector according to the deep semantic features of the sentence pair;

[0012] Step four, input the weight of the sentence pair feature vector into the full connection layer to generate the similarity of the sentence pair.

[0013] Further, in step one:

[0014] In the sentence pair data set, a plurality of text data are included, wherein each data has a pair of sentences separated by a comma;

[0015] The preprocessing includes removing the question sentence pair, part-of-speech tagging, word segmentation processing, and vectorization processing.

[0016] Further, step two comprises the following steps:

[0017] Step 2.1, input the preprocessed sentence pair into the DeBERTa model, perform word embedding on the sentence pair by the DeBERTa to obtain dynamic word vectors, and extract the preliminary semantic features of the sentence pair; the DeBERTa model uses a decoupled attention mechanism DAM to represent each word with two vectors, respectively encoding its content and relative position, and then respectively calculating the weight through the Self-attention mechanism of the Transformer according to the content and relative position of the word;

[0018] Step 2.2, input the word vectors obtained in step 2.1 into the recursive convolutional neural network model to extract deep semantic features of the sentence pair; the recursive convolutional neural network model includes a bidirectional long short-term memory network BI-LSTM, a CNN convolutional layer, and an adaptive pooling layer.

[0019] Further, in step 2.2, the deep semantic feature generation method is:

[0020] The word vector is input to the CNN convolution layer after being output by the bidirectional LSTM model, and the CNN convolution kernel trains the feature vector to obtain a deep semantic feature vector; the deep semantic feature vector obtained by the CNN convolution layer enters the adaptive pooling layer, in which the weight size of each position is adjusted according to different local regions in the input feature channel; after the CNN convolution layer, the adaptive pooling layer extracts high-level semantic information F of the last layer of the feature extraction layer using a classification network, F element of R H×W×C , H, W and C are the height, width and depth of the image respectively; for each HxW-dimensional feature map F i , F i element of F, a 1xKxK convolution kernel is used to sequentially obtain the information of each position and its surrounding positions to generate the importance weight of the position.

[0021] Further, in step four, the Chinese sentence pair semantic similarity calculation method is:

[0022] The deep semantic features of the sentence pair are normalized by using a Softmax function, the Pearson coefficient between the deep semantic features is calculated, and the similarity of the sentence pair is calculated by using cosine similarity:

[0023]

[0024] Wherein, a i , b i represent the deep semantic feature vector of each word in the first sentence and the second sentence of the sentence pair respectively;

[0025] According to the similarity of the sentence pair, it is judged whether the sentences are similar or not, and the similar judgment is 1 and the dissimilar judgment is 0.

[0026] Further, the semantic similarity network model further comprises an optimization module, the optimization module comprises an Adam module and a dropout module, and the semantic similarity network model is optimized through the optimization module.

[0027] Compared with the prior art, the present application has the following beneficial effects:

[0028] 1) In view of the problem that the traditional maximum pooling cannot always obtain effective information, the present application proposes a word vector weight algorithm, uses an adaptive pooling method to replace the maximum pooling in the recurrent convolutional neural network, assigns different importance weight indicators to different positions of the vector, and can better reflect the original sentence pair input, improve the performance of the recurrent convolutional neural network model, and is superior to the traditional maximum pooling method.

[0029] 2) In view of the difficulty of reasonably assigning the weight of the feature vector, an Attention mechanism is introduced to calculate the weight and encode after the pooling layer, and the Attention layer is used to assign the corresponding weight to the feature of the sentence pair, so that the model can focus on the key information in the data during training, reduce the attention to the low correlation information that is not important in the text, extract more direct semantic dependency relationship in the sentence, better assign the weight of the deep semantic feature vector, and improve the accuracy of the semantic similarity calculation. BRIEF DESCRIPTION OF DRAWINGS

[0030] Figure 1 BERT pre-training language model structure;

[0031] Figure 2 DeBERTa pre-training language model structure in the application;

[0032] Figure 3 Text sentence pair semantic similarity calculation flowchart;

[0033] Figure 4 The overall framework of the model. DETAILED DESCRIPTION

[0034] The technical solutions of the application will be described in detail below through examples, but the protection scope of the application is not limited to the examples.

[0035] The application proposes a Chinese sentence semantic similarity calculation method based on a recursive convolutional neural network, which inputs a pair of preprocessed sentence pair text into a semantic similarity network model, calculates the similarity, and judges whether it is a duplicate sentence pair. The similarity of the sentence pairs in the data set is judged, and the results are analyzed and discussed.

[0036] The semantic similarity calculation method is calculated based on a semantic similarity network model, as shown in Figure 4 The semantic similarity network model includes a DeBERTa model, a recursive convolutional neural network model, an Attention layer, a full connection layer, and an optimization module. The maximum pooling layer in the recursive convolutional neural network model is replaced by an adaptive pooling layer. The full connection layer adopts a Softmax function. The optimization module includes a dropout module and an Adam module.

[0037] The semantic similarity calculation method, as shown in Figure 3 The specific steps are as follows:

[0038] Step 1, obtain a sentence pair data set, extract the sentence pairs in the data set, and pre-process the sentence pairs to obtain the text sentence pairs after word segmentation;

[0039] 1) The sentence pair dataset includes thousands of text data, each of which contains a pair of sentences separated by ",".

[0040] 2) Extract sentence pairs from each data set and preprocess them, including removing problematic sentence pairs, part-of-speech tagging, word segmentation, and vectorization.

[0041] Part-of-speech tagging and word segmentation are both basic tasks in natural language processing that are relatively low-level and have relatively mature technologies. They can be easily implemented using existing tools. In this embodiment, the jieba tool is used to complete the part-of-speech tagging of Chinese, and the Pangu word segmentation tool is used to segment Chinese sentence pairs. After word segmentation, the sentence pairs obtained are vectorized by WordEmbedding for each word to obtain sentence pair vector features.

[0042] Step 2: Extract the preliminary semantic features and deep semantic features of the sentence pairs in the DeBERTa model and the recursive convolutional neural network model. The specific steps are as follows:

[0043] Step 2.1: Input the preprocessed sentence pairs into the DeBERTa model, use DeBERTa to embed the sentence pairs into dynamic word vectors, and extract the preliminary semantic features of the sentence pairs.

[0044] like Figure 1 As shown in the figure, in the traditional BERT model, each word is represented by only one vector, which is just the sum of the word's content embedding (Token Embeddings), separation embedding (Segment Embeddings), and position embedding (PositionEmbeddings).

[0045] like Figure 2 As shown in the figure, the DeBERTa model proposed in the present invention uses the decoupled attention mechanism DAM to represent each word with two vectors, encoding its content and relative position respectively, and then calculates the weight according to the content and relative position of the word through the Transformer's Self-attention mechanism, increasing the calculation between content and position, and further emphasizing the correlation between content and position.

[0046] 1) The calculation process of the decoupled attention input representation of the DeBERTa model is as follows;

[0047]

[0048] Among them, H i 、H j represents the content embedding of the i-th and j-th words, P i|j represents the relative position embedding of word i and word j, Representative content-to-content computation, and Representative position-to-position computation. Representative content-to-position and position-to-content computation.

[0049] 2) The relative distance of the i-th and j-th words in the decoupled attention mechanism of the DeBERTa model is calculated as follows:

[0050]

[0051] where k represents the maximum relative distance.

[0052] 3) The calculation formula of the decoupled attention mechanism of the DeBERTa model is as follows:

[0053] Q c = HW q,c ,K c = HW k,c ,V c = HW v,c

[0054] Q r = PW q,r ,K r = PW k,r

[0055]

[0056] where H is the input information, P is the relative position information, Q c ,K c, V c is the content embedding, Q r ,K r is the relative position embedding, W q,c ,W k,c ,W v,c is the content weight matrix, W q,r ,W k,r is the relative position weight matrix, is the Q c of the i-th word, is the K c of the j-th word, is the K r from i to j, is the Q r from j to i, T is the transpose, attention matrix, is the attention matrix, is the element of H o is the output information.

[0057] Step 2.2: Input the word vector obtained in step 2.1 into the recursive convolutional neural network model to extract the deep semantic features of the sentence pair;

[0058] The recursive convolutional neural network model includes a bidirectional long short-term memory network BI-LSTM, a CNN convolutional layer and an adaptive pooling layer.

[0059] The method for generating deep semantic feature vectors is as follows: after the word vector is spliced ​​through the output of the bidirectional LSTM model, it is input into the CNN convolution layer. The CNN convolution kernel trains the feature vector to obtain the deep semantic feature vector; the deep semantic feature vector obtained by the CNN convolution layer enters the adaptive pooling layer. In the adaptive pooling layer, the weight of each position is adjusted according to the different local areas in the input feature channel.

[0060] Max pooling only takes the maximum value of the features in the pooling domain, which will affect the accuracy of text feature extraction and thus affect the subsequent classification work. Therefore, this invention adopts an adaptive pooling method to improve the accuracy of text feature extraction by calculating the importance weight index of different positions in the feature map.

[0061] Adaptive pooling method:

[0062] After the CNN convolutional layer, the adaptive pooling layer uses the classification network to extract the high-level semantic information F, F∈R, which is the last layer of the feature extraction layer. H×W×C , H, W, C are the height, width, and depth of the image respectively. For each feature map F of H×W dimensions i , F i ∈F, a 1×K×K convolution kernel is used to sequentially obtain information about each position and its surrounding positions to generate the importance weight of the position. The specific steps are as follows:

[0063] 1) For the convolution kernel size K, different sizes of convolution kernels are used for processing feature maps of different sizes to obtain the weights of each position of different text channels. The relationship between the convolution kernel size and the number of channels is as follows:

[0064]

[0065] Among them, H is the height, W is the width, C is the number of channels, and K is the size of the convolution kernel.

[0066] 2) Performing Instance Normalization on the same channel in each feature map is beneficial for processing the results of our depthwise convolution. By calculating the mean and variance of each channel, we change the distribution of the results to make them more consistent, which can produce better results. The calculation formula for Instance Normalization is as follows:

[0067]

[0068] where u is the mean, s is the variance, t is the index of input feature, i is the index of corresponding channel, x is the input segmented image, m is the output after segmentation. u 2 represents the mean, ti represents the variance.

[0069] 3) After the importance degree weight calculation, each channel is normalized after sequentially passing through Instance Normalization, Sigmoid activation, and amplifying the difference to generate the final weight M F ∈R H×W×C .

[0070] In the recursive convolutional neural network model, the original maximum pooling layer is replaced with an adaptive pooling layer, which adjusts the weight size of each position according to different local regions in the input feature channel, can generate a fixed size of output feature map, and improves the performance of the recursive convolutional neural network model. In the adaptive pooling layer, different channels of the feature map can independently extract local features and perform feature conversion. The adaptive pooling layer uses a deep convolution operation combined with different size convolution kernels, and the number of convolution kernels is the same as the number of channels of the input feature map, and each convolution kernel only processes one channel. The advantage of this is not only to reduce the introduction of parameters and computational complexity, but also from the results, the final weight is generated by capturing the local feature map region on each feature channel independently, which is better than the effect after linear combination after calculating on multiple channels at the same time. The adaptive pooling layer adjusts the weight size of each position according to different local regions in the input feature channel. This strategy can further utilize the learning ability of the network, and can dynamically adjust the weight of different local features in the text classification task, which has obvious benefits for the classification of text and the generation of classification network with more accurate position information.

[0071] Step three, using the Attention layer to generate the weight of the sentence pair feature vector according to the deep semantic features of the sentence pair;

[0072] The Attention layer is used to assign corresponding weights to the sentence pair features. In order to enable the recursive convolutional neural network to focus on the key information in the data during training, reduce the attention degree of unimportant low correlation information in the text, and extract more direct semantic dependency relationships in the sentence, the Attention mechanism is introduced after the RCNN network, and the weights of different feature vectors are calculated through the Attention mechanism to encode the text sentence. Attention mechanism calculation method:

[0073] e i ​= v i tanh(W i h i +b i )

[0074]

[0075] wherein, v i represents a parameter vector; W i represents a weight matrix; b i represents a bias vector; h i represents an input hidden vector. e i , e j indicates the assigned weight, alpha i is the degree of association, and S indicates the Attention output.

[0076] Step four, inputting the weight of the sentence pair feature vector into the full connection layer to generate the similarity of the sentence pair;

[0077] The input full connection layer calculates the similarity of the sentence pair through Softmax, and judges whether the sentences are similar or not;

[0078] The Chinese sentence pair semantic similarity calculation method is:

[0079] The Softmax function is used for normalizing the deep semantic feature vectors of the sentence pair, the Pearson coefficient between the deep semantic feature vectors is calculated, and the cosine similarity is used to calculate the similarity of the sentence pair:

[0080]

[0081] wherein, a i , b i respectively represent the deep semantic feature vectors of each word in the first sentence and the second sentence in the sentence pair.

[0082] According to the sentence pair similarity, whether the sentences are similar or not is judged, the decision threshold is not strictly specified, and can be freely adjusted. In the embodiment, 0.6 is set as the boundary. The similarity judgment is 1, and the dissimilarity judgment is 0.

[0083] Step five, optimizing the semantic similarity network model through the optimization module;

[0084] The Adam module is also added to the semantic similarity network model to optimize the model and reduce the loss parameter of the model. In addition, the learning rate of the semantic similarity network model is continuously adjusted, various parameters are debugged, and the dropout module is added to randomly discard a part of neurons to reduce the dependence on the training samples, so that the target of improving the performance of the model is finally achieved.

[0085] The model of the present application and the existing model are experimented using the dataset ChineseSTS, and the sentence pairs are segmented using a segmentation tool. In the model of the present application, the parameter settings are as shown in the following figure:

[0086] Table 1 simulation parameters

[0087]

[0088] The experimental data are randomly divided into a training set and a test set according to a ratio of 7:3. The data used in the experiment include a short text pair dataset from ChineseSTS, a medical field dataset and a payment Al financial sentence pair dataset.

[0089] The model is experimented using three different field public datasets, and the results of comparison between the present method and the model of the prior art are shown in the following figure:

[0090] Table 2 training results of short text dataset

[0091]

[0092] Table 3 training results of medical field dataset

[0093]

[0094] Table 4 training results of payment Al financial sentence pair dataset

[0095]

[0096] From the above results, it can be seen that the precision, accuracy and recall rate of the DeBERTa-RCNN-Attention (self-adaptive pooling) model (the model of the present application) are the highest in the three datasets.

[0097] Through comparison of the components of each model, it is found that the results of the model are apparently affected to a great extent by the training method. The BERT model and the RoBERTa are trained on a large amount of data, and can obtain part of the semantic information of a sentence from pre-training, but lack extraction of interactive features and weight components, and lack special training of specific training samples, which will affect the calculation and discrimination of the sentence pair text similarity in a specific field.

[0098] And the recursive convolutional neural network (RCNN) which depends on DeBERTa language model and adaptive pooling, for the input of two sentences, respectively, word segmentation preprocessing, input into DeBERTa language model, for each word of each sentence, respectively, produce two dynamic word vectors according to its content and relative position, then input the two vector sets into the bidirectional long short-term memory network (Bi-LSTM) to get semantic feature vectors through forward and backward training, then input the semantic feature vectors into CNN for convolution training to get deep semantic feature vectors, then use the Attention mechanism to assign different weights according to the deep semantic feature vectors, and use the generated weight vector to calculate the similarity of the sentence pair. From the results, it can be seen that DeBERTa-RCNN-Attention (adaptive pooling) can better understand the interaction and semantic correlation features between the sentence pair, so as to better judge whether the two sentences are similar.

[0099] As described above, although the present application has been shown and described with respect to a certain preferred embodiments, it is not to be construed as a limitation on the present application itself. Various changes in form and details can be made therein without departing from the spirit and scope of the present application.

Claims

1. A method for calculating the semantic similarity of Chinese sentences based on recursive convolutional neural networks, characterized in that: The method performs calculations based on a semantic similarity network model, which includes a DeBERTa model, a recursive convolutional neural network model, an attention layer, and a fully connected layer. The maximum pooling layer in the recursive convolutional neural network model is replaced with an adaptive pooling layer. The fully connected layer uses a Softmax function. The method includes the following steps: Step 1: Obtain a sentence pair dataset, extract sentence pairs from the dataset, preprocess the sentence pairs, and obtain text sentence pairs after word segmentation; Step 2: Extract the preliminary semantic features and deep semantic features of the sentence pairs in the DeBERTa model and the recursive convolutional neural network model; Step 3: Use the Attention layer to generate the weight of the sentence pair feature vector based on the deep semantic features of the sentence pair; Step 4: Input the weight of the sentence pair feature vector into the fully connected layer to generate the similarity of the sentence pair; Step 2 includes the following steps: Step 2.1: Input the preprocessed sentence pairs into the DeBERTa model, use DeBERTa to embed the sentence pairs into dynamic word vectors, and extract the preliminary semantic features of the sentence pairs. The DeBERTa model uses the decoupled attention mechanism (DAM) to represent each word with two vectors, encoding its content and relative position respectively. Then, the Transformer's self-attention mechanism is used to calculate the weight based on the content and relative position of the word. Step 2.2: Input the word vector obtained in step 2.1 into the recursive convolutional neural network model to extract the deep semantic features of the sentence pair; The recursive convolutional neural network model includes a bidirectional long short-term memory network (BI-LSTM), a CNN convolutional layer, and an adaptive pooling layer; In step 2.2, the deep semantic feature generation method is: After the word vectors are spliced ​​through the output of the bidirectional LSTM model, they are input into the CNN convolutional layer. The CNN convolution kernel trains the feature vectors to obtain deep semantic feature vectors. The deep semantic feature vectors obtained by the CNN convolutional layer enter the adaptive pooling layer. In the adaptive pooling layer, the weight of each position is adjusted according to the different local areas in the input feature channel. After the CNN convolutional layer, the adaptive pooling layer uses the classification network to extract the high-level semantic information F of the last layer of the feature extraction layer. , H, W, C are the height, width, and depth of the image respectively; for each Feature map of dimension , , using a The convolution kernel is used to obtain the information of each position and its surrounding positions in turn to generate the importance weight of the position.

2. The calculation method according to claim 1, characterized in that In step one: The sentence pair dataset includes several text data, each of which contains a pair of sentences separated by ",". Preprocessing includes removing problematic sentence pairs, part-of-speech tagging, word segmentation, and vectorization.

3. The calculation method according to claim 1, characterized in that In step 4, the semantic similarity calculation method of Chinese sentence pairs is: use The function normalizes the deep semantic features of the sentence pair, calculates the Pearson coefficient between the deep semantic features, and uses cosine similarity to calculate the similarity of the sentence pair: ； in, 、 Represents the deep semantic feature vector of each word in the first and second sentences of the sentence pair respectively; The similarity of the sentences is determined based on the similarity of the sentences, with 1 for similarity and 0 for dissimilarity.

4. The calculation method according to claim 1, characterized in that The semantic similarity network model also includes an optimization module, which includes an Adam module and a dropout module. The semantic similarity network model is optimized by the optimization module.

Images