Chinese sentence semantic similarity calculation method based on recursive convolutional neural network

Abstract

The invention discloses a Chinese sentence semantic similarity calculation method based on a recursive convolutional neural network. Belongs to the field of data cleaning in data mining, and the method performs calculation based on a semantic similarity network model, and comprises the following steps: obtaining a sentence pair data set, and extracting sentence pairs in the data set to obtain word-segmented text sentence pairs; extracting preliminary semantic features and deep semantic features of the sentence pairs; using an Attention layer to generate weights of feature vectors of the sentence pairs according to the deep semantic features of the sentence pairs; and inputting the weights of the feature vectors of the sentence pairs into the full connection layer to generate the similarity of the sentence pairs. According to the method, the output connotation of the word vector is enriched, the input of the original sentence pair is better reflected by using the self-adaptive pooling layer, and the performance of the model is improved; according to the method, an Attention layer is designed to distribute corresponding weights for sentence pair features, so that a model is focused on key information in data, a more direct semantic dependency relationship in sentences is extracted, and the accuracy of semantic similarity judgment of the sentences is improved.

Description

Technical Field

[0001] The present invention belongs to the field of data cleaning in data mining, and specifically is a method for calculating the semantic similarity of Chinese sentences based on a recursive convolutional neural network. Background Art

[0002] With the rapid development of Internet technology, massive amounts of information are constantly emerging on the Internet, and text is the most important carrier of information. Research on deep mining of text information is of great significance for people to quickly and accurately obtain the required content. Semantic text similarity calculation is the link between text representation and upper-level applications. At present, in the field of information retrieval, semantic text similarity calculation plays an extremely important role in tasks such as text classification, text clustering, and entity disambiguation. In the field of artificial intelligence, tasks such as question-answering systems and intelligent retrieval also require semantic text similarity algorithms as support. In addition, semantic text similarity calculation is also widely used in natural language processing tasks such as plagiarism detection, text summarization, and machine translation.

[0003] Given two texts A and B, semantic text similarity calculation aims to measure the degree of semantic similarity between the two texts. Generally, the smaller the semantic similarity value of the text, the greater the semantic difference between the two texts, that is, the less similar they are at the semantic level; conversely, the larger the value, the more similar the semantics expressed by the two texts. Since human language expression is very complex, the text contains many synonyms, abbreviations, negative words, etc., as well as variable syntactic structures, which increase the difficulty of semantic text similarity calculation. In order to solve these problems, academia and industry have conducted a lot of research and practice, and proposed a series of models and methods for semantic text similarity calculation problems.

[0004] At present, the research on semantic text similarity calculation mainly includes string-based, statistics-based, knowledge-based and deep learning-based methods. String-based methods directly compare the original text, mainly including edit distance, longest common subsequence, N-Gram and Jaccard similarity. The string-based method is simple in principle and easy to implement. It directly compares the original text and is mostly used for fast fuzzy matching of text. Its shortcomings are mainly that it does not take into account the meaning of words and the relationship between words, and cannot handle problems such as synonyms and polysemy. At present, string-based methods are rarely used alone to calculate text similarity. The statistical-based method originates from a distribution hypothesis, which assumes that words with similar contexts have similar semantics. This type of calculation method first converts the text into a vector through a certain strategy, then transforms through various vector spaces, and finally calculates the distance between vectors representing the text, and measures the similarity between texts through metrics in the vector space. The mainstream statistical-based methods include vector space models and topic models. This method ignores the relationship between words, but in many cases, there is a connection between words, so it is unreasonable to simply assume that words are independent of each other. With the rapid development of deep learning methods in recent years, the research enthusiasm based on strings has gradually faded, but its ideas are still present in deep learning methods. The semantic text similarity calculation methods based on knowledge bases can be divided into two categories according to the type of knowledge base. One is the ontology-based method, which uses structured semantic dictionaries for calculation. Its basic idea is to use the concept information contained in these semantic dictionaries and the hierarchical relationship between concepts to calculate semantic text similarity. The other is the network knowledge-based method, which mainly uses large network knowledge base resources, such as Wikipedia and Baidu Encyclopedia, to calculate similarity through web page content and hyperlinks between web pages. The method based on structured semantic dictionaries does take into account the semantic information between words to a certain extent, but the labor cost is high, experts are required to participate in establishing semantic dictionary relationships, and semantic dictionaries in specific fields are prone to heterogeneity problems, resulting in the algorithm not being universal.

[0005] Since the advent of distributed word vectors in 2013, deep learning-based methods have made great progress in the field of semantic text similarity. Currently, the best models are all implemented based on deep learning methods. Therefore, this type of method is of great significance in the field of similarity calculation. At present, semantic text similarity calculation methods based on deep learning can be divided into two categories: unsupervised methods and supervised methods. Both unsupervised and supervised methods need to be developed on the basis of distributed word vectors. Unsupervised learning methods do not require labeled data sets to calculate the semantic similarity between texts. This type of method is more general and widely used in some specific fields with scarce resources. The basic idea of ​​this type of method is to use the information in the data set itself for self-supervised training or mathematical analysis to perform weighted summation of word vectors in the sentence to obtain sentence vectors, and finally calculate the distance between vectors to represent the semantic text similarity. Unsupervised methods do not require labeled training sets to train the model, which reduces the cost of manual labeling and is widely used in some specific fields with scarce resources. However, due to the inability to integrate labeled information and some prior knowledge, their calculation accuracy is lower than that of supervised methods.

[0006] All of the above methods have their merits, but their shortcomings are also very obvious. In order to solve the problem of low calculation accuracy of traditional methods and unsupervised methods in deep learning, we now use more supervised methods to calculate text similarity. The BERT pre-trained language model, which is widely used today, can understand the context information of sentences very well and vectorize sentences. However, the output of its word vector is not comprehensive and cannot fully express the connotation of word vector. In recursive convolutional neural networks, the commonly used maximum pooling and average pooling methods cannot always obtain effective information, which in turn affects the distribution of feature vector weights. Therefore, these problems will eventually affect the effect of neural network training and have a negative impact on model performance. Summary of the invention

[0007] In order to solve the problem of uneven weight distribution of feature vectors in the prior art, the present invention proposes a method for calculating the semantic similarity of Chinese sentences based on a recursive convolutional neural network. The present invention can capture the semantics of multiple words and effectively obtain the vector representation of the semantics of sentences. By assigning different importance weight indicators to different positions of the vector, the original sentence pair input can be better reflected, and the accuracy of the semantic similarity judgment of the sentence pair can be further improved.

[0008] Technical solution: A method for calculating the semantic similarity of Chinese sentences based on a recursive convolutional neural network. The method performs calculations based on a semantic similarity network model. The semantic similarity network model includes a DeBERTa model, a recursive convolutional neural network model, an Attention layer, and a fully connected layer. The maximum pooling layer in the recursive convolutional neural network model is replaced by an adaptive pooling layer. The fully connected layer uses a Softmax function. The method includes the following steps:

[0009] Step 1: Obtain a sentence pair dataset, extract sentence pairs from the dataset, preprocess the sentence pairs, and obtain text sentence pairs after word segmentation;

[0010] Step 2: Extract the preliminary semantic features and deep semantic features of the sentence pairs in the DeBERTa model and the recursive convolutional neural network model;

[0011] Step 3: Use the Attention layer to generate the weight of the sentence pair feature vector based on the deep semantic features of the sentence pair;

[0012] Step 4: Input the weight of the sentence pair feature vector into the fully connected layer to generate the similarity of the sentence pair.

[0013] Furthermore, in step one:

[0014] The sentence pair dataset includes several text data, each of which contains a pair of sentences separated by ",";

[0015] Preprocessing includes removing problematic sentence pairs, part-of-speech tagging, word segmentation, and vectorization.

[0016] Furthermore, step 2 includes the following steps:

[0017] Step 2.1: Input the preprocessed sentence pairs into the DeBERTa model, embed the sentence pairs through DeBERTa to obtain dynamic word vectors, and extract the preliminary semantic features of the sentence pairs; the DeBERTa model uses the decoupled attention mechanism DAM to represent each word with two vectors, encode its content and relative position respectively, and then calculate the weight according to the content and relative position of the word through the Transformer's Self-attention mechanism;

[0018] Step 2.2: Input the word vector obtained in step 2.1 into the recursive convolutional neural network model to extract the deep semantic features of the sentence pair; the recursive convolutional neural network model includes a bidirectional long short-term memory network BI-LSTM, a CNN convolutional layer and an adaptive pooling layer.

[0019] Furthermore, in step 2.2, the deep semantic feature generation method is:

[0020] After the word vector is output by the concatenated bidirectional LSTM model, it is input into the CNN convolution layer. The CNN convolution kernel trains the feature vector to obtain the deep semantic feature vector. The deep semantic feature vector obtained by the CNN convolution layer enters the adaptive pooling layer. In the adaptive pooling layer, the weight of each position is adjusted according to the different local areas in the input feature channel. After the CNN convolution layer, the adaptive pooling layer uses the classification network to extract the high-level semantic information F, F∈R, of the last layer of the feature extraction layer. H×W×C , H, W, C are the height, width, and depth of the image respectively; for each feature map F of H×W dimension i , F i ∈F, a 1×K×K convolution kernel is used to obtain the information of each position and its surrounding positions in turn to generate the importance weight of the position.

[0021] Furthermore, in step 4, the method for calculating the semantic similarity of Chinese sentence pairs is:

[0022] The Softmax function is used to normalize the deep semantic features of the sentence pairs, the Pearson coefficient between the deep semantic features is calculated, and the cosine similarity is used to calculate the similarity of the sentence pairs:

[0023]

[0024] Among them, a i 、b i Represents the deep semantic feature vector of each word in the first sentence and the second sentence in the sentence pair respectively;

[0025] The similarity of the sentences is used to determine whether the sentences are similar, with 1 being similar and 0 being dissimilar.

[0026] Furthermore, the semantic similarity network model also includes an optimization module, which includes an Adam module and a dropout module, and the semantic similarity network model is optimized by the optimization module.

[0027] Compared with the prior art, the present invention has the following beneficial effects:

[0028] 1) In response to the problem that traditional maximum pooling cannot always obtain effective information, the present invention proposes a weight algorithm for word vectors, and uses adaptive pooling to replace the maximum pooling method in the recursive convolutional neural network, assigning different importance weight indicators to different positions of the vector, which can better reflect the original sentence input and improve the performance of the recursive convolutional neural network model, which is better than the traditional maximum pooling method.

[0029] 2) In order to address the difficulty in reasonably allocating feature vector weights, the Attention mechanism is introduced after the pooling layer to calculate and encode the weights. The Attention layer is used to assign corresponding weights to sentence features, so that the model can focus on the key information in the data during training, reduce the attention to unimportant and low-relevance information in the text, extract more direct semantic dependencies in the sentence, better allocate the weights of deep semantic feature vectors, and improve the accuracy of semantic similarity calculation. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 Pre-training language model structure for BERT;

[0031] Figure 2 It is the DeBERTa pre-trained language model structure in the present invention;

[0032] Figure 3 Flowchart for calculating semantic similarity of text sentence pairs;

[0033] Figure 4 The overall framework of the model. DETAILED DESCRIPTION

[0034] The technical solution of the present invention is described in detail below through embodiments, but the protection scope of the present invention is not limited to the embodiments.

[0035] This paper proposes a Chinese sentence semantic similarity calculation method based on recursive convolutional neural network. A pair of preprocessed sentence pairs are input into the semantic similarity network model to calculate their similarity and determine whether they are repeated sentence pairs. The similarity of the sentence pairs in the data set is judged and the results are analyzed and discussed.

[0036] The semantic similarity calculation method is based on a semantic similarity network model for calculation. The semantic similarity network model is as follows: Figure 4 As shown, it includes a DeBERTa model, a recursive convolutional neural network model, an Attention layer, a fully connected layer, and an optimization module; the maximum pooling layer in the recursive convolutional neural network model is replaced by an adaptive pooling layer; the fully connected layer adopts a Softmax function; and the optimization module includes a dropout module and an Adam module.

[0037] The semantic similarity calculation method is as follows: Figure 3 As shown, the specific steps are as follows:

[0038] Step 1: Obtain a sentence pair dataset, extract sentence pairs from the dataset, preprocess the sentence pairs, and obtain text sentence pairs after word segmentation;

[0039] 1) The sentence pair dataset includes thousands of text data, each of which contains a pair of sentences separated by ",".

[0040] 2) Extract sentence pairs from each data and preprocess the sentence pairs, including removing problematic sentence pairs, part-of-speech tagging, word segmentation, and vectorization.

[0041] Part-of-speech tagging and word segmentation are both basic tasks in natural language processing that are relatively low-level and have relatively mature technologies. They can be easily implemented using existing tools. In this embodiment, the jieba tool is used to complete the part-of-speech tagging of Chinese, and the Pangu word segmentation tool is used to perform word segmentation on Chinese sentence pairs. After word segmentation, the sentence pairs obtained perform WordEmbedding vectorization processing on each word to obtain the sentence pair vector features.

[0042] Step 2: In the DeBERTa model and the recursive convolutional neural network model, extract the preliminary semantic features and deep semantic features of the sentence pairs. The specific steps are as follows:

[0043] Step 2.1: Input the preprocessed sentence pairs into the DeBERTa model, embed the sentence pairs into dynamic word vectors through DeBERTa, and extract the preliminary semantic features of the sentence pairs;

[0044] like Figure 1 As shown in the figure, in the traditional BERT model, each word is represented by only one vector, which is just the sum of the word’s content embedding (Token Embeddings), segment embedding (Segment Embeddings), and position embedding (PositionEmbeddings).

[0045] like Figure 2 As shown, the DeBERTa model proposed in the present invention uses the decoupled attention mechanism DAM to represent each word with two vectors, encodes its content and relative position respectively, and then calculates the weight according to the content and relative position of the word through the Self-attention mechanism of Transformer, thereby increasing the calculation between content and position and further emphasizing the correlation between content and position.

[0046] 1) The calculation process of the decoupled attention input representation of the DeBERTa model is as follows;

[0047]

[0048] Among them, H i , H j represents the content embedding of the i-th and j-th words, P i|j represents the relative position embedding of word i and word j, represents content-to-content computation, and Represents the calculation from content to location and location to content, Represents a position-to-position calculation.

[0049] 2) The relative distance calculation formula between the i-th and j-th words in the decoupled attention mechanism of the DeBERTa model is as follows:

[0050]

[0051] Wherein, k represents the maximum relative distance.

[0052] 3) The calculation formula of the decoupled attention mechanism of the DeBERTa model is as follows:

[0053] Q c =HW q,c ,K c =HW k,c ,V c =HW v,c

[0054] Q r =PW q,r ,K r =PW k,r

[0055]

[0056] Among them, H is the input information, P is the relative position information, Q c ,K c, V c For content embedding, Q r ,K r is the relative position embedding, W q,c ,W k,c ,W v,c is the content weight matrix, W q,r ,W k,r is the relative position weight matrix, is the Q of the i-th word c , is the K of the jth word c , K from i to j r , Q for j to i r , Τ is the transposed,attention matrix, is the attention matrix, for The element H o To output information.

[0057] Step 2.2: Input the word vector obtained in step 2.1 into the recursive convolutional neural network model to extract the deep semantic features of the sentence pair;

[0058] The recursive convolutional neural network model includes a bidirectional long short-term memory network BI-LSTM, a CNN convolutional layer and an adaptive pooling layer.

[0059] The method for generating deep semantic feature vectors is as follows: after the word vector is spliced ​​with the output of the bidirectional LSTM model, it is input into the CNN convolution layer, and the CNN convolution kernel trains the feature vector to obtain the deep semantic feature vector; the deep semantic feature vector obtained by the CNN convolution layer enters the adaptive pooling layer, and in the adaptive pooling layer, the weight of each position is adjusted according to the different local areas in the input feature channel.

[0060] Maximum pooling only takes the maximum value of the features in the pooling domain, which will affect the accuracy of text feature extraction, thereby affecting the subsequent classification work. Therefore, the present invention adopts an adaptive pooling method to improve the accuracy of text feature extraction by calculating the importance weight index of different positions in the feature map.

[0061] Adaptive pooling method:

[0062] After the CNN convolutional layer, the adaptive pooling layer uses the classification network to extract the high-level semantic information F, F∈R, to the last layer of the feature extraction layer. H×W×C , H, W, C are the height, width, and depth of the image respectively. For each feature map F of H×W dimensions i , F i ∈F, a 1×K×K convolution kernel is used to obtain the information of each position and its surrounding positions in turn to generate the importance weight of the position. The specific steps are as follows:

[0063] 1) For the size of the convolution kernel K, convolution kernels of different sizes are used for processing feature maps of different sizes to obtain the weights of each position of different text channels. The relationship between the size of the convolution kernel and the number of channels is as follows:

[0064]

[0065] Among them, H is the height, W is the width, C is the number of channels, and K is the size of the convolution kernel.

[0066] 2) Perform Instance Normalization on the same channel in each feature map, which is beneficial for processing the results of our deep convolution. By calculating the mean and variance of each channel, the distribution of the results is changed to make the distribution more consistent, which can get better results. The calculation formula of Instance Normalization is as follows:

[0067]

[0068] Among them, u is the mean, σ 2 is the variance, t is the index of the input feature, i is the index of the corresponding channel, x is the input segmented image, and m is the output after segmentation. ti represents the mean, Represents variance.

[0069] 3) After the importance degree weight calculation, each channel is normalized after instance normalization, sigmoid activation, and amplification of the difference to generate the final weight M F ∈R H×W×C .

[0070] In the recursive convolutional neural network model, the original maximum pooling layer is replaced by an adaptive pooling layer. The adaptive pooling layer adjusts the weight of each position according to different local areas in the input feature channel, and can generate an output feature map of fixed size, thereby improving the performance of the recursive convolutional neural network model. In the adaptive pooling layer, different channels of the feature map can independently extract local features and perform feature conversion. The adaptive pooling layer uses a deep convolution operation with a combination of convolution kernels of different sizes. The number of convolution kernels is the same as the number of channels of the input feature map, and each convolution kernel only processes one channel. The advantage of this is not only to reduce the introduction of parameters and the amount of calculation, but also from the results, the final weight is generated by independently capturing the area on the local feature map on each feature channel, which is better than the effect of calculating and linearly combining on multiple channels at the same time. The adaptive pooling layer adjusts the weight of each position according to different local areas in the input feature channel. This strategy can further utilize the learning ability of the network, and can dynamically adjust the weights of different local features in the text classification task, which is of obvious benefit to the classification of text and the use of classification networks to generate more accurate location information.

[0071] Step 3: Use the Attention layer to generate the weight of the sentence pair feature vector based on the deep semantic features of the sentence pair;

[0072] Use the Attention layer to assign corresponding weights to sentence features. In order to enable the recursive convolutional neural network to focus on the key information in the data during training, reduce the attention to unimportant low-relevance information in the text, and extract more direct semantic dependencies in the sentence, the Attention mechanism is introduced after the RCNN network. The Attention mechanism calculates the weights of different feature vectors and encodes the text sentences. Attention mechanism calculation method:

[0073] e i=v i tanh(W i h i +b i )

[0074]

[0075] Among them, v i represents the parameter vector; W i represents the weight matrix; b i represents the bias vector; h i Represents the hidden vector of the input. i 、e j represents the weight of the assignment, α i is the degree of association, and S represents the Attention output.

[0076] Step 4: Input the weight of the sentence pair feature vector into the fully connected layer to generate the similarity of the sentence pair;

[0077] The input fully connected layer calculates the similarity of sentence pairs through Softmax to determine whether the sentences are similar or not;

[0078] The method for calculating the semantic similarity of Chinese sentence pairs is:

[0079] The deep semantic feature vectors of the sentence pairs are normalized using the Softmax function, the Pearson coefficient between the deep semantic feature vectors is calculated, and the cosine similarity is used to calculate the similarity of the sentence pairs:

[0080]

[0081] Among them, a i 、b i Represent the deep semantic feature vectors of each word in the first sentence and the second sentence in the sentence pair respectively.

[0082] The similarity of the sentences is determined based on the similarity of the sentence pairs. The judgment threshold is not strictly defined and can be adjusted freely. In this embodiment, 0.6 is set as the threshold. The similarity judgment is 1, and the dissimilarity judgment is 0.

[0083] Step 5: Optimize the semantic similarity network model through the optimization module;

[0084] The Adam module is also added to the semantic similarity network model to optimize the model and reduce the loss parameters of the model. In addition, the semantic similarity network model of the present invention continuously adjusts the learning rate, debugs various parameters, and adds a dropout module to randomly discard a part of neurons to reduce the dependence on training samples, ultimately achieving the goal of improving model performance.

[0085] The ChineseSTS dataset was used to experiment with the model of the present invention and the existing model, and the word segmentation tool was used to segment the sentence pairs. In the model of the present invention, the parameter settings are shown in the following figure:

[0086] Table 1 Simulation parameters

[0087]

[0088] The experiment randomly splits the labeled dataset in a ratio of 7:3 to obtain the training set and the test set. The data used in this experiment include the short text pair dataset from ChineseSTS, the medical field dataset, and the Alipay financial sentence pair dataset.

[0089] The model was experimented with three public datasets from different fields. Each dataset compared the method with the existing model. The results are shown in the following figure:

[0090] Table 2 Short text dataset training results

[0091]

[0092] Table 3. Training results of medical dataset

[0093]

[0094] Table 4 Training results of Alipay financial sentence pair dataset

[0095]

[0096] From the above results, it can be seen that the DeBERTa-RCNN-Attention (adaptive pooling) model (model of the present invention) has the highest precision, accuracy and recall in the three data sets.

[0097] By comparing the components of each model, it is found that the results of the model seem to be greatly affected by the training method. The BERT model and RoBERTa are trained on massive data, and they can obtain part of the semantic information of the sentence from pre-training, but lack the extraction of interactive features and weight components, as well as the lack of specialized training for specific training samples, which will affect the calculation and discrimination of the similarity of sentences to texts in the characteristic field.

[0098] The recursive convolutional neural network (RCNN) that relies on the DeBERTa language model and adaptive pooling performs word segmentation preprocessing on the two input sentences, inputs them into the DeBERTa language model, and generates two dynamic word vectors for each word in each sentence based on its content and relative position. These two vector sets are then input into the bidirectional long short-term memory network (Bi-LSTM) for forward and backward training to obtain a semantic feature vector, which is then input into the CNN for convolution training to obtain a deep semantic feature vector. The Attention mechanism is then used to assign different weights based on the deep semantic feature vector, and the generated weight vector is used to calculate the similarity of the sentence pairs. From the results, it can be seen that DeBERTa-RCNN-Attention (adaptive pooling) can better understand the interaction and semantic correlation features between sentence pairs, thereby better judging whether two sentences are similar.

[0099] As described above, although the present invention has been shown and described with reference to specific preferred embodiments, it should not be construed as limiting the present invention itself. Various changes in form and details may be made without departing from the spirit and scope of the present invention.

Claims

1. A method for calculating the semantic similarity of Chinese sentences based on recursive convolutional neural network, characterized in that: The method is based on a semantic similarity network model for calculation, and the semantic similarity network model includes a DeBERTa model, a recursive convolutional neural network model, an Attention layer and a fully connected layer; the maximum pooling layer in the recursive convolutional neural network model is replaced by an adaptive pooling layer; the fully connected layer adopts a Softmax function; the method includes the following steps: Step 1: Obtain a sentence pair dataset, extract sentence pairs from the dataset, preprocess the sentence pairs, and obtain text sentence pairs after word segmentation; Step 2: Extract the preliminary semantic features and deep semantic features of the sentence pairs in the DeBERTa model and the recursive convolutional neural network model; Step 3: Use the Attention layer to generate the weight of the sentence pair feature vector based on the deep semantic features of the sentence pair; Step 4: Input the weight of the sentence pair feature vector into the fully connected layer to generate the similarity of the sentence pair.

2. The calculation method according to claim 1, characterized in that: In step one: The sentence pair dataset includes several text data, each of which contains a pair of sentences separated by ",". The preprocessing includes removing problematic sentence pairs, part-of-speech tagging, word segmentation, and vectorization.

3. The calculation method according to claim 1, characterized in that: Step 2 includes the following steps: Step 2.1: Input the preprocessed sentence pairs into the DeBERTa model, embed the sentence pairs into dynamic word vectors through DeBERTa, and extract the preliminary semantic features of the sentence pairs; The DeBERTa model uses the decoupled attention mechanism DAM to represent each word with two vectors, encoding its content and relative position respectively, and then calculates the weight through the Transformer's Self-attention mechanism according to the content and relative position of the word; Step 2.2: Input the word vector obtained in step 2.1 into the recursive convolutional neural network model to extract the deep semantic features of the sentence pair; The recursive convolutional neural network model includes a bidirectional long short-term memory network BI-LSTM, a CNN convolutional layer and an adaptive pooling layer.

4. The calculation method according to claim 3, characterized in that: In step 2.2, the deep semantic feature generation method is: After the word vector is output by the spliced ​​bidirectional LSTM model, it is input into the CNN convolution layer. The CNN convolution kernel trains the feature vector to obtain a deep semantic feature vector. The deep semantic feature vector obtained by the CNN convolution layer enters the adaptive pooling layer. In the adaptive pooling layer, the weight of each position is adjusted according to the different local areas in the input feature channel. After the CNN convolutional layer, the adaptive pooling layer uses the classification network to extract the high-level semantic information F, F∈R, to the last layer of the feature extraction layer. H×W×C , H, W, C are the height, width, and depth of the image respectively; for each feature map F of H×W dimension i , F i ∈F, a 1×K×K convolution kernel is used to obtain the information of each position and its surrounding positions in turn to generate the importance weight of the position.

5. The calculation method according to claim 1, characterized in that: In step 4, the semantic similarity calculation method of Chinese sentence pairs is: The Softmax function is used to normalize the deep semantic features of the sentence pairs, the Pearson coefficient between the deep semantic features is calculated, and the cosine similarity is used to calculate the similarity of the sentence pairs: Among them, a i , b i Represents the deep semantic feature vector of each word in the first sentence and the second sentence in the sentence pair respectively; The similarity of the sentences is used to determine whether the sentences are similar, with 1 for similarity and 0 for dissimilarity.

6. The calculation method according to claim 1, characterized in that: The semantic similarity network model also includes an optimization module, which includes an Adam module and a dropout module. The semantic similarity network model is optimized by the optimization module.

Images