Semantic retrieval method based on multi-feature fusion

Abstract

The invention discloses a semantic retrieval method based on multi-feature fusion, which is based on artificial intelligence and large model technology, and comprises the following steps of: firstly, cleaning words after word segmentation of a jieba model by using word frequency and inverse document frequency to improve semantic search efficiency; the method comprises the steps that firstly, a user search statement and words in a target text are subjected to entity recognition in combination with a BERT model, a long-short term memory network and a conditional random field model, finally, text similarity calculation is conducted from the four feature dimensions of keywords, word forms, word orders and word vectors, the text with the similarity larger than a threshold value is returned, and the semantic search accuracy is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and more specifically to a semantic retrieval method based on multi-feature fusion. Background Art

[0002] With the rapid advancement of digital transformation, the way of answering questions in tender announcements has been continuously upgraded. Traditional Q&A requires manual answers one by one, but with the introduction of natural language processing (NLP) technology, intelligent Q&A has become more important. This method can automatically answer based on the knowledge base, thereby providing a faster response speed and reducing the pressure of manual Q&A work. However, the current intelligent question system is mainly based on matching technology, which automatically identifies questions for matching and returns set answers, but cannot answer when the consulting question is out of range. In addition, different descriptions of the same question will reduce the matching rate with the knowledge base, and the research on the semantic understanding of the question is relatively insufficient. After the emergence of large language models, the level of understanding of long articles and complex contexts has been significantly improved, and the ability to automatically summarize and summarize knowledge has been comprehensively strengthened, making the intelligent question-answering library based on language models a feasible and more effective choice. To solve this problem, this patent combines a large model to propose an intelligent question-answering system for tender document question-answering scenarios, improve the matching rate of consulting questions, improve the accuracy of intelligent questions and answers, and improve the efficiency of bidding. Summary of the invention

[0003] In order to solve the above problems, the present invention provides a semantic retrieval method based on multi-feature fusion. The method is based on artificial intelligence and large model technology. First, the word frequency and inverse document frequency are used to clean the words after word segmentation by the Jieba model to improve the efficiency of semantic search. Then, the BERT model, long short-term memory network and conditional random field model are combined to perform entity recognition on the words in the user search sentence and the target text. Finally, the similarity of the text is calculated from the four feature dimensions of keywords, word form, word order and word vector, and the text with a similarity greater than a threshold is returned to improve the accuracy of semantic search.

[0004] The present invention discloses a semantic retrieval method based on multi-feature fusion, which comprises the following steps:

[0005] Step 1: Use the Jieba model to segment Chinese words, calculate the weights of the word frequency and inverse document frequency after segmentation, and then clean them according to the weights to improve the retrieval quality;

[0006] Step 2: Combine the BERT model, long short-term memory network and conditional random field model to perform entity recognition on the words in the user's search sentence and the target text to determine their semantics;

[0007] Step 3: Calculate the similarity of texts from four feature dimensions: keywords, word forms, word order, and word vectors, and return texts with similarities greater than a threshold to improve the accuracy of semantic search.

[0008] In some embodiments, in step 1, the Jieba model is used to segment Chinese words, and the segmented words are cleaned according to weights to improve the retrieval quality. The present invention combines word frequency and inverse document frequency to calculate the weight of words, where word frequency represents the frequency of the word in the text. The greater the frequency, the more important the word is to the text. The word frequency expression formula is:

[0009]

[0010] Where i represents a word, j represents a text, n represents the total number of all texts, and tf ij represents the frequency of word i appearing in text j,

[0011] Inverse document frequency means that words that appear in most texts are not as important as words that appear in only a small part of the text. This method can weaken the importance of high-frequency words that appear in most texts, while enhancing the importance of low-frequency words that appear in a small part of the text. Its formula is expressed as:

[0012]

[0013] Where i represents a word, n represents the total number of all texts, and df i Indicates the number of texts in which word i appears.

[0014] Finally, the word weight calculation formula is:

[0015]

[0016] For each word i, the larger the Wei ght value is, the more important the word is and the greater the impact on the results of semantic retrieval is. Words with weight values ​​greater than the threshold are retained.

[0017] In some embodiments, in step 2: entity recognition is performed on words in the text and words summarized in user search sentences to determine their semantics. The present invention combines the BERT model, the long short-term memory network (BiLSTM) and the conditional random field model (CRF) for entity recognition. BERT first uses pre-training of a large-scale corpus to understand the bidirectional contextual relationship of sentences at the text level and converts the text into a high-dimensional word vector representation. Then, BiLSTM performs sequence modeling on the output of BERT to capture the long-distance dependencies in the question, thereby more comprehensively understanding the semantic information. Finally, CRF performs label dependency modeling on the basis of BiLSTM to further improve the accuracy and robustness of named entity recognition.

[0018] In some embodiments, based on the search text, the similarity between the search text and the target text is calculated. The present invention performs similarity calculation based on four feature dimensions: keywords, word form, word order, and word vector. The keyword similarity indicates the probability of the keywords appearing together in the two texts. If a keyword appears in both texts, it is recorded once. Assume that the keyword in text 1 is T 1 =(k 1 , k 2 , …, k m ), the keyword in text 2 is T 2 =(k 1 , k 2 , …, k n ), len(T 1 ) is the number of keywords that appear in text 1, len(T 2 ) is the number of keywords appearing in text 2, CKW(T 1 ∪T 2 ) represents the number of common keywords between two texts, and the keyword similarity calculation formula is:

[0019]

[0020] Word form similarity indicates the similarity between two texts in word form, and aims to find the co-occurring words between two texts. Assume that the word contained in text 1 is R 1 =(w 1 , w 2 ,…,wx), the word R contained in text 2 2 =(w 1 , w 2 ,…,w n ),lem(T 1 ) is the number of words in text 1, lem(T 2 ) is the number of words in text 2, CW(R 1 ∪R 2 ) represents the number of co-occurring words between two texts, then the formula for calculating word form similarity is:

[0021]

[0022] Word order similarity indicates the relative position of words in two texts. Assume that lem(T 1 ) is the number of words in text 1, lem(T 2 ) is the number of words in text 2, mc is the number of words in text T 1 and T 2 The number of matched characters, move is the number of transpositions, and the formula for calculating word order similarity is:

[0023]

[0024] Word vector similarity calculates the similarity of two texts based on the overlap of word vectors, where the word vector w 1 and w 2 The cosine similarity calculation formula is:

[0025]

[0026] Where I and R are the user's current input text and the target text respectively, and max(α*CosDis(w, R)) is the maximum value of the cosine similarity between the word vectors corresponding to all words in the text R and the word vector corresponding to w. The parameter α is used to adjust the amplification factor of the cosine similarity between two word vectors, which can be set and adjusted according to the different word vector tables obtained.

[0027] Combining the above-mentioned four features and weighting them linearly, we can get the multi-feature text structure similarity, and the calculation formula is:

[0028] sim str =A*sim 1 +B*sim 2 +C*sim 3 +D*sim 4

[0029] Among them, A, B, C, and D are the weights of the four features, satisfying A+B+C+D=1.

[0030] Finally, the similarity is judged. If the similarity is str If the similarity is greater than the set threshold, it means that the two texts are similar and the answer is returned to the user; otherwise, the two texts are considered to be dissimilar.

[0031] Compared with the prior art, the present invention has the following beneficial effects:

[0032] The invention is based on artificial intelligence and large model technology. Firstly, the word frequency and inverse document frequency are used to clean the words after word segmentation by the Jieba model to improve the efficiency of semantic search. Then, the BERT model, long short-term memory network and conditional random field model are combined to perform entity recognition on the words in the user search sentence and the target text. Finally, the similarity of the text is calculated from the four feature dimensions of keywords, word form, word order and word vector, and the text with similarity greater than the threshold is returned to improve the accuracy of semantic search. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 It is an implementation flow chart of the present invention. DETAILED DESCRIPTION

[0034] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0035] The present invention is based on artificial intelligence and large model technology. First, the word frequency and inverse document frequency are used to clean the words after word segmentation by the Jieba model to improve the efficiency of semantic search. Then, the BERT model, long short-term memory network and conditional random field model are combined to perform entity recognition on the words in the user search sentence and the target text. Finally, the similarity of the text is calculated from the four feature dimensions of keywords, word form, word order and word vector, and the text with similarity greater than the threshold is returned to improve the accuracy of semantic search. Specifically:

[0036] First, the Jieba model is used to segment Chinese words, and the segmented words are cleaned according to the weights to improve the retrieval quality. The present invention combines word frequency and inverse document frequency to calculate the weight of words, where word frequency indicates the frequency of the word appearing in the text. The greater the frequency, the more important the word is to the text. The word frequency expression formula is:

[0037]

[0038] Where i represents a word, j represents a text, n represents the total number of all texts, and tf ij It represents the frequency of word i appearing in text j.

[0039] Inverse document frequency means that words that appear in most texts are not as important as words that appear in only a small part of the text. This method can weaken the importance of high-frequency words that appear in most texts, while enhancing the importance of low-frequency words that appear in a small part of the text. Its formula is expressed as:

[0040]

[0041] Where i represents a word, n represents the total number of all texts, and df i Indicates the number of texts in which word i appears.

[0042] Finally, the word weight calculation formula is:

[0043]

[0044] For each word i, the larger the Weight value, the more important the word is and the greater the impact on the results of semantic retrieval. Words with weight values ​​greater than the threshold are retained.

[0045] Then, entity recognition is performed on the words in the text and the words summarized by the user search sentences to determine their semantics. The present invention combines the BERT model, the long short-term memory network (BiLSTM) and the conditional random field model (CRF) for entity recognition. BERT first uses the pre-training of a large-scale corpus to understand the bidirectional contextual relationship of the sentence at the text level and convert the text into a high-dimensional word vector representation. Then, BiLSTM performs sequence modeling on the output of BERT, effectively capturing the long-distance dependencies in the question, thereby more comprehensively understanding the semantic information. Finally, CRF performs label dependency modeling on the basis of BiLSTM, further improving the accuracy and robustness of named entity recognition.

[0046] Again, based on the search text, the similarity between it and the target text is calculated. The present invention calculates similarity from four feature dimensions: keywords, word form, word order, and word vector. The keyword similarity indicates the probability of keywords appearing together in two texts. If a keyword appears in both texts, it is recorded once. Assume that the keyword in text 1 is T 1 =(k 1 , k 2 , …, k m ), the keyword in text 2 is T 2 =(k 1 , k 2 , …, k n ), len(T 1 ) is the number of keywords that appear in text 1, len(T 2 ) is the number of keywords appearing in text 2, CKW(T 1 ∪T 2 ) represents the number of common keywords between two texts, and the keyword similarity calculation formula is:

[0047]

[0048] Word form similarity indicates the similarity between two texts in word form, and aims to find the co-occurring words between two texts. Assume that the word contained in text 1 is R 1 =(w 1 , w 2 ,…,w m ), the word R contained in text 2 2 =(w 1 , w 2 ,…,w n ),lem(T 1 ) is the number of words in text 1, lem(T 2 ) is the number of words in text 2, CW(R 1 ∪R 2) represents the number of co-occurring words between two texts, then the formula for calculating word form similarity is:

[0049]

[0050] Word order similarity indicates the relative position of words in two texts. Assume that lem(T 1 ) is the number of words in text 1, lem(T 2 ) is the number of words in text 2, mc is the number of words in text T 1 and T 2 The number of matched characters, move is the number of transpositions, and the formula for calculating word order similarity is:

[0051]

[0052] Word vector similarity calculates the similarity of two texts based on the overlap of word vectors, where the word vector w 1 and w 2 The cosine similarity calculation formula is:

[0053]

[0054] Where I and R are the user's current input text and the target text respectively, and max(α*CosDis(w, R)) is the maximum value of the cosine similarity between the word vectors corresponding to all words in the text R and the word vector corresponding to w. The parameter α is used to adjust the amplification factor of the cosine similarity between two word vectors, which can be set and adjusted according to the different word vector tables obtained.

[0055] Combining the above-mentioned four features and weighting them linearly, we can get the multi-feature text structure similarity, and the calculation formula is:

[0056] sim str =A*sim 1 +B*sim 2 +C*sim 3 +D*sim 4

[0057] A, B, C, and D are weights of the four features, satisfying A+B+C+D=1. In this implementation, the weights are set to 0.2, 0.15, 0.15, and 0.5.

[0058] Finally, the similarity is judged. If the similarity is str If the similarity is greater than the set similarity threshold, which is 0.25 in this implementation, it means that the two texts are similar, and the answer is returned to the user; otherwise, the two texts are considered to be dissimilar.

[0059] The above are only some embodiments of the present invention. It should be pointed out that for ordinary technicians in this field, other modifications and improvements can be made without departing from the creative concept of the present invention, which all belong to the protection scope of the present invention.

Claims

1. A semantic retrieval method based on multi-feature fusion, characterized in that: The method comprises the following steps: Step 1: Use the Jieba model to segment Chinese words, calculate the weights of the word frequency and inverse document frequency after segmentation, and then clean them according to the weights to improve the retrieval quality; Step 2: Combine the BERT model, long short-term memory network and conditional random field model to perform entity recognition on the words in the user's search sentence and the target text to determine their semantics; Step 3: Calculate the similarity of texts from four feature dimensions: keywords, word forms, word order, and word vectors, and return texts with similarities greater than a threshold to improve the accuracy of semantic search.

2. The method according to claim 1, characterized in that: In step 1, the weight of the word is calculated by combining the word frequency and the inverse document frequency, where the word frequency indicates the frequency of the word in the text. The greater the frequency, the more important the word is to the text. The word frequency expression formula is: Where i represents a word, j represents a text, n represents the total number of all texts, and tf ij represents the frequency of word i appearing in text j, Inverse document frequency means that words that appear in most texts are not as important as words that appear in only a small part of the text. This method can weaken the importance of high-frequency words that appear in most texts, while enhancing the importance of low-frequency words that appear in a small part of the text. Its formula is expressed as: Where i represents a word, n represents the total number of all texts, and df i Indicates the number of texts in which word i appears; Finally, the word weight calculation formula is: For each word i, the larger the Weight value, the more important the word is and the greater the impact on the results of semantic retrieval. Words with weight values ​​greater than the threshold are retained.

3. The method according to claim 1, characterized in that In step 2: entity recognition is performed on the words in the text and the words summarized in the user search sentences to determine their semantics. The present invention combines the BERT model, the long short-term memory network (BiLSTM) and the conditional random field model (CRF) for entity recognition. BERT first uses the pre-training of a large-scale corpus to understand the bidirectional contextual relationship of the sentence at the text level and converts the text into a high-dimensional word vector representation. Then, BiLSTM performs sequence modeling on the output of BERT to capture the long-distance dependency relationship in the question sentence, thereby more comprehensively understanding the semantic information. Finally, CRF performs label dependency modeling on the basis of BiLSTM to further improve the accuracy and robustness of named entity recognition.

4. The method according to claim 1, characterized in that In step 3, based on the search text, the similarity between it and the target text is calculated. The present invention performs similarity calculation from four feature dimensions: keywords, word form, word order and word vector. The keyword similarity represents the probability of the keywords appearing together in the two texts. If a keyword appears in both texts, it is recorded once. Assume that the keywords in text 1 are T1=(k1, k2, ..., k m ), the keywords in text 2 are T2=(k1, k2, ..., k n ), len(T1) is the number of keywords in text 1, len(T2) is the number of keywords in text 2, CKW(T1∪T2) represents the number of common keywords between the two texts, then the keyword similarity calculation formula is: Word form similarity indicates the similarity between two texts in word form, and aims to find the co-occurring words between two texts. Assume that the words contained in text 1 are R1 = (w1, w2, ..., w m ), the words contained in text 2 are R2=(w1,w2,…,w n ), lem(T1) is the number of words in text 1, lem(T2) is the number of words in text 2, CW(R1∪R2) represents the number of co-occurring words between the two texts, then the formula for calculating word form similarity is: Word order similarity indicates the relative position of words in two texts. Assuming that lem(T1) is the number of words in text 1, lem(T2) is the number of words in text 2, mc is the number of characters matched between texts T1 and T2, and move is the number of transpositions, the formula for calculating word order similarity is: Word vector similarity calculates the similarity of two texts based on the overlap of word vectors. The cosine similarity calculation formula of word vectors w1 and w2 is: Where I and R are the user's current input text and the target text respectively. max(α*CosDis(w, R)) is the maximum value of the cosine similarity between the word vectors corresponding to all words in the text R and the word vector corresponding to w. The parameter α is used to adjust the amplification factor of the cosine similarity between two word vectors, which can be set and adjusted according to the different word vector tables obtained. Combining the above-mentioned four features and weighting them linearly, we can get the multi-feature text structure similarity, and the calculation formula is: sim str =A*sim1+B*sim2+C*sim3+D*sim4 Among them, A, B, C, and D are the weights of the four features, satisfying A+B+C+D=1. Finally, the similarity is judged. If the similarity is str If the similarity is greater than the set threshold, it means that the two texts are similar and the answer is returned to the user; otherwise, the two texts are considered to be dissimilar.

Images