Ancient poetry semantic retrieval method based on Seq2Seq and DBSCAN

Abstract

The invention discloses an ancient poetry semantic retrieval method based on Seq2Seq and DBSCAN, and the method comprises the steps: generating an ancient poetry sequence into a white text based on a Seq2Seq model and an LSTM network; secondly, based on a pre-constructed dictionary, segmenting the translated text through a reverse maximum matching algorithm, and constructing an ancient poetry semantic feature vector library; generating semantic feature vectors of query statements on the basis of poetry query input by a user, extracting continuous phrases in the query statements and the semantic features of the ancient poetry by adopting a cosine similarity algorithm based on N-Gram and a dynamic sliding window, calculating the similarity, and retrieving the ancient poetry closest to the query of the user; and finally, carrying out clustering analysis on the semantic feature vectors by utilizing a DBSCAN algorithm, and expanding and recommending similar ancient poetry. The problem that a traditional keyword matching method cannot accurately understand deep semantics of poems is solved, efficient and accurate personalized recommendation can be achieved, and retrieval efficiency is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of natural language processing and information retrieval, and particularly relates to a method for semantic retrieval of ancient Chinese poems based on Seq2Seq and DBSCAN. Background Art

[0002] As a treasure of Chinese culture, ancient Chinese poems contain rich cultural connotations and artistic values. With the development of information technology and digital technology, the digital resources of ancient Chinese poems have shown exponential growth. How to quickly and accurately retrieve the content that meets the user's needs from a huge database of ancient Chinese poems has become a major challenge in current research.

[0003] Currently, traditional methods for retrieving ancient Chinese poems mainly rely on keyword matching technology, that is, the query keywords input by the user are matched with the words appearing in the ancient Chinese poem text. However, this method has several limitations. Especially in the retrieval of ancient Chinese poems, the language has a high degree of conciseness and metaphor. Traditional keyword matching cannot understand the deep semantics of poems, cannot recognize different expressions, and is prone to irrelevant or inaccurate retrieval results. In addition, keyword matching cannot capture the context relationship of ancient Chinese poems and ignores the artistic conceptions and emotions contained in the poems. Therefore, this method cannot meet the user's requirements for high precision and high efficiency when dealing with complex language structures and large-scale data.

[0004] In recent years, natural language processing technologies based on machine learning and deep learning have gradually been introduced into the retrieval of ancient Chinese poems. Some studies have tried to use neural network models for semantic representation of poems, improving the accuracy of retrieval. However, these methods still have some deficiencies. In the prior art, deep learning models such as Seq2Seq model or BERT are usually used alone to process poem texts. Although it can improve the accuracy of semantic understanding, it often ignores the precise recommendation of the user's personalized needs. In addition, deep learning-based models usually have high requirements for computing resources, and the training and inference speeds are slow, making it difficult to meet the needs of users for real-time retrieval and recommendation. Summary of the Invention

[0005] Aiming at the problems existing in the prior art, the present invention provides a method for semantic retrieval of ancient Chinese poems based on Seq2Seq and DBSCAN. Through the innovative integration of Seq2Seq and DBSCAN algorithms, it can not only efficiently understand the semantics of poems, but also provide personalized clustering recommendations based on semantic features, overcoming the deficiencies of the prior art in personalized recommendation and retrieval efficiency for traditional matching methods and deep learning models. The present invention also combines the DBSCAN clustering algorithm, which can achieve efficient and accurate personalized recommendation, greatly improving the retrieval efficiency and user experience.

[0006] To solve the above technical problems, the present invention provides the following technical solutions: An ancient poem semantic retrieval method based on Seq2Seq and DBSCAN, comprising the following steps:

[0007] S1. Collect ancient poem sample data of several categories and preprocess the sample data;

[0008] S2. Based on the Seq2Seq model, integrating the LSTM network, translate the ancient poem text through the process of the encoder and the decoder, where the encoder converts the input ancient poem text sequence into a semantic vector of a fixed length, and the decoder generates a vernacular output based on this semantic vector;

[0009] S3. Based on the pre-constructed dictionary, segment the vernacular text by the reverse maximum matching algorithm, combine with Word2Vec word embedding, map the words into semantic feature vectors, and construct an ancient poem semantic feature vector library;

[0010] S4. Preprocess the vernacular query statement and generate a semantic feature vector of the query statement using Word2Vec technology;

[0011] S5. Adopt a cosine similarity algorithm based on N-Gram and dynamic sliding window to extract continuous phrases in the query statement and the ancient poem semantic features, and optimize the similarity calculation by combining with the dynamic sliding window; According to the similarity calculation result, retrieve the ancient poem that is most similar to the user's query;

[0012] S6. Use the DBSCAN algorithm to perform clustering analysis on the semantic feature vectors, and expand and recommend similar ancient poems based on the clustering results.

[0013] Further, the specific step S1 is: Collect ancient poem sample data through web crawling technology, set dynamic crawling rules during the crawling process, and combine regular expressions and data cleaning algorithms to remove noise information in the text, and perform deduplication and standardization processing on the ancient poem sample data to ensure data consistency.

[0014] Further, the foregoing step S2 includes the following sub-steps:

[0015] S2.1. For the input ancient poem text, perform word segmentation and padding to convert the ancient poem text into an ancient poem sequence,

[0016] S2.2. Input the ancient poem sequence into the encoder, and set the encoder parameters, including the input sequence length, vocabulary size, word embedding dimension, and LSTM hidden layer dimension, and encode the ancient poem text sequence into a semantic vector,

[0017] S2.3. The decoder is composed of an LSTM layer. First, generate the index of the word, and then generate a vernacular sequence.

[0018] S2.4. Generate the corresponding relationship between the ancient Chinese poem text and the corresponding vernacular sequence.

[0019] Furthermore, the aforementioned step S3 includes the following sub-steps:

[0020] S3.1. Set the maximum matching length of the reverse maximum matching algorithm, initialize the word segmentation result list, and input the ancient Chinese poem text to be segmented.

[0021] S3.2. Judge whether the length of the text to be segmented is greater than 0. If so, execute step S3.3; otherwise, execute S3.5.

[0022] S3.3. Search forward from the current position to find the longest vocabulary for matching. If a matching word is found, record it as a vocabulary unit, update the pointer position and move it forward. If the current character is not in the dictionary, move the pointer to the previous character, re-match, and update the pointer position at the same time.

[0023] S3.4. Add the matched word to the word segmentation result list, return to execute step S3.2 until the pointer moves to the beginning of the text.

[0024] S3.5. Reverse the word segmentation result list to form the correct order, and output the segmented result in the form of a list.

[0025] S3.6. Use the Word2Vec model to train the corpus and convert each segmented text into a semantic vector.

[0026] S3.7. Save the semantic vector generated by each word segmentation, including the vector representation of each word, and construct a semantic feature vector library. S3.8. Form a matrix with the semantic feature vectors of all words, perform PCA dimensionality reduction on the matrix, and save it.

[0027] Furthermore, the aforementioned step S4 includes the following sub-steps:

[0028] S4.1. Collect the user's query statement and use the reverse maximum matching algorithm to accurately segment the query statement.

[0029] S4.2. Initialize the word segmentation model and perform part-of-speech tagging on the query statement.

[0030] S4.3. Input the segmented query statement, output the part-of-speech of each word, and save the part-of-speech tagging.

[0031] S4.4. Combine the special ancient Chinese poem stop word list to remove the words with no actual meaning from the query text, and retain nouns, verbs, adjectives, and proper nouns.

[0032] S4.5. Convert the cleaned query statements into semantic feature vectors using the Word2Vec model and save them.

[0033] Further, the foregoing step S5 includes the following sub-steps:

[0034] S5.1. Extract consecutive phrases from the query statements and ancient poetry texts, that is, regard two consecutive phrases as the features of the query statements and ancient poetry texts.

[0035] S5.2. Customize a dynamic sliding window and adjust the window range according to the word frequency distribution of the text.

[0036] S5.3. Calculate the cosine similarity as shown in the following formula, and adjust the calculated similarity to the range of [0, 2] to ensure unity.

[0037]

[0038] In the formula, A represents the feature vector of the query statement, B represents the feature vector of the ancient poetry text; i is the index, representing the i-th dimension of the current feature vector, A i represents the i-th feature of the query statement, B i represents the i-th feature of the ancient poetry text.

[0039] S5.4. According to the calculation result of the cosine similarity, retrieve the ancient poetry most similar to the query statement and push it to the user.

[0040] Further, the foregoing step S6 includes the following sub-steps:

[0041] S6.1. Set the parameters of the DBSCAN clustering analysis algorithm, including the maximum neighborhood distance and the minimum number of samples.

[0042] S6.2. Perform DBSCAN clustering analysis on the query semantic feature vectors, identify clusters of semantically similar poems, and obtain the indices of ancient poems belonging to the same cluster.

[0043] S6.3. Based on the clustering analysis results, recommend relevant ancient poems belonging to the same cluster to the user.

[0044] Further, the foregoing ancient poetry semantic retrieval method based on Seq2Seq and DBSCAN further includes collecting user feedback information through a Web page and optimizing semantic matching and personalized recommendation in combination with sentiment analysis, specifically including the following sub-steps:

[0045] S7.1. Dynamically display the content and additional information of ancient poems on the web page, including the author, dynasty, and category.

[0046] S7.2. Add a 1-5 star rating system on the web page to collect the ratings of users on the relevance of poem recommendations. S7.3. Add an emotional preference collection system on the web page to collect users' emotional preferences, including: like, neutral, dislike.

[0047] S7.4. Use the BERT model to perform sentiment analysis on the results collected by the emotional preference collection system to obtain users' emotional feedback on the poems, including: positive, negative, and neutral.

[0048] S7.5. Regularly perform sentiment analysis on the emotional feedback and store the analysis results and ratings in the database together to form a feedback history record.

[0049] S7.6. Regularly collect the feedback history record and dynamically adjust the parameters of the semantic matching algorithm and the DBSCAN clustering algorithm: maximum neighborhood distance, minimum number of samples, to obtain more accurate clustering results and optimize the personalized recommendation strategy.

[0050] Compared with the prior art, the beneficial technical effects of the present invention adopting the above technical solutions are as follows:

[0051] The present invention adopts an ancient poem semantic retrieval method based on Seq2Seq and DBSCAN, which can accurately understand the deep semantics of ancient poems, thereby overcoming the defect that the traditional keyword matching method cannot capture the deep meaning of poems and improving the accuracy of semantic matching. At the same time, the present invention innovatively combines the DBSCAN clustering algorithm to intelligently classify the semantic feature vectors of ancient poems and automatically expand and recommend highly relevant poems according to the user's query content, so as to provide a more intelligent personalized recommendation service. In addition, by using the N-Gram method to extract continuous phrase features and combining with a dynamic sliding window to optimize the cosine similarity calculation, the context relevance is further enhanced, thereby greatly improving the matching accuracy between the query statement and the poem semantic features. In order to optimize the word segmentation effect, the present invention adopts the reverse maximum matching (MBM) algorithm for accurate word segmentation and combines with Word2Vec word vector modeling, so that the semantic feature vector library greatly reduces redundant data while ensuring semantic integrity. In addition, the vector dimension is reduced by PCA to improve the calculation efficiency, so that the system can achieve efficient and real-time retrieval in a large-scale poem library. At the same time, the present invention breaks through the traditional static matching mode, uses the Web-side user feedback information combined with the BERT sentiment analysis technology to dynamically optimize the semantic matching algorithm and the DBSCAN clustering parameters, so that the system can continuously and adaptively adjust the recommendation strategy, thereby improving user satisfaction and recommendation effect. In summary, the present invention effectively integrates the Seq2Seq and DBSCAN algorithms, and through innovative technologies such as deep semantic understanding, intelligent clustering recommendation, and accurate similarity calculation, significantly reduces the misdetection rate, thereby ensuring that users can quickly and accurately obtain poems that meet their intentions and comprehensively improving the retrieval experience. Brief Description of the Drawings

[0052] Figure 1 is a flowchart of the ancient poem semantic retrieval system of Seq2Seq and DBSCAN provided by an embodiment of the present invention;

[0053] Figure 2 is a flowchart for translating ancient Chinese poems into vernacular Chinese;

[0054] Figure 3 is a flowchart for word segmentation by the Reverse Maximum Matching (MBM) algorithm; Detailed Description of the Invention

[0055] To better understand the technical content of the present invention, specific embodiments are hereby given and described in conjunction with the accompanying drawings as follows.

[0056] In the present invention, various aspects of the present invention are described with reference to the accompanying drawings, in which many illustrative embodiments are shown. The embodiments of the present invention are not limited to those described in the drawings. It should be understood that the present invention can be implemented by any one of the various concepts and embodiments introduced above, as well as the concepts and embodiments described in detail below, because the concepts and embodiments disclosed in the present invention are not limited to any embodiment. In addition, some aspects disclosed in the present invention can be used alone, or in any suitable combination with other aspects disclosed in the present invention.

[0057] As Figure 1 shown, the present invention provides an ancient poem semantic retrieval method based on Seq2Seq and DBSCAN, including the following steps:

[0058] S1. Data collection and preprocessing: Automatically collect ancient Chinese poem samples such as Tang poems, Song ci - poems, and Yuan qu - poems from multiple online platforms through web - crawling technology, remove noise information in combination with regular expressions, and perform data deduplication and standardization processing to construct a high - quality ancient Chinese poem corpus;

[0059] S2. Ancient poem semantic representation: Adopt the Seq2Seq model, use LSTM as the encoder and decoder structure, convert the ancient Chinese poem sequence into a fixed - length semantic vector, and generate a vernacular Chinese translation;

[0060] S3. Construction of semantic feature vector library: Based on a pre - constructed dictionary, segment the translated text through the Reverse Maximum Matching (MBM) algorithm, combine with the Word2Vec word embedding technology, map words into semantic feature vectors, and construct an ancient Chinese poem semantic feature vector library;

[0061] S4. Query statement processing: Receive the vernacular Chinese query input by the user in the Web system, perform text preprocessing such as word segmentation, part - of - speech tagging, and stop - word removal, and generate a semantic feature vector of the query statement using the Word2Vec technology;

[0062] S5, Similarity calculation: Adopt the cosine similarity algorithm based on N-Gram and dynamic sliding window, extract the continuous phrases in the semantic features of the query statement and ancient poem words, and optimize the similarity calculation by combining the dynamic sliding window;

[0063] S6, Retrieval and recommendation: According to the similarity calculation results, retrieve the ancient poems most similar to the user's query and display them on the Web page;

[0064] S7, Personalized recommendation: Use the DBSCAN algorithm to perform clustering analysis on the semantic feature vectors, and expand and recommend similar ancient poems based on the clustering results;

[0065] S8, System optimization: Collect user feedback information through the Web page, and optimize the semantic matching and personalized recommendation system by combining sentiment analysis.

[0066] As a preferred embodiment of the present invention, in step S1, a crawling program is written in Python. By setting the headers parameter, it simulates the behavior of the browser, overcomes the anti-crawling mechanism, and automatically crawls ancient poem samples from multiple cultural website platforms (such as Ancient Poems Network, Poetry Collection, etc.). The collected ancient poems cover multiple historical periods such as Tang poems, Song ci-poems, and Yuan qu-poems, ensuring the comprehensiveness and representativeness of the data. The crawling technology sets dynamic crawling rules to ensure the acquisition of the latest high-quality data. Data preprocessing includes: Data cleaning: Use regular expressions to clean the collected data, remove noise information such as punctuation marks, special symbols, page numbers, etc., to ensure that the data is pure and interference-free. Data standardization processing: Normalize the cleaned data, unify the encoding format to UTF-8, and store all text data in CSV format uniformly to ensure that the data formats are consistent and facilitate subsequent processing and storage. Specifically, it includes the following sub-steps:

[0067] S1.1, Write a crawling program in Python. According to the base_url of web pages such as Ancient Poems Network and Poetry Collection, replace the page parameter through the generate_urls function to generate multiple page URLs to be crawled, and achieve paged dynamic crawling; S1.2, Set time.sleep(random.uniform(1, 3)), randomly delay for 1 to 3 seconds to simulate the browsing behavior of users and make the crawling process more natural;

[0068] S1.3, Use BeautifulSoup combined with lxml to parse the HTML page, find specific tags through XPath, and extract Tang poems, Song... S1-4: Set the User-Agent request header through headers to simulate the requests of common browsers and overcome the anti-crawling mechanism;

[0069] S1.5. Write the safe_fetch(url) function to capture network request exceptions and enhance the robustness of the program;

[0070] S1.6. Save the fetched data as a CSV format file. The file contains the original text (text), author (author), title (title), and classification information (category) of the poems, and the file name is data1;

[0071] S1.7. Use the re module to set the regular expressions re.sub(r"[^\w\s]","",content) and re.sub(r"\s+","",content_cleaned) to remove all noise information in data1: punctuation marks, special symbols, and page numbers;

[0072] S1.8. Standardize the UTF-8 encoding of all text content;

[0073] S1.9. Remove duplicate poem content to ensure the consistency and quality of the dataset. After deduplication, the data is stored as data2.

[0074] As a preferred embodiment of the present invention, refer to Figure 2 , step S2 includes the following sub-steps:

[0075] S2.1. Load the pre-trained Seq2Seq model through the tensorflow framework;

[0076] S2.2. Input the processed ancient poem text data2, use the Tokenizer in Keras to tokenize and pad the input ancient poems, and convert the text into a sequence recognizable by the model;

[0077] S2.3. The encoder receives the sequence and sets the parameters: the maximum length of the input sequence is max_sequence_length = 50, the vocabulary size is vocab_size = 10000, the word embedding dimension is embedding_dim = 256, and the LSTM hidden layer dimension is latent_dim = 512. Encode the ancient poem sequence into a semantic vector;

[0078] S2.4. The decoder consists of LSTM layers, receives the states (state_h and state_c) of the encoder, uses model.predict() for prediction, obtains the index of the generated word, and gradually generates the target vernacular sequence;

[0079] S2.5. Write the translated vernacular data into the translate column in data2. The file content contains the original text and the corresponding vernacular translation;

[0080] As a preferred embodiment of the present invention, refer to Figure 3 , step S3 includes the following sub-steps:

[0081] S3.1. Load the pre-trained reverse maximum matching algorithm (MBM) word segmentation tool, and set the parameters: the maximum matching length max_word_length = 5; input the text to be segmented, set the pointer to the end of the text, and initialize the word segmentation result list as data3;

[0082] S3.2. Check whether the length of the text to be segmented is greater than 0. If it is, proceed to S3.3; otherwise, proceed to step S3.5;

[0083] S3.3. Search forward from the current position to find the longest vocabulary for matching, reduce the number of segmentations, try to retain complete words, and avoid incorrect splitting; if a matching word is found, record it as a vocabulary unit and update the pointer position to move forward. If the current character is not in the dictionary, move the pointer to the previous character, re-match, and update the pointer position at the same time;

[0084] S3.4. Add the matched word to the word segmentation result list data3; return to execute step S3.2 until the pointer moves to the beginning of the text;

[0085] S3.5. Reverse the word segmentation result list to form the correct order, output the segmented result in the form of a list, and obtain the segmented dataset data3;

[0086] S3.6. Load the Word2Vec model through the gensim library, and set parameters such as vector_size = 100, window = 5, min_count = 1, etc.; use the Word2Vec model to train the corpus and convert each segmented text into a semantic vector;

[0087] S3.7. Save the semantic vector generated by each word segmentation as a JSON format file data4, which contains the vector representation of each word, and construct a semantic feature vector library;

[0088] S3.8. Form a matrix with the semantic feature vectors of all words; perform PCA dimensionality reduction on the matrix, set the number of principal components as n_components = 50, and save the dimensionality-reduced semantic feature vectors as data5;

[0089] As a preferred embodiment of the present invention, step S4 includes the following sub-steps:

[0090] S4.1. Receive the query statement input by the user through the Web page; use the reverse maximum matching algorithm (MBM) to accurately segment the query statement and save it as data6;

[0091] S4.2. Initialize the pkuseg word segmentation model, set model_name='web', and use the web model for part-of-speech tagging; S4.3. Input the segmented text data5, return the part-of-speech information of each word in the format of (word, part-of-speech), and write the part-of-speech tagging into data6 with the column name POS.

[0092] S4.4. Combine the special ancient poetry stop word list "stopwords.txt" and the part-of-speech tagging results, remove the words with no actual meaning (such as "de", "le", "zhi", etc.) from the query text, and only extract the imagery words, nouns ('n'), verbs ('v'), adjectives ('a'), and proper nouns ('nz'), and store them as data6.

[0093] S4.5. Load the Word2Vec model through the gensim library, set parameters such as vector_size = 100, window = 5, min_count = 1, etc.; use the Word2Vec model to convert the cleaned query statement into a semantic feature vector; store the query semantic vector as data7.

[0094] As a preferred embodiment of the present invention, step S5 includes the following sub-steps:

[0095] S5.1. Use the N-Gram technique to extract the consecutive phrases of the query statement and the ancient poetry text, set ngram_range=(2,2) to extract 2-Gram, that is, consecutive two phrases as the features of the query statement and the ancient poetry text.

[0096] S5.2. Dynamically slide the window and adjust the window range according to the word frequency distribution of the text.

[0097] S5.3. Adjust the calculated similarity to the range of [0,2] to ensure unity; calculate the similarity as follows:

[0098]

[0099] In the formula, A represents the feature vector of the query statement, B represents the feature vector of the ancient poetry text; i is the index, representing the i-th dimension of the current feature vector, A i represents the i-th feature of the query statement, B i represents the i-th feature of the ancient poetry text.

[0100] S5.4. According to the similarity calculation result, retrieve the ancient poetry most similar to the query statement. Push the most relevant poem to the Web page for the user to view.

[0101] As a preferred embodiment of the present invention, step S6 includes the following sub-steps:

[0102] S6.1. Set the parameters of the DBSCAN clustering analysis algorithm: the maximum neighborhood distance eps = 0.5 and the minimum number of samples min_samples = 5;

[0103] S6.2. Perform DBSCAN clustering analysis on the queried semantic feature vectors to identify clusters of semantically similar poems and obtain the indexes of ancient poems belonging to the same cluster;

[0104] S6.3. Based on the clustering analysis results, recommend other relevant ancient poems belonging to the same cluster to the user to improve the relevance and accuracy of the recommendation results;

[0105] As a preferred embodiment of the present invention, the present invention further includes collecting user feedback information through a Web page and optimizing semantic matching and personalized recommendation in combination with sentiment analysis, specifically including the following sub-steps:

[0106] S7.1. Use HTML to design a web page to dynamically display the content of ancient poems and additional information (such as author, dynasty, category, etc.);

[0107] S7.2. Use the HTML input[type="radio"] element to add a 1-5 star rating system to the page, allowing users to rate the relevance of the poem recommendation, and transmit the selected rating by the user to the backend for storage;

[0108] S7.3. Add sentiment preference buttons to the page, providing "like", "neutral", and "dislike" buttons. When users click, they can select their sentiment preferences and submit the feedback results to the backend for processing and storage;

[0109] S7.4. Create a Flask application and set an interface to receive ratings and feedback; use a MySQL database to save user ratings and feedback information; submit user ratings and feedback to the backend system through a REST API; use the transformers library of Hugging Face to load a pre-trained BERT model for sentiment analysis to determine whether the user's sentiment feedback on the poem is positive, negative, or neutral;

[0110] S7.5. Perform sentiment analysis in the Flask API and store the analysis results and ratings in the database together to form a feedback history record; regularly collect user ratings and sentiment feedback, and perform statistics and summary; by analyzing the data of user feedback, the parameters of the semantic matching algorithm and the DBSCAN clustering algorithm: the eps and min_samples values can be dynamically adjusted to obtain more accurate clustering results. According to user feedback, optimize the personalized recommendation strategy.

[0111] Table 1 below explains the variables in each step.

[0112] Table 1

[0113]

[0114]

[0115]

[0116] Although the present invention has been described above with reference to preferred embodiments, it is not intended to limit the present invention. Those of ordinary skill in the art to which the present invention pertains can make various modifications and refinements without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention shall be determined by the claims.

Claims

1. A method for semantic retrieval of ancient Chinese poetry words based on Seq2Seq and DBSCAN, characterized in that, It includes the following steps: S1. Collect sample data of ancient Chinese poems in several categories and preprocess the sample data; S2. Based on the Seq2Seq model, integrating the LSTM network, translate the ancient Chinese poem text through the process of encoder and decoder. The encoder converts the input ancient poem sequence into a semantic vector of fixed length, and the decoder generates a vernacular output based on this semantic vector; S3. Based on the pre-constructed dictionary, segment the vernacular text by the reverse maximum matching algorithm, combine with Word2Vec word embedding, map words to semantic feature vectors, and construct a semantic feature vector library of ancient poem words; S4. Preprocess the vernacular query statement and generate a semantic feature vector of the query statement using Word2Vec technology; S5. Adopt a cosine similarity algorithm based on N-Gram and dynamic sliding window to extract continuous phrases in the query statement and the semantic features of ancient poem words, and optimize the similarity calculation by combining with the dynamic sliding window; According to the similarity calculation result, retrieve the ancient Chinese poem closest to the user's query; S6. Use the DBSCAN algorithm to perform clustering analysis on the semantic feature vectors, and expand and recommend similar ancient Chinese poems based on the clustering results.

2. The method for ancient poem semantic retrieval based on Seq2Seq and DBSCAN according to claim 1, wherein Step S1 is specifically: collect sample data of ancient Chinese poems through web crawling technology, set dynamic crawling rules during the crawling process, and combine regular expressions and data cleaning algorithms to remove noise information in the text, and perform deduplication and standardization processing on the sample data of ancient Chinese poems to ensure data consistency.

3. A method for semantic retrieval of ancient Chinese poetry words based on Seq2Seq and DBSCAN according to claim 1, characterized in that, Step S2 includes the following sub-steps: S2.

1. For the input ancient Chinese poem text, perform word segmentation and padding to convert the ancient Chinese poem text into an ancient Chinese poem sequence. S2.

2. Input the ancient Chinese poem sequence into the encoder, and set the encoder parameters, including the input sequence length, vocabulary size, word embedding dimension, and LSTM hidden layer dimension, and encode the ancient poem sequence into a semantic vector. S2.

3. The decoder consists of an LSTM layer. First, generate the index of the word, and then generate a vernacular sequence. S2.

4. Generate the corresponding relationship between the ancient Chinese poem text and the generated vernacular sequence.

4. A method for semantic retrieval of ancient Chinese poetry words based on Seq2Seq and DBSCAN according to claim 1, characterized in that, Step S3 includes the following sub-steps: S3.

1. Set the maximum matching length of the reverse maximum matching algorithm, initialize the word segmentation result list, and input the ancient Chinese poem text to be segmented. S3.

2. Judge whether the length of the text to be segmented is greater than 0. If so, execute step S3.3; otherwise, execute S3.5; S3.

3. Search forward from the current position to find the longest vocabulary for matching. If a matching word is found, record it as a vocabulary unit, update the pointer position and move forward. If the current character is not in the dictionary, move the pointer to the previous character, re-match, and update the pointer position at the same time. S3.

4. Add the matched word to the word segmentation result list, return to execute step S3.2 until the pointer moves to the beginning of the text; S3.

5. Reverse the word segmentation result list to form the correct order, and output the segmented result in the form of a list; S3.

6. Use the Word2Vec model to train the corpus and convert each segmented text into a semantic vector. S3.

7. Save the semantic vectors generated by each word segmentation, including the vector representation of each word, and construct a semantic feature vector library. S3.

8. Compose the semantic feature vectors of all words into a matrix, perform PCA dimensionality reduction on the matrix, and save it.

5. A method for semantic retrieval of ancient Chinese poems based on Seq2Seq and DBSCAN according to claim 1, characterized in that, Step S4 includes the following sub-steps: S4.

1. Collect the user's query statement, and use the reverse maximum matching algorithm to accurately segment the query statement. S4.

2. Initialize the word segmentation model, and perform part-of-speech tagging on the query statement. S4.

3. Input the segmented query statement, output the part-of-speech of each word, and save the part-of-speech tagging. S4.

4. Combine the special ancient poetry stop word list, remove the words without practical meaning from the query text, and retain nouns, verbs, adjectives, and proper nouns. S4.

5. Use the Word2Vec model to convert the cleaned query statement into a semantic feature vector and save it.

6. The method for ancient poem semantic retrieval based on Seq2Seq and DBSCAN according to claim 1, wherein, Step S5 includes the following sub-steps: S5.

1. Extract the continuous phrases of the query statement and the ancient poetry text, that is, take two consecutive phrases as the features of the query statement and the ancient poetry text. S5.

2. A custom dynamic sliding window, adjust the window range according to the word frequency distribution of the text. S5.

3. Calculate the cosine similarity as shown in the following formula, and adjust the calculated similarity to the range of [0, 2] to ensure unity. In the formula, A represents the feature vector of the query statement, and B represents the feature vector of the ancient poetry text. i is the index, representing the i-th dimension of the current feature vector, A i represents the i-th feature of the query statement, B i represents the i-th feature of the ancient poetry text; S5.

4. According to the calculation result of the cosine similarity, retrieve the ancient poetry most similar to the query statement and push it to the user.

7. A method for semantic retrieval of ancient Chinese poetry words based on Seq2Seq and DBSCAN according to claim 1, characterized in that Step S6 includes the following sub-steps: S6.

1. Set the parameters of the DBSCAN clustering analysis algorithm, including the maximum neighborhood distance and the minimum number of samples. S6.

2. Perform DBSCAN clustering analysis on the semantic feature vectors of the query, identify the clusters of semantically similar poems, and obtain the indexes of the ancient poems belonging to the same cluster. S6.

3. Based on the clustering analysis results, recommend relevant ancient poems belonging to the same cluster to the user.

8. A method for semantic retrieval of ancient Chinese poem words based on Seq2Seq and DBSCAN according to claim 1, characterized in that, It also includes collecting user feedback information through a Web page, and optimizing semantic matching and personalized recommendation in combination with sentiment analysis, specifically including the following sub-steps: S7.

1. Dynamically display the content and additional information of ancient poems on the web page, including the author, dynasty, and category. S7.

2. Add a 1-5 star rating system to the web page to collect the user's rating on the recommendation relevance of the poem. S7.

3. Add a sentiment preference collection system to the web page to collect the user's sentiment preferences, including: like, neutral, dislike. S7.

4. Use the BERT model to perform sentiment analysis on the results collected by the sentiment preference collection system to obtain the user's sentiment feedback on the poem, including: positive, negative, and neutral. S7.

5. Regularly perform sentiment analysis on the sentiment feedback, and store the analysis results and ratings together in the database to form a feedback history record. S7.

6. Regularly collect the feedback history record, and dynamically adjust the parameters of the semantic matching algorithm and the DBSCAN clustering algorithm: the maximum neighborhood distance and the minimum number of samples, to obtain more accurate clustering results and optimize the personalized recommendation strategy.

Images