Artificial Intelligence-Based Traditional Culture Database Construction Method

Abstract

This invention discloses an artificial intelligence-based method for constructing a traditional culture database, belonging to the field of database construction technology. This method first analyzes the data source update time to determine the statistical duration, calculates the data source activity and attention to obtain the sampling frequency matching value, and matches the data source sampling frequency, solving the problem of inflexibility in traditional sampling strategies. Next, it segments user queries into words, constructs a set of query keywords to determine the retrieval domain, and sorts them according to the document matching value to improve retrieval efficiency. Finally, it collects user behavior data, analyzes document popularity values ​​to determine cached documents and cache points, caches them in the corresponding database, and discards low-popularity documents when space is insufficient, updating every three days to adapt to changes in document popularity. Compared with existing technologies, this invention can accurately grasp the characteristics of the data source to adjust the sampling strategy, quickly locate valuable information, and dynamically manage the cache based on user behavior and document popularity, improving the database's timeliness, retrieval efficiency, and user experience.

Description

Technical Field

[0001] This invention belongs to the field of database construction technology, specifically relating to a method for constructing a traditional culture database based on artificial intelligence. Background Technology

[0002] In the era of digital information explosion, academic research and other fields rely heavily on literature data. However, literature is scattered across diverse data sources such as academic databases and professional forums, with significant differences in format, update frequency, and structure. Against this backdrop, existing technologies have revealed many problems.

[0003] At the data acquisition level: Traditional data acquisition methods struggle to flexibly adjust acquisition strategies based on the characteristics of the data source;

[0004] At the retrieval level: Due to the inability to predict complex query intent, users spend a long time filtering results and the response is slow.

[0005] At the cache management level: Traditional caching mechanisms cannot dynamically adjust cached content based on the popularity of documents and user behavior. To address this, we propose a traditional culture database construction method based on artificial intelligence. Summary of the Invention

[0006] The purpose of this invention is to provide a method for constructing a traditional culture database based on artificial intelligence, so as to solve the problems mentioned in the background art.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a method for constructing a traditional culture database based on artificial intelligence, comprising the following steps:

[0008] Step 1: Analyze the update time of the data source to determine the statistical duration. Within the statistical duration, count the number of new documents published, the number of existing documents updated, the number of document downloads, browsing time, and number of collections. Calculate the data source activity and data source attention respectively. Analyze the statistical duration, data source activity, and data source attention comprehensively to obtain the sampling frequency matching value. Match the data source sampling frequency according to the sampling frequency matching value.

[0009] Step 2: Segment the user's query into words, construct a set of query keywords by analyzing the frequency and inverse values ​​of the words, determine the search domain based on the set of query keywords, analyze the document matching values ​​of the documents in the search domain, and sort the documents in the search domain according to the document matching values;

[0010] Step 3: Collect user behavior data, analyze the literature popularity value based on the behavior data, analyze the cached literature based on the literature popularity value, determine the literature cache point by sending test data packets to each cache node, and cache the cached literature in the cache literature library corresponding to the literature cache point.

[0011] Preferably, in step one, the specific process for determining the statistical duration is as follows:

[0012] For each data source, first obtain the time points of the six most recent updates of that data source, calculate the time interval between two adjacent update time points, and record it as the neighbor duration. Then, use the standard deviation calculation formula to calculate the standard deviation value among all neighbor durations, and record it as the neighbor standard deviation. Compare the neighbor standard deviation with the preset neighbor standard deviation threshold. If the neighbor standard deviation is greater than or equal to the corresponding threshold, recalculate the neighbor standard deviation by adding two update time points in each batch until the update interval difference is less than the corresponding threshold. Finally, add up all adjacent update time intervals corresponding to the time when the neighbor standard deviation calculation is stopped, and divide by the number of intervals to obtain the average update duration of the data source.

[0013] Starting from the time when the data source last completed a data update, the update time is shifted forward by an average of 100%. The time reached at this point is the end time. This duration is recorded as the statistical duration TC.

[0014] Preferably, in step one, the specific process for calculating data source activity, data source attention, and sampling frequency matching value is as follows:

[0015] Within the statistical period, the number of new documents published by the data source and the number of updates to existing documents are obtained, and different weight coefficients are assigned to each. After normalizing the number of new documents published by the data source and the number of updates to existing documents, the data sources are multiplied by the corresponding weight coefficients and finally added together to obtain the data source activity SH.

[0016] For each document in the data source, the number of downloads, browsing time, and number of favorites by users are monitored within the statistical period. Then, the downloads, browsing time, and number of favorites of all documents in the data source are summed to obtain the total number of document downloads WX, the total browsing time WL, and the total number of document favorites WS in the data source. After normalizing the total number of document downloads WX, the total browsing time WL, and the total number of document favorites WS in the data source, the data source attention GD is obtained using the formula: GD=WX×a1+WL×a2+WS×a3, where a1, a2, and a3 are preset weight coefficients.

[0017] By using the formula: Statistical Duration (TC), Data Source Activity (SH), and Data Source Attention (GD) The frequency matching value CPZ is obtained, where p1, p2, and p3 are preset weighting coefficients.

[0018] Preferably, in step one, the specific process of matching the data source's acquisition frequency based on the acquisition frequency matching value is as follows:

[0019] Several frequency matching value ranges are preset, and each frequency matching value range corresponds to a data source sampling frequency;

[0020] For each data source, its corresponding sampling frequency matching value is matched with all sampling frequency matching value intervals to output the sampling frequency corresponding to the sampling frequency matching value interval. This sampling frequency is then used to determine the literature data sampling frequency of the data source. A preset sampling update adjustment factor k is used, where k is greater than 1. The sampling update duration GT is obtained using the formula: GT=TC×k. The starting point is the moment when the data source first performs data sampling according to the determined sampling frequency. When the interval elapsed since the starting point reaches the sampling update duration, the re-matching analysis process of the sampling frequency of the data source is immediately triggered.

[0021] Preferably, in step two, the specific process of analyzing the frequency inverse values ​​of words and constructing a set of query keywords is as follows:

[0022] After a user enters a query, the query is broken down into several independent words. For each word, the number of times it appears in the query range is counted. Then, the number of times the word appears in the query range is divided by the total number of words to obtain the word frequency.

[0023] All collected literature data are organized into a document set. For each word, the number of documents in the document set containing that word is counted and organized into a lexicon. The inverse document frequency of a word is obtained by dividing the total number of documents in the document set by the total number of documents in the lexicon.

[0024] For each word, the inverse frequency value is obtained by multiplying the word frequency by the inverse document frequency. The inverse frequency value is then compared with a preset keyword inverse frequency threshold. If the inverse frequency value is greater than or equal to the preset keyword inverse frequency threshold, the word is marked as a keyword in the corresponding query statement. The keywords corresponding to the user's query statement are organized into a keyword set, which is denoted as the query keyword set.

[0025] Preferably, in step two, the specific process of determining the search domain based on the set of query keywords, analyzing the document matching values ​​of documents within the search domain, and sorting the documents within the search domain based on the document matching values ​​is as follows:

[0026] Obtain the document domains involved in the document collection, and for each document domain, obtain the corresponding specific keyword set, denoted as the domain keyword set;

[0027] For each document domain, the query keyword set is matched with the document domain keyword set, and the number of matched keywords is output as the domain matching value. The domain matching value is compared with the preset domain matching threshold. If the domain matching value is greater than the preset domain matching threshold, the document domain is marked as the search domain. All search domains are organized into a search list.

[0028] For each search domain in the search list, all documents in that domain are compiled into a domain literature database. For each document in the domain literature database, the number of words from the query keyword set contained in the document is counted and recorded as the document matching value. All documents in the domain literature database are sorted from high to low according to the size of the document matching value.

[0029] Preferably, in step three, the specific process of analyzing the literature popularity value is as follows:

[0030] Collect and record each user's query behavior data, including: query statement, query time, and corresponding search results; search results include: a list of matched documents, the search field in which the documents are located, the time when the user clicked on the documents, and the reading time; at the same time, record in real time the number of times each document in the document database is collected, the number of comments, the number of likes, and the publication time of the documents;

[0031] For each user, obtain all search domains corresponding to all their query statements in the last three days and generate an unordered list containing all search domains; then count the frequency of each different search domain in the list and record it as the domain deduplication value. For each search domain, if the domain deduplication value is greater than the preset domain deduplication threshold, then mark the search domain as a cached domain.

[0032] For each cache domain, obtain the corresponding domain literature library. For each document in the domain literature library, subtract the document publication time from the current time to obtain the document publication duration FT.

[0033] Attempt to obtain the user's most recent click time on the document. If this time can be obtained, subtract the most recent click time from the current time to get the document's idle time. A preset idle time threshold is set. If the document's idle time is greater than the preset threshold, the threshold is used as the document's idle time. If there has never been a click record, the threshold is directly used as the document's idle time XT. Obtain the document's comment count PL, collection count SC, and user reading time YT.

[0034] After normalizing the number of comments (PL), number of favorites (SC), user reading time (YT), publication time (FT), and idle time (XT) of the document, the following formula is used: The literature popularity value WXD is obtained, where r1, r2, r3, r4, and r5 are preset weight coefficients.

[0035] Preferably, in step three, the specific process of analyzing cached documents based on their popularity values, determining document cache points by sending test data packets to each cache node, and caching the cached documents in the cached document library corresponding to each document cache point is as follows:

[0036] By comparing the popularity value of a document with a preset popularity threshold, if the popularity value is greater than the preset popularity threshold, the document is marked as a cached document.

[0037] Each cache node is configured with a corresponding cached literature library. By sending test data packets to each cache node, the round-trip time from the user device to each cache node is measured. The cache node with the shortest round-trip time is marked as the literature cache point, and all cached literature is cached in the cached literature library corresponding to the literature cache point.

[0038] If the cache space of the cached document library is insufficient, a priority eviction algorithm is used to evict documents with the lowest popularity values ​​in turn.

[0039] Set up a cached literature database update mechanism, which triggers an update operation every three days. When the update operation is triggered, all literature in the cached literature database is deleted, the cached literature is re-analyzed, and then stored in the cached literature database.

[0040] Preferably, a computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the above method.

[0041] Preferably, a computer-readable storage medium stores a computer program thereon, wherein the computer program, when executed by a processor, implements the steps of the above method.

[0042] Compared with the prior art, the beneficial effects of the present invention are:

[0043] (1) The method and system for constructing a traditional culture database based on artificial intelligence determines the average update duration by calculating the neighbor duration and standard deviation of the data source, thereby obtaining the statistical duration and accurately grasping the update pattern and stability of the data source. Within the statistical duration, the activity of the data source is measured by comprehensively considering the release of new documents and the updating of old documents. At the same time, the data source attention is calculated by combining the user's download, browsing and collection data of documents. Based on this, the sampling frequency matching value is analyzed to make the sampling frequency fit the actual update frequency of the data source and avoid over- or under-sampling. Furthermore, the sampling frequency update duration is set, and the sampling frequency matching value is recalculated upon expiration to adjust the sampling frequency in a timely manner, adapt to the dynamic changes of the data source, maintain the timeliness and relevance of the database content, and solve the problem that traditional data collection methods are difficult to flexibly adjust the collection strategy according to the characteristics of the data source.

[0044] (2) The method and system for constructing a traditional culture database based on artificial intelligence, when a user enters a query statement, performs word segmentation on the query statement, calculates the frequency inverse value of the words, constructs a set of query keywords based on the frequency inverse value, and understands the user's query intent; matches the set of query keywords with the set of text domain keywords to determine the retrieval domain, and then sorts the documents in the domain literature database from high to low according to the document matching value of each document in the retrieval domain with the set of query keywords; this enables the newly constructed literature database to display a list of retrieval domains when the user enters a query statement, and after clicking on a certain retrieval domain, the documents in the corresponding domain literature database will be presented to the user according to the matching value from high to low, which makes it convenient to quickly locate the most valuable information for the user, greatly shortens the time spent by the user in filtering results, effectively improves the retrieval efficiency and user experience, and solves the problem of long time spent and slow response caused by the inability to predict complex query intent.

[0045] (3) This method and system for constructing a traditional culture database based on artificial intelligence collects user query behavior data, generates an unordered list of all search fields in the last three days, analyzes the domain deduplication value of each field, and marks the cached fields after comparing it with a preset threshold to capture the fields that users have been frequently paying attention to recently. For the literature in each cached field, the literature popularity value is analyzed by comprehensively considering multiple dimensions such as the publication time of the literature, user clicks and idle time, number of comments, number of collections and user reading time, to determine the literature that may be requested to be accessed. The literature cache point is determined by the cache node positioning method based on network latency detection, and the literature that may be accessed is stored in the corresponding cached literature library. Users can obtain data at a faster speed and significantly shorten the query response time. The cached literature library is updated every three days to adapt to the dynamic changes in the popularity of the literature and ensure that the cached literature library stores the literature that is most likely to be accessed by users at present. This solves the problem that the traditional caching mechanism cannot dynamically adjust the cached content according to the popularity of the literature and user behavior. Attached Figure Description

[0046] Figure 1 This is a flowchart of the present invention. Detailed Implementation

[0047] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0048] Example 1

[0049] Please see Figure 1 This invention provides a method for constructing a traditional culture database based on artificial intelligence, including: an intelligent data acquisition module, an intelligent retrieval module, and an intelligent distributed caching module;

[0050] The intelligent data acquisition module analyzes the update time points of the data source to obtain the statistical duration. Within the statistical duration, it counts the number of new document releases, the number of existing document updates, the number of document downloads, browsing time, and number of bookmarks. It calculates the data source activity and data source attention separately. Finally, it performs a comprehensive analysis based on the statistical duration, data source activity, and data source attention to obtain the sampling frequency matching value. The module then matches the data source's acquisition frequency based on this value. The specific process is as follows:

[0051] We communicate and connect with various data sources through data acquisition technologies such as web crawling, data interface calling, and sensor data acquisition.

[0052] For each data source, first obtain the time points of the six most recent updates of that data source, calculate the time interval between two adjacent update time points, and record it as the neighbor duration. Then, use the standard deviation calculation formula to calculate the standard deviation value among all neighbor durations, and record it as the neighbor standard deviation. A preset neighbor standard deviation threshold is set, and the neighbor standard deviation is compared with the preset neighbor standard deviation threshold. If the neighbor standard deviation is greater than or equal to the corresponding threshold, it indicates that the update time interval fluctuates greatly and the stability is poor. In this case, the neighbor standard deviation is recalculated by adding two update time points in each batch until the update interval difference is less than the corresponding threshold. Finally, the average update duration of the data source is obtained by summing all adjacent update time intervals corresponding to the time when the neighbor standard deviation calculation operation stops and dividing by the number of intervals.

[0053] Starting from the time when the data source last completed a data update, the update time is shifted forward by an average of the time until the end of the update. This time is recorded as the statistical duration TC.

[0054] Within the statistical period, the number of new documents published by the data source and the number of updates to existing documents are obtained, and different weight coefficients are assigned to each. After normalizing the number of new documents published by the data source and the number of updates to existing documents, the data sources are multiplied by the corresponding weight coefficients and finally added together to obtain the data source activity SH.

[0055] For each document in the data source, the number of downloads, browsing time, and number of favorites by users are monitored within the specified time period. Then, the downloads, browsing time, and number of favorites of all documents in the data source are summed to obtain the total number of document downloads WX, the total browsing time WL, and the total number of document favorites WS. After normalizing the total number of document downloads WX, the total browsing time WL, and the total number of document favorites WS, the data source attention GD is obtained using the formula: GD=WX×a1+WL×a2+WS×a3, where a1, a2, and a3 are preset weight coefficients. The larger the data source attention GD, the higher the attention the documents presented in the data source receive from users.

[0056] By using the formula: Statistical Duration (TC), Data Source Activity (SH), and Data Source Attention (GD) The sampling frequency matching value CPZ is obtained, where p1, p2, and p3 are preset weight coefficients. The larger the sampling frequency matching value corresponding to the data source, the more frequently the data source is updated, and the higher the user's attention and activity level to its content. More frequent data collection is required to ensure that the latest and most popular information is obtained and to avoid data omission or outdatedness.

[0057] Several frequency matching value intervals are preset, and each frequency matching value interval corresponds to a data source acquisition frequency. The larger the lower and upper bounds of the frequency matching value interval, the smaller the corresponding acquisition frequency value, that is, the faster the acquisition frequency.

[0058] For each data source, its corresponding sampling frequency matching value is matched with all sampling frequency matching value intervals to output the sampling frequency corresponding to the sampling frequency matching value interval. This sampling frequency is then used to determine the literature data sampling frequency of the data source. A preset sampling update adjustment factor k is used, where k is greater than 1. The sampling update duration GT is obtained using the formula: GT=TC×k. The starting point is the moment when the data source first performs data sampling according to the determined sampling frequency. When the interval elapsed since the starting point reaches the sampling update duration, the re-matching analysis process of the sampling frequency of the data source is immediately triggered.

[0059] It should be noted that by calculating the neighbor interval duration and standard deviation, the average update duration is determined, and thus the statistical duration is obtained. This allows for a precise grasp of the data source's update patterns and stability. Based on this, the sampling frequency matching value is calculated, ensuring that the sampling frequency matches the actual update frequency of the data source, avoiding over- or under-collection. Within the statistical duration, the activity of the data source is measured by comprehensively considering the release of new documents and the updating of old documents. At the same time, the data source's attention is calculated by combining user data on document downloads, browsing, and collection. This ensures that the sampling frequency integrates the data source's own updates with user attention, guaranteeing that the collected documents are both new and relevant, thereby enhancing the data value. By setting the sampling frequency update duration and recalculating the sampling frequency matching value upon expiration, the sampling frequency can be adjusted in a timely manner to adapt to dynamic changes in the data source and maintain the timeliness and relevance of the database content.

[0060] The intelligent search module segments the user's query into words, constructs a set of query keywords by analyzing the frequency and inverse values ​​of the words, determines the search domain based on the query keyword set, analyzes the document matching values ​​of documents within the search domain, and sorts the documents within the search domain according to the document matching values. The specific process is as follows:

[0061] When a user enters a query, natural language processing technology is used to break the query into several independent words. For each word, the number of times the word appears in the query range is counted. Then, the number of times the word appears in the query range is divided by the total number of words to obtain the word frequency.

[0062] The literature data acquired by the intelligent data acquisition module is organized into a document set. For each word, the number of documents in the document set containing that word is counted and organized into a word file set. The inverse document frequency of the word is obtained by dividing the total number of documents in the document set by the total number of documents in the word file set.

[0063] For each word, the inverse frequency value is obtained by multiplying the word frequency by the inverse document frequency. The larger the inverse frequency value, the more critical the word is in the query and the stronger its indicative power for determining the topic of the query. A preset keyword inverse frequency threshold is set. The inverse frequency value of a word is compared with the preset keyword inverse frequency threshold. If the inverse frequency value of a word is greater than or equal to the preset keyword inverse frequency threshold, the word is marked as a keyword in the corresponding query. The keywords corresponding to the user's query are organized into a keyword set, which is denoted as the query keyword set.

[0064] To obtain the document collection, we need to identify the various literature domains involved. For each literature domain, we need to obtain a specific set of keywords corresponding to it, denoted as the domain keyword set. The specific keyword set corresponding to the literature domain can be obtained in the following way: First, we perform a full-text scan of the literature in the domain and use text mining technology to extract words that appear frequently and are representative of the domain. At the same time, we refer to professional dictionaries, terminology in authoritative academic papers, and suggestions from domain experts to screen and supplement the extracted words, and finally construct the specific keyword set corresponding to the literature domain.

[0065] For each document domain, the query keyword set is matched with the document domain keyword set, and the number of matched keywords is output as the domain matching value. A preset domain matching threshold is set, and the domain matching value is compared with the preset domain matching threshold. If the domain matching value is greater than the preset domain matching threshold, the document domain is marked as the search domain, and all search domains are organized into a search list.

[0066] For each search domain in the search list, all documents in that domain are compiled into a domain literature database. For each document in the domain literature database, text matching technology is used to count the number of words from the query keyword set contained in the document, which is recorded as the document matching value. All documents in the domain literature database are sorted from highest to lowest according to the size of the document matching value. When a user enters a test statement, a search list is first displayed. When the user clicks on a search domain in the search list, the documents in the corresponding domain literature database are sorted from highest to lowest according to the size of the document matching value.

[0067] It should be noted that when a user enters a query, the query is first segmented into words, and then the frequency inverse value of each word is calculated. Based on the frequency inverse value, the set of query keywords is analyzed. Through word segmentation, frequency inverse value calculation, and keyword set construction, the user's query intent is better understood. The set of query keywords is then matched with the set of domain keywords to determine the search domain. For each search domain, the documents in the domain's literature database are sorted from highest to lowest based on the document matching value between each document in the search domain and the set of query keywords. This allows the newly constructed literature database to display a list of search domains when a user enters a query. When a user clicks on a search domain, the documents in the corresponding domain's literature database are presented to the user in descending order of document matching value. This facilitates quick and easy location of the most valuable information, greatly reducing the time spent filtering results and effectively improving search efficiency and user experience.

[0068] The intelligent distributed caching module collects user behavior data and analyzes the popularity of documents based on this data. It then analyzes the cached documents based on these popularity values, determines the document caching points by sending test data packets to each caching node, and caches the documents in the corresponding cached document library. The specific process is as follows:

[0069] Collect and record each user's query behavior data, including: query statement, query time, and corresponding search results; search results include: a list of matched documents, the search field in which the documents are located, the time when the user clicked on the documents, and the reading time, etc.; at the same time, record in real time the number of times each document in the document database is collected, the number of comments, the number of likes, and the publication time of the documents, etc.

[0070] For each user, obtain all search domains corresponding to all their query statements in the last three days and generate an unordered list containing all search domains; then, through programming counting logic, count the frequency of each different search domain in the list and record it as the domain deduplication value. Set a domain deduplication threshold. For each search domain, if the domain deduplication value is greater than the preset domain deduplication threshold, mark the search domain as a cached domain.

[0071] For each cache domain, obtain the corresponding domain literature library. For each document in the domain literature library, subtract the document publication time from the current time to obtain the document publication duration FT.

[0072] Attempt to obtain the user's most recent click time on the document. If this time can be obtained, subtract the most recent click time from the current time to get the document's idle time. A preset idle time threshold is set. If the document's idle time is greater than the preset threshold, the threshold is used as the document's idle time. If there has never been a click record, the threshold is directly used as the document's idle time XT. Obtain the document's comment count PL, collection count SC, and user reading time YT.

[0073] After normalizing the number of comments (PL), number of favorites (SC), user reading time (YT), publication time (FT), and idle time (XT) of the document, the following formula is used: The document popularity value WXD is obtained, where r1, r2, r3, r4, and r5 are preset weight coefficients. The larger the document popularity value, the more attention and popularity the document receives from users. This means that the document is more likely to be requested during the user's search process.

[0074] A preset document popularity threshold is set. By comparing the document popularity value with the preset document popularity threshold, if the document popularity value is greater than the preset document popularity threshold, the document is marked as a cached document.

[0075] Each cache node is configured with a corresponding cached literature library. By sending test data packets to each cache node, the round-trip time from the user's device to each cache node is measured. The cache node with the shortest round-trip time is marked as the literature cache point, and all cached literature is cached in the corresponding cached literature library of the literature cache point to ensure that users can obtain data as quickly as possible when searching for relevant literature. If the cache space of the cached literature library is insufficient, a priority eviction algorithm is used to evict the literature with the lowest popularity value in turn.

[0076] Set up a cached literature database update mechanism, which triggers an update operation every three days. When the update operation is triggered, all literature in the cached literature database is deleted, the cached literature is re-analyzed, and then stored in the cached literature database.

[0077] It should be noted that by collecting user query behavior data, an unordered list of all search domains within the last three days is generated. The domain deduplication value of each domain is analyzed and compared with a preset threshold. If the value exceeds the threshold, it is marked as a cached domain. This method facilitates the identification of domains frequently accessed by users recently. For documents within each cached domain, a comprehensive analysis of multiple factors, including publication duration, user clicks and idle time, number of comments, number of favorites, and user reading time, is conducted to determine the documents that are likely to be requested. A cache node location method based on network latency detection is used to analyze document cache points, and documents that are likely to be requested are stored in the corresponding cache document library. This allows users to retrieve data faster, significantly shortening query response time and greatly improving user experience. A cache document library update operation is triggered every three days. This mechanism adapts to the dynamic changes in document popularity, ensuring that the cache document library stores the documents most likely to be accessed by customers at any given time.

[0078] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for constructing a traditional culture database based on artificial intelligence, characterized by: Includes the following steps: Step 1: Analyze the update time of the data source to determine the statistical duration. Within the statistical duration, count the number of new documents published, the number of existing documents updated, the number of document downloads, browsing time, and number of collections. Calculate the data source activity and data source attention respectively. Analyze the statistical duration, data source activity, and data source attention comprehensively to obtain the sampling frequency matching value. Match the data source sampling frequency according to the sampling frequency matching value. The specific process for determining the statistical duration is as follows: For each data source, first obtain the time points of the six most recent updates of that data source, calculate the time interval between two adjacent update time points, and record it as the neighbor duration. Then, use the standard deviation calculation formula to calculate the standard deviation value between all neighbor durations, and record it as the neighbor standard deviation. Compare the neighbor standard deviation with the preset neighbor standard deviation threshold. If the neighbor standard deviation is greater than or equal to the corresponding threshold, then recalculate the neighbor standard deviation by adding two update time points in each batch until the update interval difference is less than the corresponding threshold. Finally, sum all the adjacent update time intervals corresponding to the time when the calculation of the neighbor standard deviation is stopped, and divide by the number of intervals to obtain the average update duration of the data source. Starting from the time when the data source last completed a data update, the update time is shifted forward by an average of the time until the end of the update. This time is recorded as the statistical duration TC. The analysis process for data source activity and data source attention is as follows: Within the statistical period, the number of new documents published by the data source and the number of updates to existing documents are obtained, and the data source activity is calculated by combining the preset weights. For each document in the data source, the download count, browsing time, and number of times the user favorited the document within the summation period are calculated, and the data source attention is calculated by combining the preset weights. The process of matching the data source's acquisition frequency based on the acquisition frequency matching value is as follows: Several frequency matching value ranges are preset, and each frequency matching value range corresponds to a data source sampling frequency; For each data source, its corresponding sampling frequency matching value is matched with all sampling frequency matching value intervals to output the sampling frequency corresponding to the sampling frequency matching value interval. This sampling frequency is then used to determine the literature data sampling frequency of the data source. A preset sampling update adjustment factor k is used, where k is greater than 1. The sampling frequency update duration GT is obtained using the formula: GT=TC×k. Step 2: Segment the user's query into words, construct a set of query keywords by analyzing the frequency and inverse values ​​of the words, determine the search domain based on the set of query keywords, analyze the document matching values ​​of the documents in the search domain, and sort the documents in the search domain according to the document matching values; The specific process of analyzing the frequency inverse values ​​of words and constructing a set of query keywords is as follows: After a user enters a query, the query is broken down into several independent words. For each word, the number of times it appears in the query range is counted. Then, the number of times the word appears in the query range is divided by the total number of words to obtain the word frequency. All collected literature data are organized into a document set. For each word, the number of documents in the document set containing that word is counted and organized into a lexicon. The inverse document frequency of a word is obtained by dividing the total number of documents in the document set by the total number of documents in the lexicon. For each word, the inverse frequency value is obtained by multiplying the word frequency and the inverse document frequency of the word. The inverse frequency value of the word is compared with the preset keyword inverse frequency threshold. If the inverse frequency value of the word is greater than or equal to the preset keyword inverse frequency threshold, the word is marked as a keyword of the corresponding query statement. The keywords corresponding to the user's query statement are organized into a keyword set, which is called the query keyword set. The process of determining the search domain based on the set of query keywords, analyzing the document matching values ​​of documents within the search domain, and sorting the documents within the search domain according to the document matching values ​​is as follows: Obtain the document domains involved in the document collection, and for each document domain, obtain the corresponding specific keyword set, denoted as the domain keyword set; For each document domain, the query keyword set is matched with the document domain keyword set, and the number of matched keywords is output as the domain matching value. The domain matching value is compared with the preset domain matching threshold. If the domain matching value is greater than the preset domain matching threshold, the document domain is marked as the search domain. All search domains are organized into a search list. For each search domain in the search list, all the literature in that domain is organized into a domain literature database. For each document in the domain literature database, the number of words in the query keyword set contained in the document is counted and recorded as the document matching value. All documents in the domain literature database are sorted according to the size of the document matching value. Step 3: Collect user behavior data, analyze the literature popularity value based on the behavior data, analyze the cached literature based on the literature popularity value, determine the literature cache point by sending test data packets to each cache node, and cache the cached literature in the cache literature library corresponding to the literature cache point. The analysis process for the literature popularity value is as follows: For each user, obtain all search domains corresponding to all their query statements in the last three days and generate an unordered list containing all search domains; then count the frequency of each different search domain in the list and record it as the domain deduplication value. For each search domain, if the domain deduplication value is greater than the preset domain deduplication threshold, then mark the search domain as a cached domain. For each cache domain, retrieve the corresponding domain literature library. For each document in the domain literature library, subtract the document's publication time from the current time to obtain the document's publication duration. Try to obtain the time of the user's most recent click on the document. If the time can be obtained, subtract the most recent click time from the current time to get the document's idle time. A preset document idle time threshold is set. If the document idle time is greater than the preset document idle time threshold, the document idle time threshold is used as the document idle time. If there has never been a click record, the document idle time threshold will be used directly as the document idle time. The number of comments, number of favorites, and user reading time of a document are collected and combined with the document's publication time and idle time to conduct a comprehensive analysis to obtain the document's popularity value. The process of determining a document cache point and caching the cached document in the cache document repository corresponding to that document cache point is as follows: By comparing the popularity value of a document with a preset popularity threshold, if the popularity value is greater than the preset popularity threshold, the document is marked as a cached document. Each cache node is configured with a corresponding cached literature library. By sending test data packets to each cache node, the round-trip time from the user device to each cache node is measured. The cache node with the shortest round-trip time is marked as the literature cache point, and all cached literature is cached in the cached literature library corresponding to the literature cache point. If the cache space of the cached document library is insufficient, a priority eviction algorithm is used to evict documents with the lowest popularity values ​​in turn. Set up a cached literature database update mechanism, which triggers an update operation every three days. When the update operation is triggered, all literature in the cached literature database is deleted, the cached literature is re-analyzed, and then stored in the cached literature database.

2. The method for constructing a traditional culture database based on artificial intelligence according to claim 1, characterized in that: In step one, the specific process for calculating data source activity, data source attention, and frequency matching value is as follows: Within the statistical period, the number of new documents published by the data source and the number of updates to existing documents are obtained, and different weight coefficients are assigned to each. After normalizing the number of new documents published by the data source and the number of updates to existing documents, the data sources are multiplied by the corresponding weight coefficients and finally added together to obtain the data source activity SH. For each document in the data source, the monitoring statistics include the number of downloads, browsing time, and number of times users save the document within the specified period. Then, the download count, browsing time, and number of collections of all documents in the data source are summed to obtain the total number of document downloads WX, the total browsing time WL, and the total number of document collections WS. After normalizing the total number of document downloads WX, the total browsing time WL, and the total number of document collections WS in the data source, the data source attention GD is obtained using the formula: GD=WX×a1+WL×a2+WS×a3, where a1, a2, and a3 are preset weight coefficients. By using the formula: Statistical Duration (TC), Data Source Activity (SH), and Data Source Attention (GD) The frequency matching value CPZ is obtained, where p1, p2, and p3 are preset weighting coefficients.

3. The method for constructing a traditional culture database based on artificial intelligence according to claim 2, characterized in that: The starting point is the moment when the data source first collects data according to the determined collection frequency. When the interval elapsed since the starting point reaches the frequency update time, the re-matching and analysis process of the data source collection frequency is immediately triggered.

4. The method for constructing a traditional culture database based on artificial intelligence according to claim 3, characterized in that: In step three, the calculation process for the literature popularity value is as follows: Collect and record each user's query behavior data, including: query statement, query time, and corresponding search results; search results include: a list of matched documents, the search field in which the documents are located, the time when the user clicked on the documents, and the reading time; at the same time, record in real time the number of times each document in the document database is collected, the number of comments, the number of likes, and the publication time of the documents; Calculate the number of comments (PL), number of favorites (SC), and user reading time (YT) for the document. After normalizing the number of comments (PL), number of favorites (SC), user reading time (YT), publication time (FT), and idle time (XT) of the document, the following formula is used: The literature popularity value WXD is obtained, where r1, r2, r3, r4, and r5 are preset weight coefficients.

5. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 4.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 4.

Images