Data updating method based on search engine

Abstract

The invention provides a data updating method based on a search engine, which comprises the following steps: acquiring search behavior data of a user, and extracting a personalized behavior feature vector of the user; predicting a future search demand of the user according to the personalized behavior feature vector of the user, and identifying a webpage which needs to be predicted and updated; predicting a behavior feature vector of the user at a future moment; dynamically evaluating the updating priority of each webpage, and generating a prediction strategy; designing a multi-level intelligent cache optimization strategy to optimize a prediction strategy of the search engine; an intelligent selective data updating mechanism is designed according to the optimized prediction strategy and the updating priority of each webpage, and special optimization is conducted on a data updating scene in a search engine; and a self-adaptive feedback loop is established, and the overall performance of the search engine is dynamically optimized. The updating efficiency of the search engine can be effectively improved, the waste of system resources is reduced, and the search experience of a user is enhanced.

Description

Technical Field

[0001] The present invention belongs to the technical field of search engines, and in particular relates to a data updating method based on a search engine. Background Art

[0002] With the explosive growth of Internet information, search engines have become an important tool for users to obtain information. The core function of a search engine is to quickly find the most relevant results in massive data and provide users with accurate and timely information. However, in order to achieve this, search engines need to constantly update their index databases to ensure the real-time and accuracy of search results. Traditional search engine data update methods mainly rely on regularly crawling web page content and updating the index library, but this approach has significant limitations.

[0003] First, the traditional regular crawling and updating strategy cannot effectively cope with the dynamic changes of network content. In the Internet environment, the frequency and scope of web page content updates are highly uncertain. If the search engine uses a fixed time interval to update data, it may lead to insufficient updates for some frequently changing web pages and excessive updates for some less changing web pages. This not only wastes a lot of computing resources and network bandwidth, but also may lead to a decrease in the real-time and relevance of search results. Especially when hot events occur, users hope to get the latest information quickly, and traditional methods often cannot meet this real-time demand.

[0004] Secondly, traditional data update methods usually ignore the dynamic changes of user behavior and search intent. User search behavior and interests are personalized and dynamic, and fixed update strategies cannot be optimized for each user's search needs. This "one-size-fits-all" approach makes it difficult for search engines to accurately capture users' current search intent and cannot intelligently pre-fetch and update data according to users' actual needs, affecting user experience.

[0005] In addition, search engines face huge challenges in terms of data volume and storage space during data updates. As the number of web pages continues to grow, the amount of index data that search engines need to manage is also increasing. How to ensure the update and maintenance of index library data without increasing the burden on the system has become a problem that needs to be solved urgently. Traditional update strategies cannot effectively utilize system resources. When processing large-scale data, problems such as resource waste and excessive system load often occur. Especially for search engine cache management, how to effectively utilize cache space to ensure fast access to important data while reducing unnecessary data redundancy and storage overhead is a difficult problem that needs to be solved urgently.

[0006] In response to the above problems, existing solutions such as update methods based on content change detection and optimization of crawling strategies using web page access logs have improved the efficiency and accuracy of search engine data updates to a certain extent, but there are still obvious shortcomings. These methods often lack in-depth analysis of user behavior, cannot achieve personalized real-time data updates, and cannot dynamically balance resource consumption and data update quality in update strategies. In addition, when facing large-scale data, these methods lack effective cache management and data selection mechanisms, resulting in waste of system resources and low update efficiency. Therefore, the current search engine data update technology still has many shortcomings, and an update method that can be intelligent, dynamic and efficient is urgently needed. Summary of the invention

[0007] The purpose of the present invention is to design a data update method based on a search engine, and by introducing a multi-dimensional method of intelligent prediction and dynamic cache management, a new search engine data update solution is provided to address the problems of poor dynamic response capability, insufficient personalization, and inefficient resource management in the prior art. The solution can effectively improve the update efficiency of the search engine, reduce system resource waste, and enhance the user's search experience.

[0008] In order to achieve the above object, the present invention provides a data updating method based on a search engine, characterized in that the method comprises the following steps:

[0009] S1. Obtain the user's search behavior data and extract the user's personalized behavior feature vector, including:

[0010] Integrate the user's search behavior data into a user behavior feature vector to represent the user's search preference within a specific time period, and use a sliding window mechanism to dynamically update the user's behavior data;

[0011] S2. Predict the user's future search needs based on the user's personalized behavior feature vector, and identify the web pages that need to be predicted and updated based on the changes in the web page content, including:

[0012] According to the changes in the webpage content, a long short-term memory network is constructed to predict the user's future behavior feature vector, and at the same time, an adaptive prediction strategy is designed according to the changes in the webpage content and the predicted user's future behavior feature vector. The adaptive prediction strategy is set for webpages with frequent changes in webpage content and strong correlation with user needs;

[0013] S3. Dynamically evaluate the update priority of each web page by combining the personalized behavior feature vector and the changes in web page content, and generate a prediction strategy, including:

[0014] A web page importance scoring model is designed based on the user's future behavior feature vector and the change of web page content. The web page importance scoring model includes a user relevance score and a content change score to obtain a comprehensive web page score. The web page priority is sorted according to the comprehensive web page score to generate a specific prediction strategy;

[0015] S4. Design a multi-level intelligent cache optimization strategy to optimize the search engine's prediction strategy, including:

[0016] A three-level cache system is designed, namely: hot cache, warm cache and cold cache. According to the comprehensive web page score, the highest priority web page is placed in the hot cache first, while the lowest priority web page is placed in the cold cache, which is expressed as follows:

[0017]

[0018] Among them, L c Indicates the cache level of web page c, which determines in which cache the web page should be placed; U c represents the comprehensive webpage score of webpage c, which measures the importance of the webpage; θ1, θ2 represent cache level thresholds, which are dynamically calculated by the webpage priority. θ1 is used to determine whether a webpage should enter the hot cache, and θ2 is used to determine whether a webpage should enter the warm cache or the cold cache;

[0019] The design dynamically adjusts the cache strategy based on the regularization term of web page access frequency and timeliness, so that the cache system can adapt to changes in user behavior and prevent low-frequency web pages from occupying cache resources for a long time, as shown below:

[0020]

[0021] in, represents the comprehensive loss function of web page c being placed in the cache level, L c is the cache level of web page c; U c represents the comprehensive webpage score of webpage c; f c Indicates the access frequency of web page c within a certain time window, which is counted by log data; Δt c represents the time interval between the last update time of web page c and the current time; A and B represent the weights that control the impact of web page access frequency and update time on cache level allocation;

[0022] S5. Design an intelligent selective data update mechanism based on the optimized prediction strategy and the update priority of each web page, and perform special optimization for the data update scenario in the search engine, including:

[0023] An update scoring mechanism is designed to evaluate whether each web page should be updated, wherein the update scoring mechanism includes the comprehensive web page score, content change score and web page cache level information of web page c, which is expressed as follows:

[0024]

[0025] Among them, P c represents the comprehensive update score of web page c, which is used to decide whether to update data; ι, κ, ν are the priority scores U c , content change rate C c and the weight of the cache level contribution, Represents the regularization term for balancing system resource consumption; represents the cache level contribution function, which is used to represent the impact of the cache level on the update decision and is defined as:

[0026]

[0027] At the same time, a dynamic update decision threshold θ is introduced u , flexibly adjust the update strategy, adaptively control the frequency of web page updates, and finally determine whether to update the web page based on the comprehensive update score of web page c and the update decision threshold, and finally obtain the web page update decision U update :

[0028] If P c ≥θ u , then web page c is selected for update, and data crawling, content parsing and index update will be performed on the web page;

[0029] If P c <θ u , then the webpage will be delayed or not updated, pending subsequent evaluation;

[0030] S6. Establish an adaptive feedback loop to dynamically optimize the overall performance of the search engine.

[0031] Furthermore, the user's search behavior data includes the user's query word vector at time t, the web page number clicked by the user, the time the user stayed on the web page, and the user's historical query frequency;

[0032] The user behavior feature vector is expressed as follows:

[0033] X t =α1·Q t +α2·log(1+T t )+α3·F t

[0034] Among them, X trepresents the comprehensive behavior feature vector of the user at time t, representing the user's search preference at that moment; Q t represents the query word vector of the user at time t, describing the semantic information of the user's search content; T t Indicates the time the user stays on the clicked web page, which is logarithmically scaled to avoid weight shift due to huge differences in stay time; F t It represents the frequency of the user's past search behavior, reflecting the user's long-term interest in a certain type of information; α1, α2, α3 represent weight coefficients, which are used to balance the impact of different features on the final feature vector.

[0035] Furthermore, the adaptive prediction strategy controls the prediction decision of the web page by introducing a dynamic threshold θ, which is automatically adjusted according to the correlation between the historical web page update frequency and the current prediction, and can dynamically determine which web pages need to be updated first according to the current prediction status, as shown below:

[0036]

[0037] Among them, S c represents the decision variable, indicating whether web page c should be predicted to be updated. If S c ≥θ, then the web page is selected for prediction; W represents the predicted user's future behavior feature vector, reflecting the user's future interests; c It represents the weight of the change of web page content, reflecting the response of web page content to the change of user needs; θ represents the adaptive threshold, which is automatically adjusted according to the historical web page update frequency and the current prediction results, and is used to control which web pages need to be updated first.

[0038] Furthermore, the user relevance score calculates the relevance of each web page to the user's future needs based on the user's behavior feature vector at a future moment; the content change score is the weight of the web page content change.

[0039] Furthermore, the user relevance score and the content change score are combined to generate a comprehensive webpage score, which is expressed as follows:

[0040] U c =λ·R c +(1-λ)·C c

[0041] Among them, U c represents the comprehensive priority score of web page c, which is used to determine the order of web page updates; R c Represents the user relevance score, reflecting the matching degree between the webpage content and the user's future needs; C crepresents the content change score, reflecting the dynamic changes of web page content; λ represents the weight adjustment coefficient, controlling the impact of user needs and web page changes on the final priority score;

[0042] Sort the web page priorities according to the comprehensive web page score of each web page, and select the web pages with scores higher than the set threshold θ p The web page is used to predict and update data, as shown below:

[0043]

[0044] Among them, S c represents the prediction decision variable of web page c, S c =1 means that the webpage will be predicted to be updated, S c =0 means that the webpage is not updated; θ p Indicates the threshold of the prediction strategy, which is used to control the number of predicted web pages to ensure that system resources are not over-consumed.

[0045] Furthermore, in S4, the loss function The regularization term in -α·log(f c ) is used to reduce the possibility of high-frequency access web pages occupying low-level caches, -β·Δt c Used to prevent outdated content from occupying the cache for a long time.

[0046] Furthermore, the S4 further includes:

[0047] According to the changes in the priority and access frequency of web pages, cache data is regularly migrated. When the cache space is close to full, a dynamic elimination mechanism is introduced to eliminate web pages based on their priority scores and the current utilization of the cache layer. Web page data with low priority and not accessed for a long time will be removed from the cache. The time decay model is used to control the cache retention time of web pages, which is expressed as follows:

[0048]

[0049] Among them, T c represents the storage time of web page c in the cache, as the priority U c Increase, storage time is extended; T max Indicates the maximum cache storage time, which indicates the longest retention time of web page data; ∈ indicates preventing division by zero error when the priority is zero.

[0050] Furthermore, the dynamically updated decision threshold θ u According to the system load and historical update status, the frequency of web page updates is adaptively controlled, as shown below:

[0051] θ u=θ0·(1+η·LoadFactor)

[0052] Among them, θ u represents the dynamic threshold used for update decision, which controls whether the web page should be updated; θ0 represents the basic update threshold, which reflects the initial value under normal load conditions; η represents the adjustment coefficient, which is used to control the influence of the load factor on the threshold; LoadFactor represents the current load factor, which reflects the system resource utilization rate, and the calculation formula is as follows:

[0053]

[0054] When the system load increases, the threshold θ u will be increased accordingly, reducing unnecessary update operations; when the load is reduced, θ u will be reduced, allowing more pages to be updated;

[0055] According to the comprehensive update score P c and update the decision threshold θ u , decide whether to update the web page, the specific steps are as follows:

[0056] If P c ≥θ u , then web page c is selected for update, and data crawling, content parsing and index update will be performed on the web page;

[0057] If P c <θ u , the webpage will be delayed or not updated, pending subsequent evaluation.

[0058] Furthermore, the S6 is specifically analyzed as follows:

[0059] According to the updated decision variables of web page c, the threshold θ is dynamically adjusted by designing a feedback optimization model through a feedback loop u ;

[0060] Design feedback optimization function to dynamically adjust the priority score U c , content change rate C c and the weights ι,κ,ν contributed by the cache levels, so that the system focuses on resource usage when the load is high, and on the content change rate and cache level when the load is low, as shown below:

[0061]

[0062] in, represents the feedback optimization function, which is used to adjust the weight parameters ι,κ,ν, affecting the calculation of the web page update score; Indicates the query response time of the current system; Represents the user's click rate data, reflecting the relevance and satisfaction of the user's query results; T max represents the maximum allowable response time of the system as a performance target; C target It represents the predetermined target click-through rate, reflecting the user click-through rate that the system expects to achieve; ω1, ω2 represent coefficients, which are used to balance the importance of response time and click-through rate in the weight optimization process;

[0063] Construct a feedback correction mechanism based on historical data to correct the regularization term in the comprehensive update score, wherein the feedback correction mechanism based on historical data corrects the regularization term according to the historical update results and user behavior feedback. Corrections are made to ensure that low-priority, high-frequency access web pages are not over-updated, but optimized according to actual needs, as shown below:

[0064]

[0065] Among them, λ new represents the updated regularization parameter, which is used to control the update frequency of frequently visited web pages; old represents the original regularization parameter; μ represents the adjustment coefficient, which is used to control the influence of the regularization parameter when the click rate deviates from the target; Indicates the actual user click rate; C target Indicates the target click-through rate.

[0066] Furthermore, the feedback optimization model is constructed as follows:

[0067] LoadFactor max It is defined as the ratio of the current system resource usage to the maximum resource, expressed as follows:

[0068]

[0069] in, Indicates the current CPU usage of the system. Indicates the current memory usage; and Indicates the maximum CPU and memory capacity of the system;

[0070] According to the load factor LoadFactor max , dynamically adjust the threshold θ u , ensuring that updates are reduced when the load is high and increased when the load is low:

[0071] θ u =θ0·(1+η·LoadFactor max )

[0072] Among them, θu represents the dynamic threshold used for update decision, which controls whether the web page should be updated; θ0 represents the basic update threshold, which reflects the initial value under normal load conditions; η represents the adjustment coefficient, which is used to control the influence of the load factor on the threshold.

[0073] The beneficial technical effects of the present invention are at least as follows:

[0074] The present invention proposes a data update method based on a search engine, aiming to solve the shortcomings of traditional methods in dealing with dynamically changing network content, personalized user behavior and large-scale data management. The method effectively improves the data update efficiency and accuracy of the search engine through the collaborative work of multiple modules.

[0075] First, the present invention introduces a user behavior prediction and intelligent pre-fetching module. By analyzing the user's search history and behavior characteristics, the deep learning algorithm is used to predict the user's future search needs and pre-fetch relevant web page data in advance. This method overcomes the defect that the traditional update strategy cannot dynamically respond to changes in user search intent, and significantly improves the real-time nature of data updates and the personalized experience.

[0076] Secondly, the present invention designs a multi-level cache management module based on fuzzy logic. This module combines multi-dimensional factors such as web page access frequency, content change rate, and user click rate to calculate the priority of data updates and optimize data storage through a multi-level cache strategy. This innovation solves the problem of improper resource utilization in traditional cache management methods, ensures that high-priority data can be accessed quickly, and reduces the storage overhead of low-priority data, effectively improving the resource utilization efficiency of the system.

[0077] In summary, the present invention provides a new search engine data update solution by introducing a multi-dimensional method of intelligent prediction and dynamic cache management to address the problems of poor dynamic response capability, insufficient personalization, and inefficient resource management in the prior art. This solution can effectively improve the update efficiency of the search engine, reduce system resource waste, and enhance the user's search experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] The present invention is further described using the accompanying drawings, but the embodiments in the accompanying drawings do not constitute any limitation to the present invention. A person skilled in the art can obtain other drawings based on the following drawings without creative work.

[0079] Figure 1 The present invention is a flowchart of a data updating method based on a search engine according to an embodiment of the present invention. DETAILED DESCRIPTION

[0080] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0081] In one or more embodiments, Figure 1 As shown, a data updating method based on a search engine is disclosed, the method includes steps S1 to S6, including:

[0082] S1. Obtain the user's search behavior data and extract the user's personalized behavior feature vector.

[0083] Specifically, real-time acquisition of user behavior data from search engines includes but is not limited to:

[0084] Q t Represents the query word vector of the user at time t (the word vector can be processed by a pre-trained NLP model such as Word2Vec or BERT);

[0085] C t Indicates the web page ID (unique identifier) ​​clicked by the user;

[0086] T t Indicates the time a user stays on a web page;

[0087] F t Indicates the user's historical query frequency.

[0088] These data constitute the input dataset of user behavior, reflecting the user's search behavior in a specific time period.

[0089] Furthermore, the multi-dimensional user behavior data is integrated into the user behavior feature vector X t , used to describe the user's search preference at a specific time t. Formula:

[0090] X t =α1·Q t +α2·log(1+T t )+α3·F t

[0091] Among them, X t Represents the comprehensive behavior feature vector of the user at time t, representing the user’s search preference at that moment. t Represents the query word vector of the user at time t, describing the semantic information of the user's search content. t Indicates the time the user stays on the clicked web page, which is logarithmically scaled to avoid weight shift due to large differences in stay time.t Indicates the frequency of the user's past search behavior, reflecting the user's long-term interest in a certain type of information. α1, α2, and α3 represent weight coefficients, which are used to balance the impact of different features on the final feature vector. This formula ensures that behavioral data of different dimensions are integrated into a feature vector with a clear physical meaning, and ensures the balance between data from different sources through reasonable weight adjustment.

[0092] Furthermore, the user's behavior feature vector is dynamic and needs to be updated as time goes by and the user's behavior changes. The present invention uses a sliding window mechanism to dynamically update the user's historical behavior to ensure the real-time nature of the feature vector. The update mechanism is:

[0093] X t+1 =β·X t +(1-β)·X new

[0094] Among them, X t+1 Represents the updated behavior feature vector of the user at time t+1. new Represents the newly generated behavior features. β represents the update coefficient, which is used to control the weight of historical behavior and new behavior. When β is close to 1, it means that the historical behavior features are more likely to be maintained, while when β is close to 0, it means that the latest behavior is more likely to be adopted. This dynamic update mechanism can effectively balance historical behavior and recent behavior, ensuring that the feature vector can reflect the user's long-term interests and current needs.

[0095] Finally, after the above processing, the behavior data of each user at different times are processed to obtain the user behavior feature vector X t , which will be used as input in subsequent steps for user future behavior prediction and data update priority evaluation. Through this series of steps, the user's personalized behavior feature vector is successfully constructed. This feature vector accurately captures the user's search preferences and can change dynamically over time, serving the subsequent user behavior prediction and data update strategy formulation.

[0096] S2. Predict the user's future search needs based on the user's personalized behavior feature vector, and identify the web pages that need to be predicted and updated based on the changes in the web page content.

[0097] Specifically, the present invention designs an adaptive time series model that combines the changes in web page content. Traditional time series models can only capture the time dependency of user behavior and cannot effectively combine the dynamic changes in web page content to predict users' future needs. To this end, the present invention improves the LSTM model so that it can process information about changes in web page content. c , and use it as a control input, allowing the model to automatically adjust its predictions based on changes in web page content.

[0098] The model not only relies on user behavior features X t , and also combined with the weight W of the web page content change c , so that changes in web page content can directly affect the prediction of users' future behavior. This design solves the problem of integrating the continuous changes in user behavior with the sudden changes in web page content, thereby ensuring that the model can accurately predict in complex search scenarios. Formula:

[0099]

[0100] in, It represents the predicted behavior feature vector of the user at the future moment, reflecting the user's possible future search interests. LSTM represents the long short-term memory network, which processes time series data and performs adaptive prediction based on changes in web page content. t W represents the user's behavioral feature input at time t. c It represents the weight of web page content change, reflecting the response of web page content to user demand. This design ensures that the model can capture sudden changes in web page content when user behavior changes, thereby generating more accurate prediction results.

[0101] Understandably, based on the predicted user behavior characteristics and web page content change weight W c , the present invention designs an adaptive pre-fetching strategy, especially for web pages whose content changes frequently and is strongly related to user needs. To this end, the present invention introduces a dynamic threshold θ to control the pre-fetching decision of web pages. This threshold is not fixed, but is automatically adjusted according to the correlation between the historical web page update frequency and the current prediction, so that the system can dynamically determine which web pages need to be updated first according to the current prediction status. Formula:

[0102]

[0103] Among them, S c represents the decision variable, indicating whether web page c should be prefetched and updated. If S c ≥θ, then the web page is selected for prefetching. W represents the predicted user's future behavior feature vector, reflecting the user's future interests. c It represents the weight of web page content change, reflecting the response of web page content to user demand changes. θ represents the adaptive threshold, which is automatically adjusted according to the historical web page update frequency and the current prediction results, and is used to control which web pages need to be updated first.

[0104] This design solves the limitation of the traditional fixed threshold strategy in dealing with the mismatch between web page content and user demand changes. Through adaptive thresholds, the system can flexibly respond to web page prefetching needs in different situations, ensuring that highly relevant web pages can be updated in a timely manner, while low-frequency changing web pages will not consume excessive system resources.

[0105] Finally, this step outputs two important results: Represents the predicted user future behavior feature vector for the next step of dynamic priority evaluation. c Represents the prefetching strategy, which is used to decide which web pages will be updated first.

[0106] These outputs ensure that the search engine can respond promptly to changes in user needs and dynamically adjust data update strategies, thereby improving the search engine's response speed and accuracy.

[0107] S3. Dynamically evaluate the update priority of each web page by combining the personalized behavior feature vector and the changes in the web page content, and generate a prediction strategy.

[0108] Specifically, in order to effectively evaluate the priority of web pages, the present invention designs a web page importance scoring model, which combines the user's future behavior characteristics with the changes in web page content to quantify the priority of the web page.

[0109] The rating is made up of two main factors:

[0110] User relevance scoring: based on predicted user behavior characteristics The present invention calculates the relevance of each web page to the user's future needs. Assume that the content of web page c is represented by a feature vector V c , then user relevance can be calculated by the following formula:

[0111]

[0112] Among them, R c It represents the user relevance score of web page c, which indicates the matching degree between the web page content and the user's future needs. Represents the user's predicted behavior feature vector in the future. c Represents the content feature vector of web page c, which represents the core information of the web page.

[0113] Content change score: In order to reflect the timeliness of web page content, the present invention introduces the weight W of web page content change c , to measure the dynamic change speed of web page content. The greater the change of a web page, the more necessary it is to update it. Combined with the web page change rate, the present invention defines the content change score as:

[0114] C c =Wc

[0115] Among them, C c W represents the content change score of web page c and the content update rate of the web page. c The weight representing the change in web page content, calculated in the previous step.

[0116] Furthermore, the user relevance score and the content change score are combined to generate a comprehensive priority score for the web page. In order to balance the impact of user needs and content changes, the present invention introduces an adjustment coefficient λ to determine the relative importance of user behavior and content changes to web page updates. Formula:

[0117] U c =λ·R c +(1-λ)·C c

[0118] Among them, U c Represents the comprehensive priority score of web page c, which is used to determine the order of web page updates. c Represents the user relevance score, reflecting the matching degree between the webpage content and the user's future needs. c It represents the content change score, which reflects the dynamic changes of the webpage content. λ represents the weight adjustment coefficient, which controls the impact of user demand and webpage changes on the final priority score.

[0119] Furthermore, web page content changes and user demand changes are often not synchronized in time, so traditional priority evaluation methods cannot effectively handle this complexity. This solution introduces an adaptive adjustment coefficient λ to flexibly adjust the relative weights of user demand and web page changes in different scenarios, ensuring that the best update decision can be made in different update scenarios.

[0120] Furthermore, according to the comprehensive priority score U of each web page c , generating a specific pre-fetching strategy. The present invention sorts the web page priorities and selects web pages with scores higher than the set threshold θ p Pre-fetch and update data on web pages. Formula:

[0121]

[0122] Among them, S c represents the prefetch decision variable of web page c, S c =1 means that the web page will be pre-fetched and updated. c =0 means that the web page is not updated. c Represents the comprehensive priority score of the web page. p Indicates the threshold of the prefetching strategy, which is used to control the number of prefetched web pages to ensure that system resources are not over-consumed.

[0123] The present invention designs a dynamic pre-fetch threshold θ p , density is dynamically adjusted based on historical queries and content changes, ensuring fast response when data changes frequently and reducing system load when data changes slowly.

[0124] Output: The output of this step includes:

[0125] S c Represents the web page prefetching strategy, which determines which web pages will be updated first.

[0126] U c Represents a comprehensive priority score list, used to indicate the order in which web pages are updated.

[0127] These outputs provide clear guidance for the system's actual update operations, ensuring that the system prioritizes updating those web pages that are most valuable to users and change the fastest, thereby improving the search engine's response speed and the timeliness of the data.

[0128] S4. Design a multi-level intelligent cache optimization strategy to optimize the search engine's prediction strategy.

[0129] Specifically, this step depends on the web page prefetching decision S generated in the previous step. c and the webpage comprehensive priority score U c . S c represents the web page prefetching decision variable, S c =1 means that the webpage needs to be updated first, S c =0 means that the webpage does not need to be updated. c It represents the comprehensive priority score of a web page, which is obtained by comprehensively evaluating the relevance to user needs and the speed at which the web page content changes.

[0130] Furthermore, the present invention designs a three-level cache system, which are:

[0131] Hot cache: Specially stores web pages that are highly relevant to users' future behavior and have high priority scores to ensure that they respond quickly to user queries.

[0132] Warm cache: Stores web pages with medium priority and moderate access frequency, ensuring that the content has a certain responsiveness but does not consume a large amount of cache resources.

[0133] Cold cache: stores web page data with lower priority and slower change frequency. Such web pages may be accessed occasionally but do not need to be updated frequently.

[0134] The present invention is scored by priority U cDynamically adjust the position of web pages in different cache levels to ensure that high-priority web pages are placed in hot caches first, while low-priority web pages are placed in cold caches. In order to deal with the complexity of the cache system, especially the characteristics of web page priorities changing over time, the present invention proposes an adaptive dynamic cache level allocation mechanism.

[0135] Furthermore, the design of the cache strategy mainly considers the priority score U of the web page. c and the pre-fetching requirement of the web page S c , the present invention designs a multi-dimensional scoring mechanism for each web page, based on which the web page will be assigned to the appropriate cache level. Formula:

[0136]

[0137] Among them, L c Indicates the cache level of web page c, which determines in which cache the web page should be placed. c represents the comprehensive priority score of web page c, which measures the importance of the web page. θ1, θ2 represent cache level thresholds, which are dynamically calculated by the web page priority. θ1 is used to determine whether a web page should enter the hot cache, and θ2 is used to determine whether a web page should enter the warm cache or the cold cache.

[0138] Furthermore, in order to optimize the effect of cache hierarchy division, the present invention designs a regularization term based on the frequency and timeliness of web page accesses for the cache frequency and priority adjustment problem in the search engine scenario. The regularization term is used to dynamically adjust the cache strategy so that the cache system can adapt to changes in user behavior and prevent low-frequency web pages from occupying cache resources for a long time. Formula:

[0139]

[0140] in, represents the comprehensive loss function of web page c being placed in the cache level, L c is a cache level decision. c represents the priority score of web page c. c Indicates the access frequency of web page c within a certain time window, which is counted by log data. c Indicates the time interval between the last update time of web page c and the current time. A and B represent the weights that control the impact of web page access frequency and update time on cache level allocation.

[0141] Among them, the regularization term in this loss function -α·log(f c ) is used to reduce the possibility of high-frequency access web pages occupying low-level caches, -β·Δt cIt is used to prevent outdated content from occupying the cache for a long time and ensure timeliness. Through this formula, the present invention can dynamically adjust the cache strategy and balance the relationship between the web page priority and the cache space usage.

[0142] Furthermore, in order to ensure efficient use of cache space and avoid cache overflow, the present invention designs a cache elimination and migration mechanism based on timeliness and dynamic adjustment of priority. The specific steps are as follows:

[0143] According to the changes in the priority and access frequency of web pages, cache data will be regularly migrated. c When the page load decreases or the access frequency decreases, the web page will be migrated from the hot cache to the warm cache or even the cold cache. This operation is determined by the above formula Decisions are made and adjustments are made through cache logs;

[0144] When the cache space is close to full, the present invention introduces a dynamic elimination mechanism to eliminate web pages based on their priority scores and the current utilization of the cache level. Web page data with low priority and not accessed for a long time will be removed from the cache. The present invention uses a time decay model to control the cache retention time of web pages:

[0145]

[0146] Among them, T c represents the storage time of web page c in the cache, as the priority U c Increase, storage time is extended. max Indicates the maximum cache storage time, which indicates the longest retention time of web page data. ∈ indicates preventing division by zero error when the priority is zero.

[0147] It can be understood that by introducing the regularization term of timeliness control, the present invention can ensure that the cache will not be occupied by low-frequency web page data for too long, while high-frequency and high-priority web pages can obtain more cache retention time. c and its update time interval Δt c , ensuring that the cache space can be flexibly adjusted to reduce the risk of cache overflow.

[0148] Furthermore, the present invention monitors the execution of the cache strategy in real time and designs a feedback loop to dynamically adjust the cache thresholds θ1, θ2 through user query logs and web page access data to ensure the adaptability of the system under high load.

[0149] These outputs ensure that the system can respond to changes in user behavior in real time, dynamically adjust cache space allocation, avoid response delays under high load conditions, and improve the intelligence level of cache management through feedback mechanisms.

[0150] S5. Design an intelligent selective data update mechanism based on the optimized prediction strategy and the update priority of each web page, and perform special optimization for the data update scenario in the search engine.

[0151] Specifically, the goal of this step is to design an intelligent selective data update mechanism based on the cache strategy and web page priority score in the previous step, and to perform special optimization for the data update scenario in the search engine. In the search engine environment, due to the frequent changes in web page content and complex user needs, data must be updated selectively to avoid resource waste and delays caused by full updates. The special feature of this solution is that it introduces additional regularization terms, dynamic threshold adjustment mechanisms, and innovative update scoring mechanisms to ensure efficient use of cache resources and maximize the system's response speed and update accuracy.

[0152] The input data includes the web page cache level information L generated in the previous step c and web page priority score U c , used for intelligent selective update decision. Among them, L c Represents the web page cache hierarchy information, which determines in which cache the web page is stored (hot cache, warm cache or cold cache). c It represents the comprehensive priority score of a web page, reflecting user needs, content relevance, and change frequency.

[0153] Furthermore, the present invention designs a special update scoring mechanism P c , which is used to evaluate whether each web page should be updated. The scoring mechanism consists of the following three parts:

[0154] Priority Rating U c It reflects the importance of a web page and is calculated based on the user's query needs and the frequency of changes in web page content.

[0155] Content change rate C c It represents the degree of change of the web page content. This value is calculated in the previous step by the weight W of the web page content. c get.

[0156] Cache level contribution L c It means that the storage situation of web pages in different cache levels is taken into consideration, and web pages in hot cache are updated first, while web pages in cold cache can be updated later.

[0157] Furthermore, in order to ensure the flexibility of the update strategy, the present invention designs a comprehensive update scoring formula with a regularization term:

[0158]

[0159] Among them, P crepresents the comprehensive update score of web page c, which is used to decide whether to update the data. ι,κ,ν are the priority scores U respectively. c , content change rate C c and the weight of cache level contribution. represents the cache level contribution function, which is used to represent the impact of the cache level on the update decision and is defined as:

[0160]

[0161] Web pages in the hot cache have a greater cache level contribution due to their frequent access and high priority, and therefore need to be updated first. Represents a newly introduced regularization term, which is used to balance system resource consumption and avoid excessive updates of high-frequency, low-change web pages. Reflects the recent update frequency of the web page to avoid redundant operations caused by web pages that are updated too frequently. The formula is defined as:

[0162]

[0163] Among them, f c Indicates the access frequency of web page c. Δt c Indicates the time interval between the last update of web page c and the current time. This regularization term ensures that web pages with low frequency of update and long time of no update are given priority, while web pages with high frequency of update need to meet higher standards before they can be updated again, to prevent excessive waste of resources.

[0164] It is understandable that by adding cache level contribution and regularization items, the system can intelligently determine which web pages should be updated first, avoiding blind updates based solely on the web page change rate or priority score. In particular, the introduction of regularization items effectively controls the system's excessive updates of high-frequency web pages, reducing unnecessary calculations and data traffic consumption.

[0165] Furthermore, the present invention introduces a dynamically updated decision threshold θ u , used to flexibly adjust the update strategy. Threshold θ u The purpose of the design is to adaptively control the frequency of web page updates based on the system load and historical updates. Specifically, θ u Dynamically adjusted according to the following formula:

[0166] θ u =θ0·(1+κ·LoadFactor)

[0167] Among them, θ uRepresents the threshold of the current update decision. θ0 represents the basic update threshold, which is set according to historical query data. κ represents the load influence coefficient, which is used to control the impact of load on the update threshold. LoadFactor represents the current load factor of the system, reflecting the utilization rate of system resources, and the calculation formula is as follows:

[0168]

[0169] Among them, when the system load increases, the threshold θ u will be increased accordingly to reduce unnecessary update operations; when the load is low, θ u The threshold value is lowered, allowing more web pages to be updated. This method of dynamically adjusting the threshold ensures that the system can adaptively adjust the update strategy under both high and low load conditions.

[0170] Further, according to the comprehensive update score P c and update the decision threshold θ u The present invention determines whether a web page is updated, and the specific steps are as follows:

[0171] If P c ≥θ u , then web page c is selected for update, and the system will crawl data, parse content, and update index for the web page.

[0172] If P c <θ u , the webpage will be delayed or not updated, pending subsequent evaluation.

[0173] The execution process of this update operation can quickly respond to changes in user behavior and ensure that high-priority content is updated in a timely manner, reducing the resource usage of low-priority content. The execution process includes:

[0174] Data crawling: Extracting content from original web pages and storing it in cache or index.

[0175] Content parsing: Use the preset parser to perform semantic analysis on web page content and generate new index data.

[0176] Index update: Replace the old index with the latest data to ensure that the latest web page information is displayed when querying.

[0177] Feedback mechanism and dynamic optimization: In order to ensure that the system can maintain optimal performance during long-term operation, the present invention designs a feedback mechanism to optimize and adjust the weight parameters α, β, γ and the coefficient λ of the regularization term by monitoring the query response time, user click rate and other data after the update operation. At the same time, the system load factor LoadFactor will also be dynamically updated to ensure load balance.

[0178] The final output is Uupdate Web page update decisions indicate which web pages need to be updated; updated web page data and indexes ensure that search engines provide the latest content to users; by monitoring system behavior, the parameter configuration of the update strategy is optimized to form an adaptive adjustment process.

[0179] This solution realizes an efficient and flexible selective data update mechanism by combining the regularization control of web page priority, content change rate, cache level contribution, and update frequency. In particular, through dynamic threshold adjustment and regularization terms, the system can adaptively update according to the load status and web page access characteristics, optimize the efficiency of system resource utilization, and ensure timely updates of high-priority web pages.

[0180] S6. Establish an adaptive feedback loop to dynamically optimize the overall performance of the search engine.

[0181] Specifically, the input of this step comes from the updated decision variable U in the previous step. update , and the system feedback data R sys The input includes:

[0182] U update Represents the update decision variable of the web page, indicating which web pages are selected for update (U update =1) or delayed update (U update =0).

[0183] R sys Represents system feedback data, which contains data in multiple dimensions, including real-time dynamic indicators such as system load, query response time, and user click rate. These data are used as feedback to guide the dynamic adjustment of system parameters.

[0184] Furthermore, in order to achieve adaptive optimization, the system needs to continuously adjust its own parameters based on feedback information from actual operation. The core optimization goal is to ensure that unnecessary update operations are reduced under high load and the utilization efficiency of system resources is improved; while under low load, the update frequency is increased to improve the real-time nature of data. The present invention achieves this goal through the following three core optimization steps:

[0185] (1) The update decision in the previous step is based on the threshold θ u To decide whether to update the web page. The present invention dynamically adjusts the threshold value according to the system load through a feedback loop. The system load factor LoadFactor is defined as the ratio of the current system resource usage to the maximum resource:

[0186]

[0187] in, Indicates the current CPU usage of the system. Indicates the current memory usage of the system. and Indicates the maximum CPU and memory capacity of the system.

[0188] (2) Based on the load factor LoadFactor max , dynamically adjust the threshold θ u , ensuring that the system can reduce update operations when the load is high, and increase update operations when the load is low:

[0189] θ u =θ0·(1+η·LoadFactor max )

[0190] Among them, θ u represents the dynamic threshold used for update decision, controlling whether the web page should be updated. θ0 represents the basic update threshold, reflecting the initial value under normal load conditions. η represents the adjustment coefficient, which is used to control the degree of influence of the load factor on the threshold.

[0191] It can be understood that by introducing the system load factor, the system can automatically increase the threshold during peak load periods, reduce unnecessary update operations, and thus alleviate resource pressure; when the load is low, the system can lower the threshold, allowing more web page content to be updated, ensuring the real-time nature of the data and user experience.

[0192] Furthermore, in order to further enhance the adaptive capability of the system, the present invention designs a weight optimization mechanism for dynamically adjusting the weights α, β, and γ of each factor in the update score. These weights affect the priority score U in the web page update decision. c , content change rate C c and cache level contribution L c The system optimizes the function by feedback Dynamically adjust these weights so that the system pays more attention to resource usage when the load is high, and pays more attention to the content change rate and cache level when the load is low. The specific optimization formula is as follows:

[0193]

[0194] in, It represents the feedback optimization function, which is used to adjust the weight parameters ι,κ,ν, affecting the calculation of the web page update score. Indicates the query response time of the current system. Indicates the user's click rate data, reflecting the relevance and satisfaction of the user's query results. max Represents the maximum allowed response time of the system as a performance target. targetIt represents the predetermined target click-through rate, reflecting the user click-through rate that the system expects to achieve. ω1, ω2 represent coefficients, which are used to balance the importance of response time and click-through rate in the weight optimization process.

[0195] Furthermore, by introducing a feedback optimization function, the system can adaptively adjust the weights of each item in the web page update score, so that when the response time is long, the system pays more attention to resource utilization, and when the click rate decreases, the system considers the content update frequency more. This mechanism ensures that the system always maintains the best performance under dynamic load.

[0196] Furthermore, in addition to optimizing the update threshold and weight distribution, the system also introduces a feedback correction mechanism based on historical data to gradually optimize the regularization term in the comprehensive update scoring formula. Specifically, the system adjusts the regularization term based on historical update results and user behavior feedback. Corrections are made to ensure that low-priority, high-frequency access web pages are not over-updated, but optimized according to actual needs. The optimization function of the regularization term is as follows:

[0197]

[0198] Among them, λ new represents the updated regularization parameter, which is used to control the update frequency of frequently visited web pages. old represents the original regularization parameter. μ represents the adjustment coefficient, which is used to control the influence of the regularization parameter when the click rate deviates from the target. Indicates the actual user click rate. target Indicates the target click-through rate.

[0199] Among them, when the user click rate is lower than the target click rate, the regularization term will be adjusted to reduce the update priority of frequently visited web pages, so as to better allocate system resources.

[0200] Furthermore, the system continuously monitors R sys The feedback data in the system includes system load, response time, user click rate, etc. According to the load factor in the feedback, the update threshold θ is adjusted in real time u , reduce the update frequency when the load is high, and increase the update frequency when the load is low. Optimize the function through feedback Dynamically adjust the weights α, β, and γ in the update score so that the system can balance the update frequency and performance in different scenarios. Combine historical data and user behavior feedback to optimize the regularization term Prevent low priority web pages from being updated frequently.

[0201] Understandably, in the patent scenario of search engines, since the system needs to process a large number of user requests and maintain the real-time nature of data, the feedback mechanism is the key to ensuring the adaptive optimization of the system. Through the feedback loop, the system can adjust various parameters in real time according to the actual operating conditions to ensure the optimal response under different loads and user behaviors. Efficient feedback optimization not only reduces unnecessary resource consumption, but also improves the accuracy of user queries and the overall performance of the system.

[0202] This solution builds a complete adaptive feedback loop by dynamically adjusting update thresholds, weight optimization, and regularization term correction. The system can flexibly adjust update strategies and cache management under different load and user behavior conditions to achieve optimal performance, which is particularly suitable for complex high-concurrency scenarios such as search engines.

[0203] These are just some preferred embodiments of the present invention, which certainly cannot be used to limit the scope of rights of the present invention. Ordinary technicians in this field can understand that all or part of the processes of implementing the above embodiments and making equivalent changes according to the claims of the present invention still fall within the scope of the invention.

Claims

1. A data updating method based on a search engine, characterized in that: The method comprises the following steps: S1. Obtain the user's search behavior data and extract the user's personalized behavior feature vector, including: Integrate the user's search behavior data into a user behavior feature vector to represent the user's search preference within a specific time period, and use a sliding window mechanism to dynamically update the user's behavior data; S2. Predict the user's future search needs based on the user's personalized behavior feature vector, and identify the web pages that need to be predicted and updated based on the changes in the web page content, including: According to the changes in the webpage content, a long short-term memory network is constructed to predict the user's future behavior feature vector, and at the same time, an adaptive prediction strategy is designed according to the changes in the webpage content and the predicted user's future behavior feature vector. The adaptive prediction strategy is set for webpages with frequent changes in webpage content and strong correlation with user needs; S3. Dynamically evaluate the update priority of each web page by combining the personalized behavior feature vector and the changes in web page content, and generate a prediction strategy, including: A web page importance scoring model is designed based on the user's future behavior feature vector and the change of web page content. The web page importance scoring model includes a user relevance score and a content change score to obtain a comprehensive web page score. The web page priority is sorted according to the comprehensive web page score to generate a specific prediction strategy; S4. Design a multi-level intelligent cache optimization strategy to optimize the search engine's prediction strategy, including: A three-level cache system is designed, namely: hot cache, warm cache and cold cache. According to the comprehensive web page score, the highest priority web page is placed in the hot cache first, while the lowest priority web page is placed in the cold cache, which is expressed as follows: Among them, L c Indicates the cache level of web page c, which determines in which cache the web page should be placed; U c represents the comprehensive webpage score of webpage c, which measures the importance of the webpage; θ1, θ2 represent cache level thresholds, which are dynamically calculated by the webpage priority. θ1 is used to determine whether a webpage should enter the hot cache, and θ2 is used to determine whether a webpage should enter the warm cache or the cold cache; The design dynamically adjusts the cache strategy based on the regularization term of web page access frequency and timeliness, so that the cache system can adapt to changes in user behavior and prevent low-frequency web pages from occupying cache resources for a long time, as shown below: in, represents the comprehensive loss function of web page c being placed in the cache level, L c is the cache level of web page c; U c represents the comprehensive webpage score of webpage c; f c Indicates the access frequency of web page c within a certain time window, which is counted by log data; Δt c represents the time interval between the last update time of web page c and the current time; A and B represent the weights that control the impact of web page access frequency and update time on cache level allocation; S5. Design an intelligent selective data update mechanism based on the optimized prediction strategy and the update priority of each web page, and perform special optimization for the data update scenario in the search engine, including: An update scoring mechanism is designed to evaluate whether each web page should be updated, wherein the update scoring mechanism includes the comprehensive web page score, content change score and web page cache level information of web page c, which is expressed as follows: Among them, P c represents the comprehensive update score of web page c, which is used to decide whether to update data; ι, κ, ν are the priority scores U c , content change rate C c and the weight of the cache level contribution, Represents the regularization term for balancing system resource consumption; represents the cache level contribution function, which is used to represent the impact of the cache level on the update decision and is defined as: At the same time, a dynamic update decision threshold θ is introduced u , flexibly adjust the update strategy, adaptively control the frequency of web page updates, and finally determine whether to update the web page based on the comprehensive update score of web page c and the update decision threshold, and finally obtain the web page update decision U update : If P c ≥θ u , then web page c is selected for update, and data crawling, content parsing and index update will be performed on the web page; If P c <θ u , then the webpage will be delayed or not updated, pending subsequent evaluation; S6. Establish an adaptive feedback loop to dynamically optimize the overall performance of the search engine.

2. A data updating method based on a search engine according to claim 1, characterized in that: The user's search behavior data includes the user's query word vector at time t, the web page number clicked by the user, the time the user stayed on the web page, and the user's historical query frequency; The user behavior feature vector is expressed as follows: X t =α1·Q t +α2·log(1+T t )+α3·F t Among them, X t represents the comprehensive behavior feature vector of the user at time t, representing the user's search preference at that moment; Q t represents the query word vector of the user at time t, describing the semantic information of the user's search content; T t Indicates the time the user stays on the clicked web page, which is logarithmically scaled to avoid weight shift due to huge differences in stay time; F t It represents the frequency of the user's past search behavior, reflecting the user's long-term interest in a certain type of information; α1, α2, α3 represent weight coefficients, which are used to balance the impact of different features on the final feature vector.

3. A data updating method based on a search engine according to claim 1, characterized in that: The adaptive prediction strategy controls the prediction decision of web pages by introducing a dynamic threshold θ. The dynamic threshold θ is automatically adjusted according to the correlation between the historical web page update frequency and the current prediction. It can dynamically determine which web pages need to be updated first according to the current prediction status, as shown below: Among them, S c represents the decision variable, indicating whether web page c should be predicted to be updated. If S c ≥θ, then the web page is selected for prediction; W represents the predicted user's future behavior feature vector, reflecting the user's future interests; c It represents the weight of the change of web page content, reflecting the response of web page content to the change of user needs; θ represents the adaptive threshold, which is automatically adjusted according to the historical web page update frequency and the current prediction results, and is used to control which web pages need to be updated first.

4. The data updating method based on search engine according to claim 1, characterized in that: The user relevance score calculates the relevance of each web page to the user's future needs based on the user's future behavior feature vector; the content change score is the weight of the web page content change.

5. A search engine based data updating method according to claim 4, characterized in that: The user relevance score and the content change score are combined to generate a comprehensive webpage score, which is expressed as follows: U c =λ·R c +(1-λ)·C c Among them, U c represents the comprehensive priority score of web page c, which is used to determine the order of web page updates; R c Represents the user relevance score, reflecting the matching degree between the webpage content and the user's future needs; C c represents the content change score, reflecting the dynamic changes of web page content; λ represents the weight adjustment coefficient, controlling the impact of user needs and web page changes on the final priority score; Sort the web page priorities according to the comprehensive web page score of each web page, and select the web pages with scores higher than the set threshold θ p The web page is used to predict and update data, as shown below: Among them, S c represents the prediction decision variable of web page c, S c =1 means that the webpage will be predicted to be updated, S c =0 means that the webpage is not updated; θ p Indicates the threshold of the prediction strategy, which is used to control the number of predicted web pages to ensure that system resources are not over-consumed.

6. A data updating method based on a search engine according to claim 1, characterized in that: In S4, the loss function The regularization term in -α·log(f c ) is used to reduce the possibility of high-frequency access web pages occupying low-level caches, -β·Δt c Used to prevent outdated content from occupying the cache for a long time.

7. A search engine based data updating method according to claim 6, characterized in that: The S4 further includes: According to the changes in the priority and access frequency of web pages, cache data is regularly migrated. When the cache space is close to full, a dynamic elimination mechanism is introduced to eliminate web pages based on their priority scores and the current utilization of the cache layer. Web page data with low priority and not accessed for a long time will be removed from the cache. The time decay model is used to control the cache retention time of web pages, which is expressed as follows: Among them, T c Indicates the storage time of web page c in the cache, as the priority U c Increase, storage time is extended; T max Represents the maximum cache storage time, indicating the longest retention time of web page data; ∈ represents the prevention of division by zero error when the priority is zero.

8. The data updating method based on search engine according to claim 1, characterized in that: The dynamically updated decision threshold θ u According to the system load and historical update status, the frequency of web page updates is adaptively controlled, as shown below: i u =θ0·(1+η·LoadFactor) Among them, θ u represents the dynamic threshold used for update decision, which controls whether the web page should be updated; θ0 represents the basic update threshold, which reflects the initial value under normal load conditions; η represents the adjustment coefficient, which is used to control the influence of the load factor on the threshold; LoadFactor represents the current load factor, which reflects the system resource utilization rate, and the calculation formula is as follows: When the system load increases, the threshold θ u will be increased accordingly, reducing unnecessary update operations; when the load is reduced, θ u will be lowered, allowing more pages to be updated; According to the comprehensive update score P c and update the decision threshold θ u , decide whether to update the web page, the specific steps are as follows: If P c ≥θ u , then web page c is selected for update, and data crawling, content parsing and index update will be performed on the web page; If P c <θ u , the webpage will be delayed or not updated, pending subsequent evaluation.

9. The data updating method based on search engine according to claim 1, characterized in that: The specific analysis of S6 is as follows: According to the updated decision variables of web page c, the threshold θ is dynamically adjusted by designing a feedback optimization model through a feedback loop u ; Design feedback optimization function to dynamically adjust the priority score U c , content change rate C c and the weights ι,κ,ν contributed by the cache levels, so that the system focuses on resource usage when the load is high, and on the content change rate and cache level when the load is low, as shown below: in, represents the feedback optimization function, which is used to adjust the weight parameters ι,κ,ν, affecting the calculation of the web page update score; Indicates the query response time of the current system; Represents the user's click rate data, reflecting the relevance and satisfaction of the user's query results; T max represents the maximum allowable response time of the system as a performance target; C target It represents the predetermined target click-through rate, reflecting the user click-through rate that the system expects to achieve; ω1, ω2 represent coefficients, which are used to balance the importance of response time and click-through rate in the weight optimization process; Construct a feedback correction mechanism based on historical data to correct the regularization term in the comprehensive update score, wherein the feedback correction mechanism based on historical data corrects the regularization term according to the historical update results and user behavior feedback. Corrections are made to ensure that low-priority, high-frequency access web pages are not over-updated, but optimized according to actual needs, as shown below: Among them, λ new represents the updated regularization parameter, which is used to control the update frequency of frequently visited web pages; old represents the original regularization parameter; μ represents the adjustment coefficient, which is used to control the influence of the regularization parameter when the click rate deviates from the target; Indicates the actual user click rate; C target Indicates the target click-through rate.

10. A data updating method based on a search engine according to claim 9, characterized in that: The feedback optimization model is constructed as follows: LoadFactor max It is defined as the ratio of the current system resource usage to the maximum resource, expressed as follows: in, Indicates the current CPU usage of the system. Indicates the current memory usage; and Indicates the maximum CPU and memory capacity of the system; According to the load factor LoadFactor max , dynamically adjust the threshold θ u , ensuring that updates are reduced when the load is high and increased when the load is low: i u =θ0·(1+η·LoadFactor max ) Among them, θ u represents the dynamic threshold used for update decision, which controls whether the web page should be updated; θ0 represents the basic update threshold, which reflects the initial value under normal load conditions; η represents the adjustment coefficient, which is used to control the influence of the load factor on the threshold.

Images