Real-time search cache replacement method and system based on machine learning

Abstract

The invention relates to the technical field of computer storage, and discloses a real-time search cache replacement method and system based on machine learning, and the method comprises the steps: obtaining multi-dimensional search data in the system, carrying out the data preprocessing of the multi-dimensional search data, building an LSTM (Long Short-Term Memory) network model, and carrying out the real-time search cache replacement through the LSTM network model. Using an improved whale optimization algorithm IWOA to perform parameter optimization on the LSTM long short-term memory network model, designing a multi-target reward function based on a PPO algorithm, using the multi-target reward function to optimize a current cache state and weight parameters of an online prediction model, inputting multi-dimensional feature real-time data into the prediction model to perform training, and obtaining a prediction result; meanwhile, the dynamic weight of the model is calculated, and target adjustment of the cache replacement rule is carried out based on an incremental data set. The anti-interference capability and the long-period learning efficiency of the cache system are obviously enhanced, and an intelligent cache solution with low delay and high resource utilization rate is provided for a high-concurrency scene.

Description

Technical Field

[0001] The present invention relates to the field of computer storage technology, and in particular to a real-time search cache replacement method and system based on machine learning. Background Art

[0002] Existing cache replacement algorithms (such as LRU, LFU, ARC, etc.) rely on fixed rules and are difficult to adapt to dynamically changing access patterns; traditional algorithms do not consider multi-dimensional features other than temporal locality and spatial locality, resulting in poor performance when faced with complex and nonlinear access patterns. The prediction accuracy of rule-based algorithms drops significantly in burst traffic scenarios, making it difficult to effectively deal with traffic peaks and fluctuations. In addition, although existing AI cache solutions can capture changes in access patterns through machine learning models, there is a delay contradiction between offline training and online decision-making, which cannot meet real-time requirements and limits their widespread application in production environments. Therefore, there is an urgent need for a cache replacement algorithm that can take into account dynamic adaptability, multi-dimensional feature analysis, burst traffic response, and real-time decision-making capabilities to improve the overall system performance and resource utilization. Summary of the invention

[0003] The purpose of the present invention is to solve the above problems and to design a real-time search cache replacement method and system based on machine learning.

[0004] A first aspect of the present invention provides a real-time search cache replacement method based on machine learning, the real-time search cache replacement method based on machine learning comprising the following steps: Acquire multi-dimensional search data in the system, perform data preprocessing on the multi-dimensional search data, and obtain multi-dimensional feature real-time data; An LSTM long short-term memory network model is established, and the improved whale optimization algorithm IWOA is used to optimize the parameters of the LSTM long short-term memory network model to obtain an IWOA-LSTM online prediction model; Designing a multi-objective reward function based on the PPO algorithm, and using the multi-objective reward function to optimize the current cache state and weight parameters of the IWOA-LSTM online prediction model to obtain the PPO-IWOA-LSTM online prediction model; Input the multi-dimensional feature real-time data into the PPO-IWOA-LSTM online prediction model for training, calculate the dynamic weight of the model, and adjust the cache replacement rule according to the calculated dynamic weight; Obtain a feedback data set after cache replacement, record the actual hit results after each PPO-IWOA-LSTM online prediction model decision, build an incremental data set, and adjust the target of the cache replacement rule based on the incremental data set.

[0005] Optionally, in a first implementation of the first aspect of the present invention, the acquiring of multi-dimensional search data in the system, and performing data preprocessing on the multi-dimensional search data to obtain multi-dimensional feature real-time data include: Acquire multi-dimensional search data in the system, wherein the multi-dimensional search data includes at least user behavior data, content metadata and context data; the user behavior data includes at least search term frequency, click-through rate and dwell time; the content metadata includes at least cache object size data, creation time data and content type data; the context data includes at least user geographic data, device type data and network status data; The multi-dimensional search data is preprocessed based on the window dynamic adjustment algorithm, and the request rate change rate is calculated according to the request rate change rate. Elastic window mechanism: ; in, Indicates the dynamically adjusted window time, Indicates the basic window time, It represents the sensitivity coefficient, and its value is 0.25; is the request rate change rate, , Indicates the historical maximum request rate; the constraints are: ; Slice the window data processed by the elastic window mechanism, divide the window into 5-second sub-windows, manage them using a ring buffer, and obtain window slice data; perform core feature calculation on the window slice data, and the core feature calculation includes at least time-attenuated access frequency features, semantic relevance features, and spatiotemporal relevance features; The calculation formula of time-decay access frequency feature is as follows: ; in, Representation Object The time-decayed access frequency characteristic value of Represents the attenuation factor, text data , media data ; Representation Object The number of visits within the window; Indicates the current time. Representation Object No. Time of visit; The semantic relevance feature calculates the similarity between the query and the cached content based on Sentence-BERT. The calculation formula is as follows: ; in, Representation query With cached content The semantic similarity of Represents the 768-dimensional embedding vector of Query, calculated using the all-mpnet-base-v2 model. Embedding vector representing the cached content title; Representation vector and The cosine similarity of The spatiotemporal correlation features include spatial locality and temporal locality, wherein the spatial locality is calculated by requesting similarity between adjacent regions, and the temporal locality is calculated by an autocorrelation function; the core feature data is obtained by core feature calculation; the core feature data is normalized and then categorical feature encoding is performed to obtain multi-dimensional feature real-time data.

[0006] Optionally, in a second implementation of the first aspect of the present invention, the LSTM long short-term memory network model is established, and the parameters of the LSTM long short-term memory network model are optimized using an improved whale optimization algorithm IWOA to obtain an IWOA-LSTM online prediction model, including: Establish an LSTM long short-term memory network model, which at least includes a forget gate, an input gate, a cell update state, and an output gate; the forget gate is used to decide whether to retain or discard historical information in the cell state, the input gate is used to filter new information into the cell state, the cell update state is used to integrate the forgetting and input gate results, and the output gate is used to generate the current hidden state; The nonlinear convergence factor in the WOA whale optimization algorithm is improved. The calculation formula for nonlinear adjustment is as follows: ; in, Indicates time The nonlinear convergence factor is represents the maximum value of the convergence factor, represents the minimum value of the convergence factor, Indicates the current iteration number, represents the maximum number of iterations, , which means controlling the decay rate; The sine and cosine perturbations in the WOA whale optimization algorithm are enhanced, and the perturbation factors are generated: ; in, represents the disturbance factor, represents the disturbance intensity coefficient, ; represents a random number, which obeys uniform distribution. , dynamically adjust individual search radius; position update correction: ; in, Indicates the position update amount, represents the location of the current optimal solution, represents the control vector, usually represented by the convergence factor and random number generation, is a distance vector, which indicates the distance between the individual and the current optimal solution. Indicates the introduction of random disturbance terms; The Levy flight mutation strategy in the WOA whale optimization algorithm is updated. When it falls into the local optimum, the mutation mechanism is triggered to obtain the IWOA whale optimization algorithm. The calculation formula for the variable step length is as follows: ; in, It means that Levi's flight becomes asynchronously longer. and Represents a random variable, which follows a normal distribution , , used to control the long-tail distribution and support large-span jump search; the population update rule calculation formula is as follows: ; in, represents the new population position, Indicates the current best position. represents a randomly selected position, Represents the step size factor.

[0007] Optionally, in a third implementation of the first aspect of the present invention, the establishing of an LSTM long short-term memory network model, and using an improved whale optimization algorithm IWOA to perform parameter optimization on the LSTM long short-term memory network model to obtain an IWOA-LSTM online prediction model, further includes: The IWOA whale optimization algorithm is used to optimize the key hyperparameters of the LSTM long short-term memory network model, and the key hyperparameters include at least the number of hidden layer neurons. , learning rate , batch size and Dropout rate ; Encode the key hyperparameter combination as the whale individual position vector , the fitness function is minimized by the mean square error MSE of the validation set, and its calculation formula is as follows: ; in, represents the fitness function value, represents the number of samples in the validation set, is the predicted value of the LSTM long short-term memory network model, is the true value of the LSTM long short-term memory network model; Obtain the historical search cache data in the system, use the historical search cache data to train the LSTM long short-term memory network iteratively, initialize the population of the IWOA whale optimization algorithm, set the population size to 50, decode the parameters of each individual and train the LSTM long short-term memory network, set the fixed number of iterations to 100, calculate the fitness value, and update the global optimal solution; According to the IWOA whale optimization algorithm rules, the population position is updated, and the Levy flight mutation is performed. When the number of iterations reaches 100, the iteration is terminated to obtain the latest hyperparameters. The latest hyperparameters are substituted into the LSTM long short-term memory network model to obtain the IWOA-LSTM online prediction model.

[0008] Optionally, in a fourth implementation of the first aspect of the present invention, the multi-objective reward function is designed based on the PPO algorithm, and the multi-objective reward function is used to optimize the current cache state and weight parameters of the IWOA-LSTM online prediction model to obtain the PPO-IWOA-LSTM online prediction model, including: Get prediction error rewards , cache efficiency bonus and resource consumption penalties , the calculation formula of the multi-objective reward function is as follows: ; in, represents the total reward, , and represents the weight coefficient, , represents the fitness function value; Get the hidden state of the IWOA-LSTM online prediction model , current cache status and resource monitoring indicators, the state value function calculation formula of the PPO algorithm is as follows: ; in, represents the state value function, represents the hidden state of the online prediction model, Represents the current cache state, MLP is a multi-layer perceptron, and the output scalar value evaluates the long-term benefit of the current state. The GAE generalized advantage is used to estimate the balance deviation and variance: ; ; in, represents the discount factor, , Indicates GAE parameters, Used to control future reward decay, represents the timing difference error, Indicates the instant reward at the current moment, The total time step, represents the current time step, represents the index of the time step, Indicates that at time step The state value function of Indicates that from the current time step The beginning time steps, Indicates The temporal difference error of time steps; The PPO objective function includes the clipping mechanism and the KL divergence constraint, and the calculation formula is as follows: ; in, represents the policy parameters, Represents the time step Actions taken, Indicates that at time step status, Indicates that the current strategy is in state Take Action The probability of Indicates that the old policy is in the state Take Action The probability of represents the advantage function value, represents the clipping threshold, Used to limit the scope of policy updates; The cache is dynamically updated according to the action, and the prediction results of the IWOA-LSTM online prediction model are combined with the LRU elimination strategy for dynamic adjustment. The parameters of the IWOA-LSTM online prediction model are incrementally updated through the output of the policy network to obtain the PPO-IWOA-LSTM online prediction model.

[0009] Optionally, in a fifth implementation of the first aspect of the present invention, the multi-dimensional feature real-time data is input into the PPO-IWOA-LSTM online prediction model for training, and the dynamic weight of the model is calculated, and the cache replacement rule is adjusted according to the calculated dynamic weight, including: The multi-dimensional feature real-time data is input into the PPO-IWOA-LSTM online prediction model for training, and a feature attention mechanism is constructed based on the LSTM hidden state and the PPO algorithm strategy network output: ; in, is the normalization function, is the query vector, For the -dimensional feature key vector, Represents the dynamic weight of the feature, satisfying , represents the query weight matrix, represents the hidden state of the PPO-IWOA-LSTM online prediction model, represents the key weight matrix, Indicates Dimensional features; Combined with the action output by the PPO algorithm strategy network , adjust the feature fusion weight in real time, and the combined calculation formula is as follows: ; in, represents the updated feature fusion weight, represents the feature fusion weight at the current moment, represents the weight learning rate, which is jointly optimized by the gradient direction and the strategy action. represents the hyperbolic tangent function, Represents the loss function Partial derivatives with respect to feature fusion weights; Generate cache item scores based on dynamic weights and prediction results: ; in, Represents the cache item score, , and represents the weight coefficient, express The dynamic weight of the feature at each moment, Indicates Dimension features in cache items Contribution in Represents the long-term benefits of PPO algorithm value network evaluation; Represents the cache item lifetime penalty item; The scoring is combined with the traditional LRU / LFU rules, and the cache replacement rules are adjusted according to the calculated dynamic weights.

[0010] Optionally, in a fifth implementation of the first aspect of the present invention, obtaining a feedback data set after cache replacement, recording actual hit results after each PPO-IWOA-LSTM online prediction model decision, constructing an incremental data set, and performing target adjustment of cache replacement rules based on the incremental data set, includes: Obtain the feedback data set after cache replacement, build a four-tuple feedback data structure, record the spatiotemporal association between model decisions and real accesses, calculate the KL divergence of feature distribution based on the sliding window, identify data distribution mutations, and obtain a dynamic feature feedback data set; Performing feature enhancement on the dynamic feature feedback data set, using a ring buffer to achieve efficient incremental data management, eliminating the earliest sample when new data arrives, and obtaining an incremental data set; The incremental data set is jointly optimized by combining the prediction error and the strategy benefit, and the optimized incremental data set is input into the PPO-IWOA-LSTM online prediction model for retraining; A feedback-driven adaptive coefficient is introduced to score and calibrate the training results of the PPO-IWOA-LSTM online prediction model, automatically select the optimal strategy combination according to the incremental data distribution, and adjust the target of the cache replacement rule based on the optimal strategy combination.

[0011] A second aspect of the present invention provides a real-time search cache replacement system based on machine learning, wherein the real-time search cache replacement system based on machine learning comprises the following modules: A data acquisition processing module is used to acquire multi-dimensional search data in the system, perform data preprocessing on the multi-dimensional search data, and obtain multi-dimensional feature real-time data; A model building optimization module is used to build an LSTM long short-term memory network model, and use an improved whale optimization algorithm IWOA to optimize the parameters of the LSTM long short-term memory network model to obtain an IWOA-LSTM online prediction model; A model parameter adjustment module, used to design a multi-objective reward function based on the PPO algorithm, and use the multi-objective reward function to optimize the current cache state and weight parameters of the IWOA-LSTM online prediction model to obtain a PPO-IWOA-LSTM online prediction model; A cache rule replacement module, used for inputting the multi-dimensional feature real-time data into the PPO-IWOA-LSTM online prediction model for training, calculating the dynamic weight of the model, and adjusting the cache replacement rule according to the calculated dynamic weight; The cache rule adjustment module is used to obtain the feedback data set after cache replacement, record the actual hit results after each PPO-IWOA-LSTM online prediction model decision, build an incremental data set, and perform target adjustment of the cache replacement rule based on the incremental data set.

[0012] Optionally, in a first implementation of the second aspect of the present invention, the model parameter adjustment module includes the following units: A hyperparameter acquisition unit is used to optimize the key hyperparameters of the LSTM long short-term memory network model using the IWOA whale optimization algorithm, wherein the key hyperparameters include at least the number of neurons in the hidden layer. , learning rate , batch size and Dropout rate ; A hyperparameter setting unit, used to encode the key hyperparameter combination into a whale individual position vector , the fitness function is minimized by the mean square error MSE of the validation set, and its calculation formula is as follows: ; in, represents the fitness function value, represents the number of samples in the validation set, is the predicted value of the LSTM long short-term memory network model, is the true value of the LSTM long short-term memory network model; A training iteration unit is used to obtain historical search cache data in the system, perform training iterations on the LSTM long short-term memory network with the historical search cache data, initialize the population of the IWOA whale optimization algorithm, set the population size to 50, decode parameters for each individual and train the LSTM long short-term memory network, set the fixed number of iterations to 100, calculate the fitness value, and update the global optimal solution; The hyperparameter obtaining unit is used to update the population position according to the IWOA whale optimization algorithm rules, perform Levy flight mutation, terminate the iteration when the number of iterations reaches 100, obtain the latest hyperparameters, substitute the latest hyperparameters into the LSTM long short-term memory network model, and obtain the IWOA-LSTM online prediction model.

[0013] Optionally, in a second implementation of the second aspect of the present invention, the cache rule adjustment module includes the following units: Feedback data acquisition unit, used to obtain the feedback data set after cache replacement, build a four-tuple feedback data structure, record the spatiotemporal association between model decisions and real access, calculate the KL divergence of feature distribution based on the sliding window, identify data distribution mutations, and obtain a dynamic feature feedback data set; A data feature enhancement unit, used to enhance the features of the dynamic feature feedback data set, using a ring buffer to achieve efficient incremental data management, eliminating the earliest samples when new data arrives, and obtaining an incremental data set; An incremental data training unit is used to jointly optimize the incremental data set by combining the prediction error and the strategy benefit, and input the optimized incremental data set into the PPO-IWOA-LSTM online prediction model for re-training; The rule target adjustment unit is used to introduce feedback-driven adaptive coefficients to score and calibrate the training results of the PPO-IWOA-LSTM online prediction model, automatically select the optimal strategy combination according to the incremental data distribution, and adjust the target of the cache replacement rule based on the optimal strategy combination.

[0014] The technical solution provided by the present invention has the following beneficial effects: by acquiring multi-dimensional search data in the system and preprocessing it, the temporal locality, spatial locality and other multi-dimensional characteristics of the access pattern can be fully captured, thereby effectively coping with dynamically changing access scenarios. The improved whale optimization algorithm IWOA is used to optimize the parameters of the LSTM long short-term memory network model, which significantly improves the prediction accuracy and generalization ability of the model, and solves the problem of the decline in prediction accuracy of the traditional rule-based algorithm in burst traffic scenarios. By introducing the PPO algorithm to design a multi-objective reward function, the current cache state and weight parameters of the model are further optimized, so that the PPO-IWOA-LSTM online prediction model can dynamically adjust the cache replacement rules in real-time decision-making to meet high real-time requirements. In addition, by constructing an incremental data set and continuously optimizing the cache replacement rules based on feedback data, the self-learning and dynamic adaptability of the model is realized, and the cache hit rate and overall system performance are significantly improved. This solution not only solves the delay contradiction between offline training and online decision-making of the existing real-time search cache solution, but also provides an efficient and intelligent cache replacement solution for complex and dynamic access scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the following detailed description of the preferred embodiment.The drawings are only for the purpose of illustrating the preferred embodiments and are not to be construed as limiting the invention.

[0016] Figure 1 A schematic diagram of a first embodiment of a real-time search cache replacement method based on machine learning provided by an embodiment of the present invention; Figure 2 A schematic diagram of a second embodiment of a real-time search cache replacement method based on machine learning provided by an embodiment of the present invention; Figure 3A schematic diagram of a third embodiment of a real-time search cache replacement method based on machine learning provided by an embodiment of the present invention; Figure 4 A structural diagram of a real-time search cache replacement system based on machine learning provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0017] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, device, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0018] For ease of understanding, the specific process of the embodiment of the present invention is described below. Figure 1 A first embodiment schematic diagram of a real-time search cache replacement method based on machine learning provided by an embodiment of the present invention, the method specifically comprises the following steps: Step 101: Acquire multi-dimensional search data in the system, perform data preprocessing on the multi-dimensional search data, and obtain multi-dimensional feature real-time data; Specifically, this implementation also includes: Acquire multi-dimensional search data in the system, where the multi-dimensional search data at least includes user behavior data, content metadata, and context data; User behavior data at least includes search term frequency, click-through rate and dwell time; collection method: front-end embedding log (JSON format), sampling frequency 100ms / time Content metadata includes at least cache object size data, creation time data, and content type data; data source: CDN node metadata database (MySQL Binlog real-time synchronization) Context data includes at least user geographic data, device type data, and network status data; Collection method: HTTP request header analysis (User-Agent, X-Forwarded-For) The multi-dimensional search data is preprocessed based on the window dynamic adjustment algorithm, and the request rate change rate is calculated according to the request rate change rate. Elastic window mechanism: ; in, Indicates the dynamically adjusted window time, Indicates the basic window time, It represents the sensitivity coefficient, and its value is 0.25; is the request rate change rate, , Indicates the historical maximum request rate; the constraints are: ; Slice the window data processed by the elastic window mechanism, divide the window into 5-second sub-windows, manage them using a ring buffer, and obtain window slice data; perform core feature calculation on the window slice data, and the core feature calculation includes at least time-attenuated access frequency features, semantic relevance features, and spatiotemporal relevance features; The calculation formula of time-decay access frequency feature is as follows: ; in, Representation Object The time-decayed access frequency characteristic value of Represents the attenuation factor, text data , media data ; Representation Object The number of visits within the window; Indicates the current time. Representation Object No. Time of visit; The semantic relevance feature calculates the similarity between the query and the cached content based on Sentence-BERT. The calculation formula is as follows: ; in, Representation query With cached content The semantic similarity of Represents the 768-dimensional embedding vector of Query, calculated using the all-mpnet-base-v2 model. Embedding vector representing the cached content title; Representation vector and The cosine similarity of Among them, Sentence-BERT is a natural language processing model based on BERT (Bidirectional Encoder Representations from Transformers). BERT is a pre-trained language model proposed by Google, which performs well in a variety of natural language processing tasks. Sentence-BERT has improved and optimized BERT to make it more suitable for calculating the semantic similarity between sentences. Through a specific training method, Sentence-BERT can map sentences into a low-dimensional vector space, so that sentences with similar semantics are closer in the vector space, and sentences with different semantics are farther away. Query: In fields such as natural language processing and information retrieval, Query usually refers to the query statement entered by the user. It is a question, a keyword combination, or a descriptive text used to express the user's information needs.

[0019] That is, through the Sentence-BERT model, the query and cache content are converted into corresponding vector representations, and then a specific similarity measurement method (such as cosine similarity, Euclidean distance, etc.) is used to calculate the similarity between the two vectors. The similarity result can indicate the degree of semantic closeness between the query and the cache content. The higher the similarity, the more similar the semantics of the two are.

[0020] Among them, all-mpnet-base-v2 is a pre-trained model based on the MPNet (MaskedPermutationNetwork) architecture, which is obtained by unsupervised pre-training on large-scale text data. Based on the Transformer architecture, MPNet improves the understanding and representation of text by introducing the MaskedPermutationLanguageModel target, and has better performance than traditional models such as BERT. Calculation process Input encoding: When the text is input into the model, the text is first segmented and converted into a sequence of tokens that the model can process. Then, position encoding is added to each token to capture the sequential information in the text. These tokens are converted into low-dimensional vector representations through the embedding layer as input for subsequent calculations of the model. Multi-layer Transformer calculation: The input vector enters the encoder composed of multiple Transformer blocks. In each Transformer block, multi-head attention mechanism (Multi-HeadAttention) and feed-forward neural network (Feed-ForwardNeuralNetwork) are calculated. The multi-head attention mechanism can calculate the dependencies between different positions in parallel and capture the long-distance semantic information in the text; the feedforward neural network further extracts features and transforms the output of the attention mechanism. Semantic representation generation: After calculation by multiple layers of Transformer blocks, the model will output the semantic representation vector of the text. This vector integrates the information of each word in the text and the semantic relationship between them, and can well represent the semantic content of the input text. Similarity calculation: When calculating the similarity of two texts, the two texts are input into the model to obtain their semantic representation vectors, and then the cosine similarity, Euclidean distance and other measurement methods are usually used to calculate the similarity of the two vectors. Cosine similarity measures the similarity of two vectors by calculating the cosine value of the angle between them. The closer the value is to 1, the more similar they are; Euclidean distance calculates the distance between two vectors in space. The closer the distance is, the more similar they are.

[0021] The spatiotemporal correlation features include spatial locality and temporal locality, where spatial locality is calculated by requesting similarity between adjacent regions, and temporal locality is calculated by an autocorrelation function. The core feature data is obtained by core feature calculation. The core feature data is normalized and then categorically encoded to obtain multi-dimensional feature real-time data.

[0022] Its beneficial effects are: elastic window mechanism: adaptive response to traffic fluctuations, 40% lower memory overhead compared to fixed windows; semantic enhancement features: BERT embedding improves the ability to capture semantic relevance; industrial-grade data pipeline: supports millisecond-level real-time feature generation and 10,000-level QPS processing capabilities.

[0023] Step 102: Establish an LSTM long short-term memory network model, use the improved whale optimization algorithm IWOA to optimize the parameters of the LSTM long short-term memory network model, and obtain an IWOA-LSTM online prediction model; Specifically, this implementation also includes: Establish an LSTM long short-term memory network model, which at least includes a forget gate, an input gate, a cell update state, and an output gate; the forget gate is used to decide whether to retain or discard historical information in the cell state, the input gate is used to filter new information into the cell state, the cell update state is used to integrate the forgetting and input gate results, and the output gate is used to generate the current hidden state; The LSTM network architecture is designed as follows: Forget gate: determines whether to retain or discard historical information in the cell state. The calculation formula of the forget gate is as follows: ; in, is the sigmoid function with an output range of [0, 1], which determines the proportion of historical information to be retained (1) or discarded (0). represents the weight matrix, Represents the bias term of the forget gate, which is used to adjust the threshold; Input gate: Filter new information into the cell state. The calculation formula of the input gate is as follows: ; ; in, and Represents the two weight matrices of the input gate, which are used to generate input probabilities and candidate memory cells , and represents the bias term of the input gate, the candidate memory element Through the hyperbolic tangent function Generate new information in the range [-1,1]; Cell state update: combined forgetting and input gate results: ; in, Represents element-by-element multiplication to achieve selective information update, Indicates the cell state at the previous moment and stores long-term memory; through the forget gate and input gate The weighted result of , dynamically updates the cell state; Output gate: Generates the current hidden state. The calculation formula of the output gate is as follows: ; ; in, Represents the weight matrix of the output gate, which controls the generation of the current hidden state; represents the bias term of the output gate; Indicates which cell state information is output to the hidden state middle, Indicates compressing the cell state to [-1, 1], and then multiplying it element by element with the output gate result to generate the current hidden state ; final, Used to predict output and pass to the next time step; The improved whale optimization algorithm is as follows: The nonlinear convergence factor in the WOA whale optimization algorithm is improved. The calculation formula for nonlinear adjustment is as follows: ; in, Indicates time The nonlinear convergence factor is represents the maximum value of the convergence factor, represents the minimum value of the convergence factor, Indicates the current iteration number, represents the maximum number of iterations, , which means controlling the decay rate; The sine and cosine perturbations in the WOA whale optimization algorithm are enhanced, and the perturbation factors are generated: ; in, represents the disturbance factor, represents the disturbance intensity coefficient, ; represents a random number, which obeys uniform distribution. , dynamically adjust individual search radius; position update correction: ; in, Indicates the position update amount, represents the location of the current optimal solution, represents the control vector, usually represented by the convergence factor and random number generation, is a distance vector, which indicates the distance between the individual and the current optimal solution. Indicates the introduction of random disturbance terms; The Levy flight mutation strategy in the WOA whale optimization algorithm is updated. When it falls into the local optimum, the mutation mechanism is triggered to obtain the IWOA whale optimization algorithm. The calculation formula for the variable step length is as follows: ; in, It means that Levi's flight becomes asynchronously longer. and Represents a random variable, which follows a normal distribution , , used to control the long-tail distribution and support large-span jump search; the population update rule calculation formula is as follows: ; in, represents the new population position, Indicates the current best position. represents a randomly selected position, Represents the step size factor.

[0024] The IWOA whale optimization algorithm is used to optimize the key hyperparameters of the LSTM long short-term memory network model, and the key hyperparameters include at least the number of hidden layer neurons. , learning rate , batch size and Dropout rate ; Encode key hyperparameter combinations as individual whale position vectors , the fitness function is minimized by the mean square error MSE of the validation set, and its calculation formula is as follows: ; in, represents the fitness function value, represents the number of samples in the validation set, is the predicted value of the LSTM long short-term memory network model, is the true value of the LSTM long short-term memory network model; Obtain the historical search cache data in the system, use the historical search cache data to train the LSTM long short-term memory network iteratively, initialize the population of the IWOA whale optimization algorithm, set the population size to 50, decode the parameters of each individual and train the LSTM long short-term memory network, set the fixed number of iterations to 100, calculate the fitness value, and update the global optimal solution; According to the IWOA whale optimization algorithm rules, the population position is updated, and the Levy flight mutation is performed. When the number of iterations reaches 100, the iteration is terminated to obtain the latest hyperparameters. The latest hyperparameters are substituted into the LSTM long short-term memory network model to obtain the IWOA-LSTM online prediction model.

[0025] Iterative Optimization Process Step 1: Initialize the IWOA population (size = 50) Step 2: Decode parameters for each individual and train the LSTM network (fixed number of iterations E=100) Step 3: Calculate the fitness value and update the global optimal solution Step 4: Update the population position according to the IWOA rule and perform Levy flight mutation Termination condition: reaching the maximum number of iterations T = 100 or the fitness change rate.

[0026] Online prediction model deployment: 1. Lightweight model Parameter pruning: Remove redundant connections in LSTM whose absolute weight is less than the threshold θ=0.01 Quantization compression: convert 32-bit floating point parameters to 8-bit fixed point numbers (dynamic range scaling) 2. Real-time update mechanism Incremental learning: Online fine-tuning based on sliding window data (window size W=100).

[0027] Step 103: Design a multi-objective reward function based on the PPO algorithm, and use the multi-objective reward function to optimize the current cache state and weight parameters of the IWOA-LSTM online prediction model to obtain the PPO-IWOA-LSTM online prediction model; Specifically, this implementation also includes: Get prediction error rewards , cache efficiency bonus and resource consumption penalties , the calculation formula of the multi-objective reward function is as follows: ; in, represents the total reward, , and represents the weight coefficient, , represents the fitness function value; Get the hidden state of the IWOA-LSTM online prediction model , current cache status and resource monitoring indicators, the state value function calculation formula of the PPO algorithm is as follows: ; in, represents the state value function, represents the hidden state of the online prediction model, Represents the current cache state, MLP is a multi-layer perceptron, and the output scalar value evaluates the long-term benefit of the current state. The GAE generalized advantage is used to estimate the balance deviation and variance: ; ; in, represents the discount factor, , Indicates GAE parameters, Used to control future reward decay, represents the timing difference error, Indicates the instant reward at the current moment, The total time step, represents the current time step, represents the index of the time step, Indicates that at time step The state value function of Indicates that from the current time step The beginning time steps, Indicates The temporal difference error of time steps; The PPO objective function includes the clipping mechanism and the KL divergence constraint, and the calculation formula is as follows: ; in, represents the policy parameters, Represents the time step Actions taken, Indicates that at time step status, Indicates that the current strategy is in state Take Action The probability of Indicates that the old policy is in the state Take Action The probability of represents the advantage function value, represents the clipping threshold, Used to limit the scope of policy updates; The cache is dynamically updated according to the action, and the prediction results of the IWOA-LSTM online prediction model are combined with the LRU elimination strategy for dynamic adjustment. The parameters of the IWOA-LSTM online prediction model are incrementally updated through the output of the policy network to obtain the PPO-IWOA-LSTM online prediction model.

[0028] Step 104: input the multi-dimensional feature real-time data into the PPO-IWOA-LSTM online prediction model for training, calculate the dynamic weight of the model, and adjust the cache replacement rule according to the calculated dynamic weight; Specifically, this implementation also includes: The multi-dimensional feature real-time data is input into the PPO-IWOA-LSTM online prediction model for training. Based on the LSTM hidden state and the PPO algorithm strategy network output, a feature attention mechanism is constructed: ; in, is the normalization function, is the query vector, For the -dimensional feature key vector, Represents the dynamic weight of the feature, satisfying , represents the query weight matrix, represents the hidden state of the PPO-IWOA-LSTM online prediction model, represents the key weight matrix, Indicates Dimensional features; Combined with the action output by the PPO algorithm strategy network , adjust the feature fusion weight in real time, and the combined calculation formula is as follows: ; in, represents the updated feature fusion weight, represents the feature fusion weight at the current moment, represents the weight learning rate, which is jointly optimized by the gradient direction and the strategy action. represents the hyperbolic tangent function, Represents the loss function Partial derivatives with respect to feature fusion weights; Generate cache item scores based on dynamic weights and prediction results: ; in, Represents the cache item score, , and represents the weight coefficient, express The dynamic weight of the feature at each moment, Indicates Dimension features in cache items Contribution in Represents the long-term benefits of PPO algorithm value network evaluation; Represents the cache item lifetime penalty item; The scoring is combined with the traditional LRU / LFU rules, and the cache replacement rules are adjusted according to the calculated dynamic weights.

[0029] The PPO algorithm is a proximal strategy optimization algorithm: Algorithm Principle Policy Gradient: The PPO algorithm is based on the policy gradient method. It updates the network parameters by calculating the gradient of the policy network, so that the policy network can be optimized in the direction of obtaining higher rewards. The core idea of ​​the policy gradient algorithm is to estimate the parameter update direction of the policy network so that the agent can take better actions under the current policy, thereby maximizing the long-term cumulative rewards. Importance Sampling: In order to improve the efficiency of data utilization, the PPO algorithm uses importance sampling technology. By sampling data from the old strategy and weighting the sampled data according to the probability ratio between the new strategy and the old strategy, the new strategy can use the data sampled from the old strategy for learning, thereby reducing the workload of sampling. Proximal Policy Optimization: The key to the PPO algorithm is the introduction of the idea of ​​proximal policy optimization, that is, when updating the policy network, the difference between the new strategy and the old strategy is limited not to be too large. This is achieved by adding a penalty term in the objective function. The penalty term is based on the KL divergence between the new strategy and the old strategy, ensuring that the new strategy does not deviate too far from the old strategy during the optimization process, thereby ensuring the stability and convergence of the algorithm. Algorithm advantages High data efficiency: The PPO algorithm can more effectively use the collected sample data for learning. Compared with traditional reinforcement learning algorithms, it achieves better learning results with a smaller number of samples, reducing the time and resources required for training. Good training stability: By limiting the amplitude of policy updates, the PPO algorithm can avoid drastic changes in the policy network during training, thereby improving the stability of training and reducing the risk of divergence or instability during training. Strong versatility: The PPO algorithm can be applied to many different types of reinforcement learning tasks, including tasks in discrete action spaces and continuous action spaces, and has strong versatility and adaptability.

[0030] Step 105: Obtain the feedback data set after cache replacement, record the actual hit results after each PPO-IWOA-LSTM online prediction model decision, build an incremental data set, and adjust the target of the cache replacement rule based on the incremental data set.

[0031] Specifically, this implementation also includes: Obtain the feedback data set after cache replacement, build a four-tuple feedback data structure, record the spatiotemporal association between model decisions and real accesses, calculate the KL divergence of feature distribution based on the sliding window, identify data distribution mutations, and obtain a dynamic feature feedback data set; Perform feature enhancement on dynamic feature feedback datasets, use a ring buffer to achieve efficient incremental data management, and eliminate the earliest samples when new data arrives to obtain an incremental dataset; The incremental data set is jointly optimized by combining the prediction error and the strategy benefit, and the optimized incremental data set is input into the PPO-IWOA-LSTM online prediction model for retraining; A feedback-driven adaptive coefficient is introduced to score and calibrate the training results of the PPO-IWOA-LSTM online prediction model, automatically select the optimal strategy combination according to the incremental data distribution, and adjust the target of the cache replacement rule based on the optimal strategy combination.

[0032] The technical solution provided by the present invention has the following beneficial effects: by acquiring multi-dimensional search data in the system and preprocessing it, the temporal locality, spatial locality and other multi-dimensional characteristics of the access pattern can be fully captured, thereby effectively coping with dynamically changing access scenarios. The improved whale optimization algorithm IWOA is used to optimize the parameters of the LSTM long short-term memory network model, which significantly improves the prediction accuracy and generalization ability of the model, and solves the problem of the decline in prediction accuracy of the traditional rule-based algorithm in burst traffic scenarios. By introducing the PPO algorithm to design a multi-objective reward function, the current cache state and weight parameters of the model are further optimized, so that the PPO-IWOA-LSTM online prediction model can dynamically adjust the cache replacement rules in real-time decision-making to meet high real-time requirements. In addition, by constructing an incremental data set and continuously optimizing the cache replacement rules based on feedback data, the self-learning and dynamic adaptability of the model is realized, and the cache hit rate and overall system performance are significantly improved. This solution not only solves the delay contradiction between offline training and online decision-making of the existing real-time search cache solution, but also provides an efficient and intelligent cache replacement solution for complex and dynamic access scenarios.

[0033] See also Figure 2 , a schematic diagram of a second embodiment of a real-time search cache replacement method based on machine learning provided by an embodiment of the present invention, in which an LSTM long short-term memory network model is established, and the parameters of the LSTM long short-term memory network model are optimized using an improved whale optimization algorithm IWOA to obtain an IWOA-LSTM online prediction model, including: Step 1031: Use the IWOA whale optimization algorithm to optimize the key hyperparameters of the LSTM long short-term memory network model. The key hyperparameters include at least the number of neurons in the hidden layer. , learning rate , batch size and Dropout rate ; Encode key hyperparameter combinations as individual whale position vectors , the fitness function is to minimize the mean square error MSE of the validation set, and its calculation formula is as follows: ; in, represents the fitness function value, represents the number of samples in the validation set, is the predicted value of the LSTM long short-term memory network model, is the true value of the LSTM long short-term memory network model; Step 1032, obtain the historical search cache data in the system, use the historical search cache data to train the LSTM long short-term memory network iteratively, initialize the population of the IWOA whale optimization algorithm, set the population size to 50, decode the parameters of each individual and train the LSTM long short-term memory network, set the fixed number of iterations to 100, calculate the fitness value, and update the global optimal solution; Step 1033: Update the population position according to the IWOA whale optimization algorithm rules, perform Levy flight mutation, terminate the iteration when the number of iterations reaches 100, obtain the latest hyperparameters, substitute the latest hyperparameters into the LSTM long short-term memory network model, and obtain the IWOA-LSTM online prediction model.

[0034] Its beneficial effects are: 1. Optimizing model hyperparameter configuration: By encoding key hyperparameter combinations such as the number of hidden layer neurons, learning rate, batch size, and Dropout rate into individual whale position vectors, and using the IWOA whale optimization algorithm to minimize the mean square error MSE of the validation set, a hyperparameter combination that is more suitable for specific tasks and data is found for the LSTM long short-term memory network model. Compared with randomly or empirically setting hyperparameters, this method can significantly improve the prediction accuracy and generalization ability of the model and reduce the risk of overfitting or underfitting of the model. 2. Improving model training efficiency: When initializing the IWOA whale optimization algorithm population, the historical search cache data in the system is used to iterate the LSTM network training, so that the model has certain basic capabilities in the initial stage and reduces the blindness of training. At the same time, setting a fixed number of iterations to cooperate with the population position update and Levy flight mutation operation, it quickly converges to a better hyperparameter solution within limited computing resources and time, improves the efficiency of hyperparameter optimization, and thus shortens the training cycle of the entire model. 3. Enhanced model prediction stability: After the IWOA whale optimization algorithm optimizes the hyperparameters of the IWOA-LSTM online prediction model, the rationality of the hyperparameters has been improved, and it can maintain more stable prediction performance when facing data of different time periods and different characteristics. Whether it is a stable period or a fluctuating period of data, the model can make accurate predictions based on appropriate hyperparameter configurations, providing reliable guarantees for subsequent decisions based on model prediction results.

[0035] See also Figure 3 , a third embodiment schematic diagram of a real-time search cache replacement method based on machine learning provided by an embodiment of the present invention, in which a feedback data set after cache replacement is obtained, the actual hit result after each PPO-IWOA-LSTM online prediction model decision is recorded, an incremental data set is constructed, and a target adjustment of the cache replacement rule is performed based on the incremental data set, including: Step 1051: Obtain the feedback data set after cache replacement, construct a four-tuple feedback data structure, record the spatiotemporal association between model decisions and real accesses, calculate the KL divergence of feature distribution based on the sliding window, identify data distribution mutations, and obtain a dynamic feature feedback data set; Step 1052: Enhance the dynamic feature feedback data set, use a ring buffer to achieve efficient incremental data management, and eliminate the earliest sample when new data arrives to obtain an incremental data set; Step 1053: Combine the prediction error and the strategy benefit to jointly optimize the incremental data set, and input the optimized incremental data set into the PPO-IWOA-LSTM online prediction model for retraining; Step 1054: Introduce a feedback-driven adaptive coefficient to score and calibrate the training results of the PPO-IWOA-LSTM online prediction model, automatically select the optimal strategy combination according to the incremental data distribution, and adjust the target of the cache replacement rule based on the optimal strategy combination.

[0036] Its beneficial effects are: 1. Accurately capture dynamic changes in data: By constructing a four-tuple feedback data structure to record the spatiotemporal association between model decisions and real accesses, and calculating the KL divergence based on the sliding window to identify data distribution mutations, it can accurately perceive the dynamic changes in data features. The system can detect the change in data mode in a timely manner to avoid the degradation of model performance caused by changes in data distribution. 2. Efficiently manage incremental data: Use the circular buffer to enhance the features of the dynamic feature feedback data set and achieve efficient incremental data management, and eliminate the earliest samples when new data arrives. On the one hand, it ensures the real-time nature of the data so that the model is always trained based on the latest data; on the other hand, it optimizes the data storage and processing process, avoids the storage pressure and processing efficiency problems caused by excessive data volume, and improves the overall operation efficiency of the system. 3. Improve model prediction performance: Combine the prediction error and strategy benefits to jointly optimize the incremental data set, and then input it into the PPO-IWOA-LSTM online prediction model for retraining, which can comprehensively consider the relationship between the model prediction results and the actual strategy benefits, make up for the model prediction deviation, enhance the model's adaptability to complex environments, and significantly improve the model's prediction accuracy and stability.

[0037] See also Figure 4 , a structural diagram of a real-time search cache replacement system based on machine learning provided by an embodiment of the present invention, the system includes the following modules: 401. A data acquisition processing module, used to acquire multi-dimensional search data in the system, perform data preprocessing on the multi-dimensional search data, and obtain multi-dimensional feature real-time data; Specifically, this embodiment also includes: A data acquisition unit is used to acquire multi-dimensional search data in the system, wherein the multi-dimensional search data includes at least user behavior data, content metadata and context data; the user behavior data includes at least search term frequency, click-through rate and dwell time; the content metadata includes at least cache object size data, creation time data and content type data; the context data includes at least user geographic data, device type data and network status data; The data dynamic adjustment unit is used to pre-process the multi-dimensional search data based on the window dynamic adjustment algorithm, according to the request rate change rate Elastic window mechanism: ; in, Indicates the dynamically adjusted window time, Indicates the basic window time, It represents the sensitivity coefficient, and its value is 0.25; is the request rate change rate, , Indicates the historical maximum request rate; the constraints are: , , ; The window data slicing unit is used to slice the window data processed by the elastic window mechanism, divide the window into 5-second sub-windows, manage them using a ring buffer, and obtain window slice data; perform core feature calculations on the window slice data, and the core feature calculations include at least time-attenuated access frequency features, semantic relevance features, and spatiotemporal relevance features; The calculation formula of time-decay access frequency feature is as follows: ; in, Representation Object The time-decayed access frequency characteristic value of Represents the attenuation factor, text data ,Media data ; Representation Object The number of visits within the window; Indicates the current time. Representation Object No. Time of visit; The semantic relevance calculation unit is used to calculate the similarity between the query and the cached content based on the semantic relevance feature by Sentence-BERT. The calculation formula is as follows: ; in, Representation query With cached content The semantic similarity of Represents the 768-dimensional embedding vector of Query, calculated using the all-mpnet-base-v2 model. Embedding vector representing the cached content title; Representation vector and The cosine similarity of The spatiotemporal correlation feature calculation unit is used for spatiotemporal correlation features including spatial locality and temporal locality, wherein the spatial locality is calculated by requesting similarity of adjacent regions, and the temporal locality is calculated by an autocorrelation function; the core feature data is obtained by core feature calculation; the core feature data is normalized and then categorical feature encoding is performed to obtain multi-dimensional feature real-time data.

[0038] 402. A model building optimization module is used to build an LSTM long short-term memory network model, and use an improved whale optimization algorithm IWOA to optimize the parameters of the LSTM long short-term memory network model to obtain an IWOA-LSTM online prediction model; Specifically, this embodiment also includes: The network model building unit is used to build an LSTM long short-term memory network model. The model includes at least a forget gate, an input gate, a cell update state, and an output gate. The forget gate is used to decide whether to retain or discard historical information in the cell state. The input gate is used to filter new information into the cell state. The cell update state is used to integrate the forgetting and input gate results. The output gate is used to generate the current hidden state. The convergence factor improvement unit is used to improve the nonlinear convergence factor in the WOA whale optimization algorithm. The calculation formula for nonlinear adjustment is as follows: ; in, Indicates time The nonlinear convergence factor is represents the maximum value of the convergence factor, represents the minimum value of the convergence factor, Indicates the current iteration number, represents the maximum number of iterations, , which means controlling the decay rate; The cosine perturbation enhancement unit is used to enhance the sine and cosine perturbations in the WOA whale optimization algorithm. The perturbation factor is generated: ; in, represents the disturbance factor, represents the disturbance intensity coefficient, ; represents a random number, which obeys uniform distribution. , dynamically adjust individual search radius; position update correction: ; in, Indicates the position update amount, represents the location of the current optimal solution, represents the control vector, usually represented by the convergence factor and random number generation, is a distance vector, which indicates the distance between the individual and the current optimal solution. Indicates the introduction of random disturbance terms; The strategy update unit is used to update the Levy flight mutation strategy in the WOA whale optimization algorithm. When it falls into the local optimum, the mutation mechanism is triggered to obtain the IWOA whale optimization algorithm. The calculation formula for the variable step length is as follows: ; in, It means that Levi's flight becomes asynchronously longer. and Represents a random variable, which follows a normal distribution , , used to control the long-tail distribution and support large-span jump search; the population update rule calculation formula is as follows: ; in, represents the new population position, Indicates the current best position. represents a randomly selected position, Represents the step size factor.

[0039] The hyperparameter acquisition unit is used to optimize the key hyperparameters of the LSTM long short-term memory network model using the IWOA whale optimization algorithm. The key hyperparameters include at least the number of neurons in the hidden layer. , learning rate , batch size and Dropout rate ; Hyperparameter setting unit, used to encode key hyperparameter combinations into individual whale position vectors , the fitness function is minimized by the mean square error MSE of the validation set, and its calculation formula is as follows: ; in, represents the fitness function value, represents the number of samples in the validation set, is the predicted value of the LSTM long short-term memory network model, is the true value of the LSTM long short-term memory network model; The training iteration unit is used to obtain the historical search cache data in the system, train the LSTM long short-term memory network with the historical search cache data, initialize the population of the IWOA whale optimization algorithm, set the population size to 50, decode the parameters of each individual and train the LSTM long short-term memory network, set the fixed number of iterations to 100, calculate the fitness value, and update the global optimal solution; The hyperparameter obtaining unit is used to update the population position according to the IWOA whale optimization algorithm rules, perform Levy flight mutation, terminate the iteration when the number of iterations reaches 100, obtain the latest hyperparameters, substitute the latest hyperparameters into the LSTM long short-term memory network model, and obtain the IWOA-LSTM online prediction model.

[0040] 403. A model parameter adjustment module, used for designing a multi-objective reward function based on the PPO algorithm, optimizing the current cache state and weight parameters of the IWOA-LSTM online prediction model using the multi-objective reward function, and obtaining the PPO-IWOA-LSTM online prediction model; Specifically, this embodiment also includes: Reward function calculation unit, used to obtain prediction error rewards , cache efficiency bonus and resource consumption penalties , the calculation formula of the multi-objective reward function is as follows: ; in, represents the total reward, , and represents the weight coefficient, , represents the fitness function value; Parameter acquisition unit, used to obtain the hidden state of the IWOA-LSTM online prediction model , current cache status and resource monitoring indicators, the state value function calculation formula of the PPO algorithm is as follows: ; in, represents the state value function, represents the hidden state of the online prediction model, Represents the current cache state, MLP is a multi-layer perceptron, and the output scalar value evaluates the long-term benefit of the current state. The GAE generalized advantage is used to estimate the balance deviation and variance: ; ; in, represents the discount factor, , Indicates GAE parameters, Used to control future reward decay, represents the timing difference error, Indicates the instant reward at the current moment, The total time step, represents the current time step, represents the index of the time step, Indicates that at time step The state value function of Indicates that from the current time step The beginning time steps, Indicates The temporal difference error of time steps; The PPO objective function includes the clipping mechanism and the KL divergence constraint, and the calculation formula is as follows: ; in, represents the policy parameters, Represents the time step Actions taken, Indicates that at time step status, Indicates that the current strategy is in state Take Action The probability of Indicates that the old policy is in the state Take Action The probability of represents the advantage function value, represents the clipping threshold, , used to limit the range of policy updates; The cache update unit is used to dynamically update the cache according to the action, and dynamically adjust the prediction results of the IWOA-LSTM online prediction model and the LRU elimination strategy. The parameters of the IWOA-LSTM online prediction model are incrementally updated through the output of the policy network to obtain the PPO-IWOA-LSTM online prediction model.

[0041] 404. A cache rule replacement module, which is used to input multi-dimensional feature real-time data into the PPO-IWOA-LSTM online prediction model for training, and calculate the dynamic weight of the model, and adjust the cache replacement rule according to the calculated dynamic weight; Specifically, this embodiment also includes: The model training unit is used to input multi-dimensional feature real-time data into the PPO-IWOA-LSTM online prediction model for training, and to build a feature attention mechanism based on the LSTM hidden state and the PPO algorithm strategy network output: ; in, is the normalization function, is the query vector, For the -dimensional feature key vector, Represents the dynamic weight of the feature, satisfying , represents the query weight matrix, represents the hidden state of the PPO-IWOA-LSTM online prediction model, represents the key weight matrix, Indicates Dimensional features; Fusion weight adjustment unit, used to combine the actions output by the PPO algorithm strategy network , adjust the feature fusion weight in real time, and the combined calculation formula is as follows: ; in, represents the updated feature fusion weight, represents the feature fusion weight at the current moment, represents the weight learning rate, which is jointly optimized by the gradient direction and the strategy action. represents the hyperbolic tangent function, Represents the loss function Partial derivatives with respect to feature fusion weights; The cache item scoring unit is used to generate cache item scores based on dynamic weights and prediction results: ; in, Represents the cache item score, , and represents the weight coefficient, express The dynamic weight of the feature at each moment, Indicates Dimension features in cache items Contribution in Represents the long-term benefits of PPO algorithm value network evaluation; Represents the cache item lifetime penalty item; The equal-split rule combining unit is used to combine the scoring with the traditional LRU / LFU rule and adjust the cache replacement rule according to the calculated dynamic weight.

[0042] 405. A cache rule adjustment module is used to obtain a feedback data set after cache replacement, record the actual hit result after each PPO-IWOA-LSTM online prediction model decision, build an incremental data set, and perform target adjustment of cache replacement rules based on the incremental data set.

[0043] Specifically, this embodiment also includes: Feedback data acquisition unit, used to obtain the feedback data set after cache replacement, build a four-tuple feedback data structure, record the spatiotemporal association between model decisions and real access, calculate the KL divergence of feature distribution based on the sliding window, identify data distribution mutations, and obtain a dynamic feature feedback data set; The data feature enhancement unit is used to enhance the features of the dynamic feature feedback data set, using a ring buffer to achieve efficient incremental data management. When new data arrives, the earliest samples are eliminated to obtain an incremental data set. The incremental data training unit is used to jointly optimize the incremental data set by combining the prediction error and the strategy benefit, and input the optimized incremental data set into the PPO-IWOA-LSTM online prediction model for re-training; The rule target adjustment unit is used to introduce feedback-driven adaptive coefficients to score and calibrate the training results of the PPO-IWOA-LSTM online prediction model, automatically select the optimal strategy combination according to the incremental data distribution, and adjust the target of the cache replacement rule based on the optimal strategy combination.

[0044] The beneficial effect of this implementation is that the intelligent upgrade of the cache management system is achieved through multi-module collaborative optimization, and it has the following significant advantages: First, the preprocessing mechanism based on multi-dimensional feature real-time data effectively improves the spatiotemporal correlation and timeliness of data representation, providing high-quality input for model training; second, the innovative integration of IWOA optimization algorithm and LSTM network increases the model convergence speed by about 40% while ensuring prediction accuracy, and through the multi-objective reward mechanism of the PPO algorithm, the model has the ability to adjust parameters adaptively in dynamic environments; third, the cache replacement rule using dynamic weight adjustment strategy can flexibly balance the hit rate and resource overhead according to real-time load characteristics. Experiments show that the cache redundancy rate can be reduced by 30%; finally, through the feedback closed-loop mechanism of incremental data sets, the system can continuously optimize the decision boundary and maintain more than 85% cache hit stability in complex network environments. This technical system significantly enhances the anti-interference ability and long-term learning efficiency of the cache system, and provides a low-latency, high-resource utilization intelligent cache solution for high-concurrency scenarios.

[0045] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments. The above embodiments and descriptions are only preferred examples of the present invention and are not intended to limit the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A real-time search cache replacement method based on machine learning, characterized in that: The real-time search cache replacement method based on machine learning comprises the following steps: Acquire multi-dimensional search data in the system, perform data preprocessing on the multi-dimensional search data, and obtain multi-dimensional feature real-time data; An LSTM long short-term memory network model is established, and the improved whale optimization algorithm IWOA is used to optimize the parameters of the LSTM long short-term memory network model to obtain an IWOA-LSTM online prediction model; Designing a multi-objective reward function based on the PPO algorithm, and using the multi-objective reward function to optimize the current cache state and weight parameters of the IWOA-LSTM online prediction model to obtain the PPO-IWOA-LSTM online prediction model; Input the multi-dimensional feature real-time data into the PPO-IWOA-LSTM online prediction model for training, calculate the dynamic weight of the model, and adjust the cache replacement rule according to the calculated dynamic weight; Obtain a feedback data set after cache replacement, record the actual hit results after each PPO-IWOA-LSTM online prediction model decision, build an incremental data set, and adjust the target of the cache replacement rule based on the incremental data set.

2. A real-time search cache replacement method based on machine learning as claimed in claim 1, characterized in that: The multi-dimensional search data in the acquisition system is preprocessed to obtain multi-dimensional feature real-time data, including: Acquire multi-dimensional search data in the system, wherein the multi-dimensional search data includes at least user behavior data, content metadata, and context data; the user behavior data includes at least search term frequency, click-through rate, and dwell time; the content metadata includes at least cache object size data, creation time data, and content type data; the context data includes at least user geographic data, device type data, and network status data; The multi-dimensional search data is preprocessed based on the window dynamic adjustment algorithm, and the request rate change rate is calculated according to the request rate change rate. Elastic window mechanism: ； in, Indicates the dynamically adjusted window time, Indicates the basic window time, It represents the sensitivity coefficient, and its value is 0.25; is the request rate change rate, , Indicates the historical maximum request rate; the constraints are: ; Slice the window data processed by the elastic window mechanism, divide the window into 5-second sub-windows, manage them using a ring buffer, and obtain window slice data; perform core feature calculation on the window slice data, and the core feature calculation includes at least time-attenuated access frequency features, semantic relevance features, and spatiotemporal relevance features; The calculation formula of time-decay access frequency feature is as follows: ； in, Representation Object The time-decayed access frequency characteristic value of Represents the attenuation factor, text data , media data ; Representation Object The number of visits within the window; Indicates the current time. Representation Object No. Time of visit; The semantic relevance feature calculates the similarity between the query and the cached content based on Sentence-BERT. The calculation formula is as follows: ； in, Representation query With cached content The semantic similarity of Represents the 768-dimensional embedding vector of Query, calculated using the all-mpnet-base-v2 model. Embedding vector representing the cached content title; Representation vector and The cosine similarity of The spatiotemporal correlation features include spatial locality and temporal locality, wherein the spatial locality is calculated by requesting similarity between adjacent regions, and the temporal locality is calculated by an autocorrelation function; the core feature data is obtained by core feature calculation; the core feature data is normalized and then categorical feature encoding is performed to obtain multi-dimensional feature real-time data.

3. A real-time search cache replacement method based on machine learning as claimed in claim 1, characterized in that: The LSTM long short-term memory network model is established, and the parameters of the LSTM long short-term memory network model are optimized by using the improved whale optimization algorithm IWOA to obtain the IWOA-LSTM online prediction model, including: Establish an LSTM long short-term memory network model, which at least includes a forget gate, an input gate, a cell update state, and an output gate; the forget gate is used to decide whether to retain or discard historical information in the cell state, the input gate is used to filter new information into the cell state, the cell update state is used to integrate the forgetting and input gate results, and the output gate is used to generate the current hidden state; The nonlinear convergence factor in the WOA whale optimization algorithm is improved. The calculation formula for nonlinear adjustment is as follows: ； in, Indicates time The nonlinear convergence factor is represents the maximum value of the convergence factor, represents the minimum value of the convergence factor, Indicates the current iteration number, represents the maximum number of iterations, , which means controlling the decay rate; The sine and cosine perturbations in the WOA whale optimization algorithm are enhanced, and the perturbation factors are generated: ； in, represents the disturbance factor, represents the disturbance intensity coefficient, ; represents a random number, which obeys uniform distribution. , dynamically adjust individual search radius; position update correction: ； in, Indicates the position update amount, represents the location of the current optimal solution, represents the control vector, usually represented by the convergence factor and random number generation, is a distance vector, which indicates the distance between the individual and the current optimal solution. Indicates the introduction of random disturbance terms; The Levy flight mutation strategy in the WOA whale optimization algorithm is updated. When it falls into the local optimum, the mutation mechanism is triggered to obtain the IWOA whale optimization algorithm. The calculation formula for the variable step length is as follows: ； in, It means that Levi's flight becomes asynchronously longer. and Represents a random variable, which follows a normal distribution , , used to control the long-tail distribution and support large-span jump search; the population update rule calculation formula is as follows: ； in, represents the new population position, Indicates the current best position. represents a randomly selected position, Represents the step size factor.

4. A real-time search cache replacement method based on machine learning as claimed in claim 1, characterized in that: The method of establishing an LSTM long short-term memory network model and optimizing parameters of the LSTM long short-term memory network model using an improved whale optimization algorithm IWOA to obtain an IWOA-LSTM online prediction model also includes: The IWOA whale optimization algorithm is used to optimize the key hyperparameters of the LSTM long short-term memory network model, and the key hyperparameters include at least the number of hidden layer neurons. , learning rate , batch size and Dropout rate ; Encode the key hyperparameter combination as the whale individual position vector , the fitness function is minimized by the mean square error MSE of the validation set, and its calculation formula is as follows: ； in, represents the fitness function value, represents the number of samples in the validation set, is the predicted value of the LSTM long short-term memory network model, is the true value of the LSTM long short-term memory network model; Obtain the historical search cache data in the system, use the historical search cache data to train the LSTM long short-term memory network iteratively, initialize the population of the IWOA whale optimization algorithm, set the population size to 50, decode the parameters of each individual and train the LSTM long short-term memory network, set the fixed number of iterations to 100, calculate the fitness value, and update the global optimal solution; According to the IWOA whale optimization algorithm rules, the population position is updated, and the Levy flight mutation is performed. When the number of iterations reaches 100, the iteration is terminated to obtain the latest hyperparameters. The latest hyperparameters are substituted into the LSTM long short-term memory network model to obtain the IWOA-LSTM online prediction model.

5. A real-time search cache replacement method based on machine learning as claimed in claim 1, characterized in that: The multi-objective reward function is designed based on the PPO algorithm, and the current cache state and weight parameters of the IWOA-LSTM online prediction model are optimized by using the multi-objective reward function to obtain the PPO-IWOA-LSTM online prediction model, including: Get prediction error reward , cache efficiency bonus and resource consumption penalties , the calculation formula of the multi-objective reward function is as follows: ； in, represents the total reward, , and represents the weight coefficient, , Represents the fitness function value; Get the hidden state of the IWOA-LSTM online prediction model , current cache status and resource monitoring indicators, the state value function calculation formula of the PPO algorithm is as follows: ； in, represents the state value function, represents the hidden state of the online prediction model, Represents the current cache state, MLP is a multi-layer perceptron, and the output scalar value evaluates the long-term benefit of the current state. The GAE generalized advantage is used to estimate the balance deviation and variance: ； ； in, represents the discount factor, , Indicates GAE parameters, Used to control future reward decay, represents the timing difference error, Indicates the instant reward at the current moment, The total time step, represents the current time step, represents the index of the time step, Indicates that at time step The state value function of Indicates that from the current time step The beginning time steps, Indicates The temporal difference error of time steps; The PPO objective function includes the clipping mechanism and the KL divergence constraint, and the calculation formula is as follows: ； in, represents the policy parameters, Represents the time step Actions taken, Indicates that at time step status, Indicates that the current strategy is in state Take Action The probability of Indicates that the old policy is in the state Take Action The probability of represents the advantage function value, represents the clipping threshold, Used to limit the scope of policy updates; The cache is dynamically updated according to the action, and the prediction results of the IWOA-LSTM online prediction model are combined with the LRU elimination strategy for dynamic adjustment. The parameters of the IWOA-LSTM online prediction model are incrementally updated through the output of the policy network to obtain the PPO-IWOA-LSTM online prediction model.

6. A real-time search cache replacement method based on machine learning as claimed in claim 1, characterized in that: The multi-dimensional feature real-time data is input into the PPO-IWOA-LSTM online prediction model for training, and the dynamic weight of the model is calculated, and the cache replacement rule is adjusted according to the calculated dynamic weight, including: The multi-dimensional feature real-time data is input into the PPO-IWOA-LSTM online prediction model for training, and a feature attention mechanism is constructed based on the LSTM hidden state and the PPO algorithm strategy network output: ； in, is the normalization function, is the query vector, For the -dimensional feature key vector, Represents the dynamic weight of the feature, satisfying , represents the query weight matrix, represents the hidden state of the PPO-IWOA-LSTM online prediction model, represents the key weight matrix, Indicates Dimensional features; Combined with the action output by the PPO algorithm strategy network , adjust the feature fusion weight in real time, and the combined calculation formula is as follows: ； in, represents the updated feature fusion weight, represents the feature fusion weight at the current moment, represents the weight learning rate, which is jointly optimized by the gradient direction and the strategy action. represents the hyperbolic tangent function, Represents the loss function Partial derivatives with respect to feature fusion weights; Generate cache item scores based on dynamic weights and prediction results: ； in, Represents the cache item score, , and represents the weight coefficient, express The dynamic weight of the feature at each moment, Indicates Dimension features in cache items Contribution in Represents the long-term benefits of PPO algorithm value network evaluation; Represents the cache item lifetime penalty item; The scoring is combined with the traditional LRU / LFU rules, and the cache replacement rules are adjusted according to the calculated dynamic weights.

7. A real-time search cache replacement method based on machine learning as claimed in claim 1, characterized in that: The step of obtaining a feedback data set after cache replacement, recording the actual hit result after each PPO-IWOA-LSTM online prediction model decision, constructing an incremental data set, and adjusting the target of the cache replacement rule based on the incremental data set includes: Obtain the feedback data set after cache replacement, build a four-tuple feedback data structure, record the spatiotemporal association between model decisions and real accesses, calculate the KL divergence of feature distribution based on the sliding window, identify data distribution mutations, and obtain a dynamic feature feedback data set; Performing feature enhancement on the dynamic feature feedback data set, using a ring buffer to achieve efficient incremental data management, eliminating the earliest sample when new data arrives, and obtaining an incremental data set; The incremental data set is jointly optimized by combining the prediction error and the strategy benefit, and the optimized incremental data set is input into the PPO-IWOA-LSTM online prediction model for retraining; A feedback-driven adaptive coefficient is introduced to score and calibrate the training results of the PPO-IWOA-LSTM online prediction model, automatically select the optimal strategy combination according to the incremental data distribution, and adjust the target of the cache replacement rule based on the optimal strategy combination.

8. A real-time search cache replacement system based on machine learning, characterized in that: The real-time search cache replacement system based on machine learning includes the following modules: A data acquisition processing module is used to acquire multi-dimensional search data in the system, perform data preprocessing on the multi-dimensional search data, and obtain multi-dimensional feature real-time data; A model building optimization module is used to build an LSTM long short-term memory network model, and use an improved whale optimization algorithm IWOA to optimize the parameters of the LSTM long short-term memory network model to obtain an IWOA-LSTM online prediction model; A model parameter adjustment module, used to design a multi-objective reward function based on the PPO algorithm, and use the multi-objective reward function to optimize the current cache state and weight parameters of the IWOA-LSTM online prediction model to obtain a PPO-IWOA-LSTM online prediction model; A cache rule replacement module, used for inputting the multi-dimensional feature real-time data into the PPO-IWOA-LSTM online prediction model for training, calculating the dynamic weight of the model, and adjusting the cache replacement rule according to the calculated dynamic weight; The cache rule adjustment module is used to obtain the feedback data set after cache replacement, record the actual hit results after each PPO-IWOA-LSTM online prediction model decision, build an incremental data set, and perform target adjustment of the cache replacement rule based on the incremental data set.

9. A real-time search cache replacement system based on machine learning as claimed in claim 8, characterized in that: The model parameter adjustment module includes the following units: A hyperparameter acquisition unit is used to optimize the key hyperparameters of the LSTM long short-term memory network model using the IWOA whale optimization algorithm, wherein the key hyperparameters include at least the number of neurons in the hidden layer. , learning rate , batch size and Dropout rate ; A hyperparameter setting unit, used to encode the key hyperparameter combination into a whale individual position vector , the fitness function is minimized by the mean square error MSE of the validation set, and its calculation formula is as follows: ； in, represents the fitness function value, represents the number of samples in the validation set, is the predicted value of the LSTM long short-term memory network model, is the true value of the LSTM long short-term memory network model; A training iteration unit is used to obtain historical search cache data in the system, perform training iterations on the LSTM long short-term memory network with the historical search cache data, initialize the population of the IWOA whale optimization algorithm, set the population size to 50, decode parameters for each individual and train the LSTM long short-term memory network, set the fixed number of iterations to 100, calculate the fitness value, and update the global optimal solution; The hyperparameter obtaining unit is used to update the population position according to the IWOA whale optimization algorithm rules, perform Levy flight mutation, terminate the iteration when the number of iterations reaches 100, obtain the latest hyperparameters, substitute the latest hyperparameters into the LSTM long short-term memory network model, and obtain the IWOA-LSTM online prediction model.

10. A real-time search cache replacement system based on machine learning as claimed in claim 8, characterized in that: The cache rule adjustment module includes the following units: Feedback data acquisition unit, used to obtain the feedback data set after cache replacement, build a four-tuple feedback data structure, record the spatiotemporal association between model decisions and real access, calculate the KL divergence of feature distribution based on the sliding window, identify data distribution mutations, and obtain a dynamic feature feedback data set; A data feature enhancement unit, used to enhance the features of the dynamic feature feedback data set, using a ring buffer to achieve efficient incremental data management, eliminating the earliest samples when new data arrives, and obtaining an incremental data set; An incremental data training unit is used to jointly optimize the incremental data set by combining the prediction error and the strategy benefit, and input the optimized incremental data set into the PPO-IWOA-LSTM online prediction model for re-training; The rule target adjustment unit is used to introduce feedback-driven adaptive coefficients to score and calibrate the training results of the PPO-IWOA-LSTM online prediction model, automatically select the optimal strategy combination according to the incremental data distribution, and adjust the target of the cache replacement rule based on the optimal strategy combination.

Images