A solid state drive data pre-fetching method based on attention mechanism

Abstract

The present application relates to the field of computer storage, and discloses a method for pre-fetching data of a solid-state hard disk based on an attention mechanism, comprising the following steps: S1, collecting access trajectory data of a solid-state hard disk, and performing fixed time window segmentation; S2, generating relevant tags according to the content, and converting these tags into numerical vectors; S3, combining the tag vector with the traditional access mode features to form a multimodal feature vector; S4, processing the multimodal feature vector using a gated recurrent unit network, and evaluating the importance of data in different time windows through an attention layer, and calculating the weight; S5, predicting data blocks that may be frequently accessed in the future based on the attention weight and the output of the gated recurrent unit network; S6, selecting data blocks with high confidence for pre-fetching based on the prediction results. The present invention can more accurately identify and predict which data blocks may be frequently accessed in the future by using a method for pre-fetching data of a solid-state hard disk based on an attention mechanism.

Description

Technical Field

[0001] The present invention relates to the field of computer storage technology, and in particular to a solid state hard disk data pre-fetching method based on an attention mechanism. Background Art

[0002] As a new type of storage device, solid-state drive (SSD) has gradually replaced traditional mechanical hard disk (HDD) and become the mainstream storage medium due to its advantages such as high speed, high reliability and low power consumption. SSD uses flash memory technology for data storage and can achieve faster data reading and writing speeds than HDD. Therefore, it has been widely used in servers, personal computers, data centers and other fields. However, with the continuous growth of data scale and the increase in access frequency, how to further optimize the performance of SSD, especially to improve the response speed of data access, is still a problem to be solved.

[0003] Data prefetching technology is a method of reducing access latency by loading data that may be accessed in advance. It is widely used in cache management and storage system optimization. Traditional data prefetching methods mainly rely on historical access patterns and simple prediction algorithms, such as sequential prefetching and prefetching based on access frequency. These methods improve data access efficiency to a certain extent, but their effects are often not ideal in the face of complex access patterns and dynamically changing data requirements. In addition, traditional methods lack in-depth analysis and utilization of data content, making it difficult to give full play to the performance advantages of SSDs.

[0004] In recent years, deep learning technology, especially attention mechanism, has made significant progress in natural language processing, computer vision and other fields. Attention mechanism can significantly improve the prediction accuracy and processing efficiency of the model by giving the model the ability to selectively focus on different parts of the input data. Applying attention mechanism to the field of SSD data prefetching can make prefetching decisions more intelligent and accurate, thereby further improving the performance of the storage system. Summary of the invention

[0005] In view of the shortcomings of the prior art, the present invention provides a solid-state hard disk data prefetching method based on an attention mechanism, which intelligently prefetches data by combining content analysis and access pattern prediction to optimize the data access efficiency and response speed of the SSD.

[0006] To achieve the above objectives, the present invention is implemented through the following technical solutions: a solid state drive data pre-fetching method based on an attention mechanism, comprising the following steps:

[0007] Collect access trajectory data of SSDs and divide the data into fixed time windows;

[0008] Perform content analysis on each data block, generate relevant labels based on the content, and convert these labels into numerical vectors;

[0009] Combining the label vector with traditional access pattern features to form a multimodal feature vector;

[0010] Processing the multimodal feature vector using a gated recurrent unit network, evaluating the importance of data in different time windows through an attention layer, and calculating attention weights;

[0011] Based on the attention weights and the gated recurrent unit network output, predicting data blocks that may be frequently accessed in the future;

[0012] According to the prediction results, data blocks with high confidence are selected for prefetching.

[0013] Preferably, the content analysis uses natural language processing technology to extract keywords from text data and uses convolutional neural network to extract image features from image data.

[0014] Preferably, for text data, the content analysis includes the following steps:

[0015] Use pre-trained language models for text embedding to convert entire texts or key paragraphs into semantically rich vector representations;

[0016] Use clustering algorithm to cluster text vectors, and each cluster center represents a topic category;

[0017] The cluster center label to which the text belongs is used as the content label, and the vector representation of the cluster center is used as a numerical vector for subsequent multimodal feature fusion.

[0018] Preferably, for the image data, the content analysis includes the following steps:

[0019] Use image semantic segmentation technology to perform advanced image processing to identify the main objects and background in the image;

[0020] Use deep feature extraction techniques for each object identified and merge these features to form a comprehensive visual feature representation of the image;

[0021] The autoencoder is used to reduce the dimension of the merged features and generate compressed numerical vector representations, which reduce the processing complexity while retaining key visual information.

[0022] Preferably, the gated recurrent unit network has at least one hidden layer, each hidden layer includes multiple GRU units, and each GRU unit updates its hidden state according to the hidden state at the previous moment and the input at the current moment, and the update formula is:

[0023]

[0024] Among them, h t is the hidden state at the current moment, z t It is the update gate. is a candidate hidden state, calculated as follows:

[0025] z t =σ(W z ·[h t-1 ,x t ]+b z )

[0026]

[0027] r t =σ(W r ·[h t-1 ,x t ]+b r )

[0028] Among them, W z ,W h ,W r is the weight matrix, b z ,b h ,b r is the bias term, r t is the reset gate and σ is the sigmoid activation function.

[0029] Preferably, the attention uses a weighted mechanism, and the output of the GRU network is weighted and summed by the calculated attention weight to form a weighted hidden state representation, and the calculation formula is:

[0030]

[0031] Among them, W a and b a are the weights and biases of the attention layer, ν is the learned context vector, and c is the final context vector, which represents the weighted representation of the entire sequence.

[0032] Preferably, the step of predicting data blocks that may be frequently accessed in the future based on the attention weights and the output of the gated recurrent unit network includes using a fully connected layer to process the outputs of the gated recurrent unit network and the attention layer to generate a final predicted data block.

[0033] Preferably, the confidence of the prediction result is determined by setting a threshold, and the pre-fetch operation is performed only when the predicted access probability is higher than the threshold.

[0034] Preferably, pre-fetching the data blocks includes loading pre-selected data blocks into a cache area of ​​the solid state drive.

[0035] The present invention also provides a solid state hard disk data pre-fetching device, comprising:

[0036] A data collection unit, used to collect access trajectory data of the solid state drive according to a fixed time window;

[0037] A content analysis unit, used to perform content analysis on each data block and generate relevant tags;

[0038] A label vectorization unit, used for converting the label into a numerical vector;

[0039] A feature fusion unit, which is used to combine the label vector with the traditional access pattern features to form a multimodal feature vector;

[0040] A neural network processing unit, including a gated recurrent unit network and an attention layer, is used to process multimodal feature vectors and calculate attention weights;

[0041] A prediction unit, which is used to predict data blocks that may be frequently accessed in the future based on the attention weights and the output of the gated recurrent unit network;

[0042] The data pre-fetch unit is used to select data blocks with high confidence for pre-fetching according to the prediction results, and load the selected data blocks into the cache area of ​​the solid state drive.

[0043] The present invention provides a solid state drive data pre-fetching method based on an attention mechanism, which has the following beneficial effects:

[0044] 1. The present invention uses a solid-state drive data prefetching method based on an attention mechanism to more accurately identify and predict which data blocks may be frequently accessed in the future. The introduction of the attention mechanism enables the system to dynamically focus on those important data blocks, thereby improving the efficiency and accuracy of prefetching. This method not only reduces unnecessary data loading, but also optimizes data access speed, thereby improving the performance of the entire system.

[0045] 2. The present invention helps to more reasonably allocate and utilize storage resources, especially in fast storage devices such as solid-state drives, by accurately predicting and pre-fetching data blocks that will be frequently accessed. This optimization can reduce dependence on slow storage devices, reduce overall system energy consumption and operating costs, and improve the response speed of data processing.

[0046] 3. The present invention greatly reduces the latency of data access by preloading the data that may be needed into faster storage media. This not only improves the user experience, especially in data-intensive applications such as big data analysis and online transaction processing, but also improves the system's ability to handle high-concurrency requests. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 It is a schematic diagram of the method flow of the present invention;

[0048] Figure 2 It is a schematic diagram of the structure of the device of the present invention;

[0049] Among them, 100, data collection unit; 200, content analysis unit; 300, label vectorization unit; 400, feature fusion unit; 500, neural network processing unit; 600, prediction unit; 700, data pre-fetching unit. DETAILED DESCRIPTION

[0050] The following will be combined with the drawings in the specification of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0051] Please see attached Figure 1 , an embodiment of the present invention provides a solid state drive data pre-fetching method based on an attention mechanism, comprising the following steps:

[0052] Step S1: Collecting access trace data of the solid state drive

[0053] The present invention firstly involves collecting access trace data of a solid state drive (SSD). The data includes the address, access time, operation type (read / write) and data size of a data block. In order to systematically process the data, the access records are segmented into fixed time windows, such as one window every minute or every five minutes, so that the changes of access patterns over time can be effectively captured.

[0054] In this embodiment, step S1 involves collecting access trace data of a solid state drive (SSD) and segmenting the data according to a predetermined fixed time window. This step is intended to provide basic data support for subsequent data prediction and pre-fetching operations.

[0055] First, you need to collect access trace data from the SSD's operating system or storage management software. This data includes but is not limited to:

[0056] Data block address: indicates the specific storage location to be accessed.

[0057] Access time: records the specific time of each operation.

[0058] Operation type: Distinguish between read operations and write operations.

[0059] Data size: The amount of data involved in each operation.

[0060] Data collection can be achieved through monitoring software, which needs to be able to access the SSD management system to capture and record the above information in real time. The collected data should be stored in the form of logs for further processing and analysis.

[0061] The collected data is divided into fixed time windows to facilitate time series analysis of the data. The length of the time window can be set according to actual needs, such as every minute, every five minutes, or every ten minutes. The key to this step is to determine the access frequency and pattern of the data blocks within each time window.

[0062] To achieve this step, the following options can be used:

[0063] Make sure all data is sorted in chronological order.

[0064] According to the set time window length, continuous data records are grouped. Each group of data contains all access records within the time period.

[0065] Record the statistics of each window: Perform statistical analysis on the data in each time window, including the number of reads and writes, the number of data blocks accessed, the most frequently accessed data blocks, etc.

[0066] By accurately collecting and segmenting SSD access trajectory data, detailed input data can be provided for data prediction and prefetching strategies based on attention mechanisms. In addition, reasonable time window segmentation helps to better understand and analyze data access patterns, thereby improving the accuracy and efficiency of data prefetching.

[0067] Step S2: Content analysis and label generation

[0068] Each data block will go through a content analysis process. For text data, natural language processing techniques such as BERT or GPT can be used to extract themes and keywords of the text. For image data, image recognition algorithms such as CNN can be used to identify the main objects and scenes in the image. The extracted information (keywords, objects, etc.) is converted into labels, and word embedding techniques such as Word2Vec or GloVe are used to convert these labels into numerical vectors, which converts unstructured content information into a structured form that can be processed by the algorithm.

[0069] In this embodiment, step S2 includes in-depth analysis of the data block content in the SSD and generating tags based on the content. Specifically, as follows:

[0070] 1. Text data processing:

[0071] For text files stored in the SSD, the text content of the file is first extracted.

[0072] Use natural language processing techniques, such as TF-IDF (Term Frequency-Inverse Document Frequency) or NLP (Natural Language Processing) models (such as BERT) to analyze the text and extract keywords or phrases. These keywords should represent the main content or theme of the text.

[0073] 2. Image data processing:

[0074] For image files, use image recognition techniques, such as convolutional neural networks (CNNs), to identify the main objects and scenes in the image. You can use pre-trained models, such as ResNet or VGG, to extract image features.

[0075] Generate descriptive labels based on the identified image content (e.g., object category, scene type).

[0076] Convert the extracted text keywords or image labels into numerical vectors. This process usually involves the use of word embedding techniques such as Word2Vec or GloVe, which can convert text keywords into dense vectors of fixed length. For images, the feature vector extracted from the CNN model can be directly used as the label vector. Perform necessary preprocessing on the generated vector, such as normalization, to ensure the numerical stability of the vector in subsequent analysis.

[0077] This step extracts key information from the data content of the SSD and converts it into a machine-readable format, providing the necessary input for data prediction and prefetching based on the attention mechanism. Through precise content analysis and efficient label vectorization, this method can improve the relevance and efficiency of data prefetching, thereby improving the performance and response speed of the SSD.

[0078] Step S3: Formation of multimodal feature vector

[0079] The numerical vector generated in step S2 is combined with traditional access pattern features (such as access frequency, read-write ratio, etc.) to form a multimodal feature vector. This process integrates multiple types of data into a unified representation through data fusion techniques such as feature concatenation or feature fusion network, enhancing the model's ability to handle different data types.

[0080] In this embodiment, step S3 includes the following specific solutions to ensure effective integration of the label vector and the access pattern features:

[0081] 1. Feature selection and preparation

[0082] First, features related to the access pattern are extracted, which may include but are not limited to the access frequency of data blocks, the read-write ratio, the periodic pattern of access time, etc. These features will be calculated from the original access data collected in step S1.

[0083] All access pattern features are standardized to eliminate the impact of different magnitudes and distribution ranges and ensure the stability and effectiveness of model training. The standardization method can be Z-score standardization or Min-Max standardization.

[0084] 2. Feature Fusion

[0085] The content label vector obtained from step S2 is merged with the access pattern feature vector in step S3. The merging can be achieved by simple vector concatenation, that is, arranging the two vectors end-to-end to form a new vector.

[0086] Considering that the merged vector may have a high dimension, dimensionality reduction techniques such as principal component analysis (PCA) or autoencoders can be used to reduce the dimension of the vector while trying to retain the important information in the original data.

[0087] 3. Specific implementation details of data processing

[0088] For each data block, its content label vector is concatenated with the access pattern feature vector. For example, if the content label vector is 100-dimensional and the access pattern feature vector is 50-dimensional, the combined vector will be 150-dimensional.

[0089] The merged vectors are then dimensionality reduced for ease of processing and storage. The specific dimensionality reduction technique chosen depends on the model requirements and computing resources.

[0090] By effectively integrating the content label vector and access pattern features, the multimodal feature vector can more comprehensively represent the characteristics of each data block, thereby improving the accuracy and efficiency of the prediction model. The use of this comprehensive feature vector can significantly improve the quality of the prediction results and provide a more scientific decision-making basis for SSD data prefetching.

[0091] Step S4: Gated Recurrent Unit (GRU) Network and Attention Layer

[0092] The multimodal feature vector is then input into a deep neural network containing a gated recurrent unit (GRU). GRU is an optimized recurrent neural network that is particularly suitable for processing time series data and can effectively capture temporal dependencies and long-term dependencies. On top of the GRU network, an attention layer is deployed to evaluate the importance of data in different time windows. The attention mechanism emphasizes more important features through learned weights, thereby improving the accuracy of predictions and the interpretability of the model. For example, the weights of different time windows can be calculated using additive or multiplicative attention models.

[0093] In this embodiment, step S4 includes the following specific scheme to ensure that time series features are effectively processed and important time windows are extracted.

[0094] 1. GRU network settings

[0095] Design a neural network with multiple GRU layers, where each GRU layer is responsible for processing the dependencies of the time series. The GRU layer is chosen because it can avoid the problem of gradient vanishing when processing long time series data and effectively capture time dynamics.

[0096] The multimodal feature vector formed in step S3 is used as input data. These input data will enter the GRU network in chronological order, and the data at each time point corresponds to an input vector.

[0097] 2. Application of Attention Mechanism

[0098] An attention layer is added after the GRU network to evaluate and weight the importance of data in different time windows. The design of the attention layer adopts additive attention (also known as soft attention), and its formula is:

[0099]

[0100] The weighted sum is calculated to form the final context vector c, which represents the weighted representation of the entire input sequence, and is calculated as:

[0101]

[0102] Among them, W a and b a are the weights and biases of the attention layer, ν is the learned context vector, and c is the final context vector, which represents the weighted representation of the entire sequence.

[0103] Combining the GRU network with a precisely defined attention mechanism can more effectively identify which historical data features are most critical to predicting future access patterns, thereby improving the accuracy of predictions. The introduction of the attention layer not only enhances the model's ability to process time series data, but also improves the model's interpretability, making the prediction process and results more transparent.

[0104] Step S5: Predicting the data blocks that will be frequently accessed in the future

[0105] Based on the output of the GRU network and the attention layer, predict the data blocks that may be frequently accessed in the future. This prediction process utilizes historical and current data access patterns and emphasizes those time windows and data blocks that are more likely to be important through attention weights. The prediction model can be a classifier or a regression model, customized according to business needs.

[0106] In this embodiment, step S5 includes the following scheme to ensure accurate prediction of data access behavior based on the output of the deep learning model:

[0107] 1. Build a prediction model

[0108] A fully connected layer (or multiple layers) is used to receive the context vector c output from the GRU and attention layer of step S4, which comprehensively considers the important information of all time windows.

[0109] The purpose of this fully connected layer is to predict which data blocks may be frequently accessed in the future based on the context vector. This layer can use sigmoid or softmax activation function to output the probability or classification result of each data block being accessed according to actual needs.

[0110] 2. Training and tuning the model

[0111] Use historical access data (data that has been processed and labeled by the previous steps) as the training set, which includes feature vectors of data blocks and their access frequency labels.

[0112] According to the model output and actual access conditions, select a suitable loss function, such as cross entropy loss, to train the model.

[0113] Use optimization algorithms such as Adam or SGD to train the model to ensure that the model converges in a suitable time and achieves high prediction accuracy.

[0114] 3. Implement prediction and assessment

[0115] After the model training is completed, the latest or real-time data is used to input the model to predict future access behavior.

[0116] According to the access probability of the data block predicted by the model, a threshold is set (such as a probability greater than 0.5 or other value set according to the specific situation), and the data blocks above this threshold are marked as likely to be frequently accessed in the future.

[0117] By accurately predicting which data blocks will be frequently accessed in the future, this step can significantly improve the efficiency of data prefetching and system performance. In addition, this prediction method reduces unnecessary data prefetching operations, thereby reducing the load on the storage system and improving the response speed.

[0118] Step S6: Data pre-fetching decision

[0119] According to the prediction result of step S5, data blocks with high confidence are selected for pre-fetching. Data pre-fetching is determined based on the probability threshold output by the prediction model. Only when the access probability of a data block exceeds the set threshold, it is marked as pre-fetched. These data blocks will be pre-loaded into the cache of the SSD for fast access.

[0120] In this embodiment, step S6 includes the following schemes to ensure effective data prefetching based on accurate prediction results:

[0121] 1. Determine the prefetch threshold

[0122] Based on the access probability predicted by the model, a prefetch threshold is set. This threshold is a key parameter for deciding whether to prefetch a data block. For example, the threshold can be set to 0.5 or adjusted to a more appropriate value based on historical performance data.

[0123] The threshold selection can be dynamically adjusted according to the actual operation of the system. For example, if the system is idle, the threshold can be lowered to pre-fetch more data in advance; if the system is heavily loaded, the threshold can be increased to reduce pre-fetching activities.

[0124] 2. Select data blocks for prefetching

[0125] According to the access probability output by the prediction model, data blocks with access probability higher than the set threshold are selected for pre-fetching. These data blocks are considered to have a higher probability of being accessed in the future.

[0126] A pre-fetch operation is performed on the selected data blocks, that is, the data blocks are moved from the slow storage (such as the HDD) to the fast storage (such as the cache area of ​​the SSD) in advance.

[0127] 3. Prefetch effect monitoring and evaluation

[0128] After performing the prefetch operation, continuously monitor the system's response time and load to evaluate the effectiveness of the prefetch strategy.

[0129] Regularly evaluate the consistency between the actual access situation of prefetched data blocks and the predicted results to adjust and optimize the prediction model and prefetch strategy.

[0130] By selectively prefetching data based on accurate data access predictions, this step can significantly improve the system's data access speed and responsiveness while reducing unnecessary data transfer and storage load. In addition, this dynamically adjusted prefetching strategy can better adapt to system load changes and optimize the use of storage resources.

[0131] The present invention significantly improves the accuracy and efficiency of data pre-fetching by combining content analysis, multimodal data fusion, time series processing, and advanced attention mechanisms. By intelligently predicting and pre-fetching data blocks that may be frequently accessed in the future, data access latency can be effectively reduced, the performance of solid-state drives can be improved, and the user experience can be improved. In addition, the method is highly adaptable and can cope with changes in various data access modes, and has broad application prospects.

[0132] In a preferred embodiment of the present invention, the specific steps for content analysis of text data are as follows, which aims to convert text data into numerical vectors with semantic information by using a pre-trained language model and a clustering algorithm for further processing and feature fusion.

[0133] 1. Text Embedding

[0134] Choose a suitable pre-trained language model, such as BERT, GPT or other similar models, to perform deep semantic analysis on text data. These models can convert text into dense vector representations, each of which captures rich semantic information in the text.

[0135] Apply the language model to the entire text or selected key paragraphs, and output the embedding vectors of each paragraph. These vectors reflect the deep semantic features of the text and provide the basis for the next step of clustering analysis.

[0136] 2. Text Vector Clustering

[0137] Apply clustering algorithms, such as K-means or hierarchical clustering, to group the text embedding vectors. The purpose of clustering is to group semantically similar texts into one category, thereby identifying different topics or concepts.

[0138] The center vector of each cluster represents the average or typical characteristics of the category and can be regarded as the representative vector of the subject category.

[0139] 3. Label Generation and Vector Representation Selection

[0140] The cluster center to which each text belongs is used as its content label. These labels reflect the main theme or concept of the text and provide identification for subsequent data processing and analysis.

[0141] The vectors of cluster centers are selected as numerical vectors for multimodal feature fusion. These vectors are suitable for machine learning processing in numerical form while maintaining the rich semantic information of text data.

[0142] This preferred embodiment not only improves the semantic depth of data preprocessing by converting text data into semantic vectors and performing effective clustering, but also simplifies the subsequent data analysis and processing process through labeling. In addition, this method is conducive to improving the accuracy and efficiency of the system when processing text data, especially in application scenarios that require in-depth analysis and prediction of text content, and can provide more accurate data support and decision-making basis.

[0143] In a preferred embodiment of the present invention, the specific steps for content analysis of image data are as follows, aiming to effectively extract and compress key visual information from images by using image semantic segmentation technology and deep feature extraction technology, as well as the application of autoencoders, so as to provide accurate input for multimodal feature fusion and data pre-fetching.

[0144] 1. Image Semantic Segmentation

[0145] Image semantic segmentation techniques, such as Mask R-CNN, U-Net or other applicable deep learning models, are used to process the image to distinguish and identify the main object and background in the image. The key to this step is to segment the various visual elements in the image into independent parts so that subsequent feature extraction can be performed on specific objects.

[0146] Select a segmentation model suitable for the image dataset to be processed, configure appropriate training parameters such as learning rate, number of iterations, etc., and perform sufficient model training to achieve high-precision segmentation results.

[0147] 2. Deep feature extraction

[0148] Use deep learning techniques to extract features for each identified object. Pre-trained convolutional neural networks (such as VGG, ResNet) can be used to extract visual features from each segmented object.

[0149] The features extracted from each object are merged to form a comprehensive visual feature representation. The merging method may include concatenation of feature vectors or more advanced feature integration through a specific fusion network.

[0150] 3. Use autoencoders for feature dimensionality reduction

[0151] Design and train an autoencoder network to process the merged feature vector. The autoencoder learns how to reconstruct the input after compression to discover and retain the most critical visual information while reducing the dimensionality of the vector.

[0152] The autoencoder outputs a compressed numerical vector representation that significantly reduces the data dimension while retaining the most important visual features for the prediction task.

[0153] This advanced image processing and data compression method can more effectively process and analyze large-scale image data, while improving the efficiency and accuracy of data pre-fetching and processing systems. In addition, this method ensures the preservation of key visual information while maintaining data compression and simplifying the processing flow, which is particularly important for data pre-fetching strategies based on visual content.

[0154] In a preferred embodiment of the present invention, the gated recurrent unit network has at least one hidden layer, each hidden layer contains multiple GRU units, and each GRU unit updates its hidden state according to the hidden state at the previous moment and the input at the current moment. This design enables the network to effectively capture and process long-distance dependencies in time series data.

[0155] The core of the GRU unit is its state update mechanism, which allows the network to avoid the gradient vanishing problem while maintaining long-term dependencies. The state update formula for each GRU unit is:

[0156]

[0157] Among them, h t is the hidden state at the current moment, z t It is the update gate. is a candidate hidden state.

[0158] Update gate and candidate hidden state calculation

[0159] Update gate: The update gate controls the contribution of the previous hidden state to the current hidden state. Its calculation formula is:

[0160] z t =σ(W z ·[h t-1 ,x t ]+b z )

[0161] Here σ is the sigmoid activation function, W z is the weight matrix of the update gate, b z is the bias term.

[0162] Candidate hidden state: The candidate hidden state is a new state that may be reached at the current moment. Its calculation involves resetting the gate r t , the formula is:

[0163]

[0164] Here tanh is the hyperbolic tangent activation function, which provides nonlinear processing capabilities. h is the weight matrix of the candidate hidden state, b h is the bias term.

[0165] Reset gate: The reset gate determines the influence of the previous hidden state on the candidate hidden state. Its calculation formula is:

[0166] r t =σ(W r ·[h t-1 ,x t ]+b r )

[0167] Here W r is the weight matrix of the reset gate, b r is the bias term.

[0168] Using the GRU network to process time series data can effectively improve the model's ability to capture time-dependent information, and is particularly suitable for processing applications such as solid-state drive data pre-fetching that need to consider historical access patterns. In addition, the GRU gating mechanism improves the stability and efficiency of the network when processing long sequences.

[0169] In a preferred embodiment of the present invention, it is described how to implement an attention layer with a weighted mechanism on top of a GRU network to optimize the data prefetching process.

[0170] 1. Configuration of the attention mechanism

[0171] A weighted attention layer is applied on top of the GRU network, the purpose of which is to perform a weighted sum of the GRU outputs at each time step by calculating the attention weights to highlight the features of those time steps that are most critical to the prediction task.

[0172] 2. Calculate attention weights

[0173] Weights and biases: Define the weight matrix W a and the bias term b a , these parameters are learned during the training process and are used to transform the hidden state h output by GRU t Transformed into an intermediate state, which is used to calculate the attention score at each time step.

[0174] Activation function of the intermediate state: The weighted hidden state and the bias are processed by the hyperbolic tangent (tanh) activation function to introduce nonlinearity and increase the expressive power of the model:

[0175] tanh(W a ·h t +b a )

[0176] Calculate the score and normalize: Use the learned context vector v (usually learned during training) to transpose the activation output of each time step and multiply it by the vector v to get the raw attention score for each time step. Then, normalize the scores of all time steps through the softmax function to ensure that the sum of the attention weights of all time steps is 1:

[0177]

[0178] 3. Calculate weighted hidden state representation

[0179] Weighted sum: Use the attention weight a calculated in the previous step t , the hidden state h of each time step t Perform weighted summation to form a weighted hidden state representation, which integrates all the important information in the entire sequence:

[0180]

[0181] This context vector c represents the weighted representation of the entire input sequence and contains the most critical information for the prediction task.

[0182] The implementation of this attention mechanism enables the model to dynamically focus on the most important parts for prediction, thereby improving the accuracy and efficiency of prediction. In addition, by emphasizing data at critical moments, data pre-fetching can be performed more effectively, resource allocation can be optimized, unnecessary data loading can be reduced, and the response speed and efficiency of the system can be improved.

[0183] The solid state drive data pre-fetching device described below and the solid state drive data pre-fetching method described above may refer to each other.

[0184] Please refer to the attached Figure 2 The present invention also provides a solid state hard disk data pre-fetching device, comprising:

[0185] The data collection unit 100 is used to collect access trajectory data of the solid state drive according to a fixed time window;

[0186] A content analysis unit 200, configured to perform content analysis on each data block and generate relevant tags;

[0187] A label vectorization unit 300, used to convert labels into numerical vectors;

[0188] A feature fusion unit 400, used to combine the label vector with the traditional access mode feature to form a multimodal feature vector;

[0189] A neural network processing unit 500, including a gated recurrent unit network and an attention layer, for processing multimodal feature vectors and calculating attention weights;

[0190] A prediction unit 600, for predicting data blocks that may be frequently accessed in the future based on the attention weights and the gated recurrent unit network output;

[0191] The data pre-fetching unit 700 is used to select data blocks with high confidence for pre-fetching according to the prediction result, and load the selected data blocks into the cache area of ​​the solid state drive.

[0192] The device of this embodiment can be used to execute the above method embodiment, and its principles and technical effects are similar, which will not be repeated here.

[0193] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A solid state drive data pre-fetching method based on attention mechanism, characterized in that: The following steps are involved: Collect access trajectory data of SSDs and divide the data into fixed time windows; Perform content analysis on each data block, generate relevant labels based on the content, and convert these labels into numerical vectors; Combining the label vector with traditional access pattern features to form a multimodal feature vector; Processing the multimodal feature vector using a gated recurrent unit network, evaluating the importance of data in different time windows through an attention layer, and calculating attention weights; Based on the attention weights and the gated recurrent unit network output, predicting data blocks that may be frequently accessed in the future; According to the prediction results, data blocks with high confidence are selected for prefetching.

2. According to the method of claim 1, the method is characterized in that: The content analysis uses natural language processing technology to extract keywords from text data and uses convolutional neural network to extract image features from image data.

3. According to the method of claim 2, the method is characterized in that: For text data, the content analysis includes the following steps: Use pre-trained language models for text embedding to convert entire texts or key paragraphs into semantically rich vector representations; Use clustering algorithm to cluster text vectors, and each cluster center represents a topic category; The cluster center label to which the text belongs is used as the content label, and the vector representation of the cluster center is used as a numerical vector for subsequent multimodal feature fusion.

4. According to the method of claim 2, the method is characterized in that: For image data, the content analysis includes the following steps: Use image semantic segmentation technology to perform advanced image processing to identify the main objects and background in the image; Use deep feature extraction techniques for each object identified and merge these features to form a comprehensive visual feature representation of the image; The autoencoder is used to reduce the dimension of the merged features and generate compressed numerical vector representations, which reduce the processing complexity while retaining key visual information.

5. According to the method of claim 1, the method is characterized in that: The gated recurrent unit network has at least one hidden layer, each hidden layer contains multiple GRU units, and each GRU unit updates its hidden state according to the hidden state at the previous moment and the input at the current moment. The update formula is: Among them, h t is the hidden state at the current moment, z t It is the update gate. is a candidate hidden state, calculated as follows: z t =σ(W z ·[h t-1 ,x t ]+b z ) r t =σ(W r ·[h t-1 ,x t ]+b r ) Among them, W z ,W h ,W r is the weight matrix, b z ,b h ,b r is the bias term, r t is the reset gate and σ is the sigmoid activation function.

6. The method for pre-fetching data of a solid state drive based on an attention mechanism according to claim 5, characterized in that: The attention uses a weighted mechanism to weight the output of the GRU network by calculating the attention weight to form a weighted hidden state representation. The calculation formula is: Among them, W a and b a are the weights and biases of the attention layer, v is the learned context vector, and c is the final context vector, which represents the weighted representation of the entire sequence.

7. The method for pre-fetching data of a solid state drive based on an attention mechanism according to claim 1, characterized in that: The step of predicting data blocks that may be frequently accessed in the future based on the attention weights and the output of the gated recurrent unit network includes using a fully connected layer to process the outputs of the gated recurrent unit network and the attention layer to generate a final predicted data block.

8. The method for pre-fetching data of a solid state drive based on an attention mechanism according to claim 1, characterized in that: The confidence of the prediction result is determined by setting a threshold, and the pre-fetch operation is performed only when the predicted access probability is higher than the threshold.

9. The method for pre-fetching data of a solid state drive based on an attention mechanism according to claim 1, characterized in that: Pre-fetching the data blocks includes loading pre-selected data blocks into a cache area of ​​the solid state drive.

10. A solid state hard disk data pre-fetching device, used to implement the method according to any one of claims 1 to 9, characterized in that: include: A data collection unit, used to collect access trajectory data of the solid state drive according to a fixed time window; A content analysis unit, used to perform content analysis on each data block and generate relevant tags; A label vectorization unit, used for converting the label into a numerical vector; A feature fusion unit, which is used to combine the label vector with the traditional access pattern features to form a multimodal feature vector; A neural network processing unit, including a gated recurrent unit network and an attention layer, is used to process multimodal feature vectors and calculate attention weights; A prediction unit, which is used to predict data blocks that may be frequently accessed in the future based on the attention weights and the output of the gated recurrent unit network; The data pre-fetch unit is used to select data blocks with high confidence for pre-fetching according to the prediction results, and load the selected data blocks into the cache area of ​​the solid state drive.

Images