Data access method and device, equipment and medium

Abstract

The invention discloses a data access method and device, equipment and a medium, and relates to the field of data access, and the method comprises the steps that a disk array card inputs an access address sequence containing a current data block access address and a historical data block access address into a probability distribution prediction model, so as to add corresponding physical position information to each address in the sequence, extracting time features and spatial features of the added sequence, and determining fusion features based on the time features and the spatial features to predict access probability distribution of each data block in the disk array next time; and after accessing the data of the actual to-be-accessed data block from the local cache by using the current data block access address, determining a to-be-added data block and a to-be-eliminated data block from the disk array based on the actual to-be-accessed data block and the access probability distribution so as to update the local cache. According to the method and the device, the data is prefetched into the local cache, so that efficient access of the data is realized when a large number of concurrent requests are faced.

Description

Technical Field

[0001] The present invention relates to the field of data access, and particularly to a data access method, apparatus, device, and medium. Background Art

[0002] For data storage, different storage media have different access speeds. With the continuous development of information technology, the demand for data storage by enterprises and high-performance computing (HPC) is showing a rapid growth trend. In this context, the redundant arrays of independent disks (RAID) technology and the solid state disk (SSD) technology have become the core technologies for data storage.

[0003] The RAID technology integrates multiple solid state disks into a disk array, which not only optimizes the redundant storage of data, improves the fault tolerance of the storage system, but also enhances the data transmission efficiency through parallel I / O (input / output) operations. In addition, the solid state disk provides the necessary performance support for data-intensive and computing-intensive applications with its excellent random read and write performance, thus ensuring that these applications can achieve high-speed and reliable data processing.

[0004] In an environment with high-intensity workloads or specific data access patterns, even a storage system integrating a redundant arrays of independent disks and a solid state disk may encounter performance bottlenecks. The root cause of this phenomenon is that the traditional data access mechanism mainly relies on the request-response mode, that is, accessing data from the solid state disks of the disk array based on requests for response. When facing a large number of concurrent requests, the frequent request and response interactions may cause a significant reduction in the performance of the solid state disk, and further lead to an increase in data access latency.

[0005] It can be seen that how to achieve efficient data access in the face of a large number of concurrent requests is a problem that needs to be solved by those skilled in the art. Summary of the Invention

[0006] The purpose of the embodiments of the present invention is to provide a data access method, apparatus, device, and medium, which prefetch data into the local cache to achieve efficient data access in the face of a large number of concurrent requests. The specific scheme is as follows:

[0007] In a first aspect, the present invention provides a data access method, which is applied to a disk array card and includes:

[0008] Determine the current access address sequence based on the current data block access address and the historical data block access address;

[0009] Input the current access address sequence into a pre-trained probability distribution prediction model to add corresponding physical location information to each address in the current access address sequence, extract time features and space features from the current access address sequence after adding, determine a fusion feature based on the extracted time features and space features, and predict the access probability distribution of each data block in the disk array according to the fusion feature; the physical location information is the physical location information of the data block corresponding to each address in the disk array.

[0010] After accessing the actual data block to be accessed from the local cache using the current data block access address, determine the data block to be added and the data block to be evicted from the disk array based on the actual data block to be accessed and the access probability distribution.

[0011] Load the data of the data block to be added into the local cache and delete the data of the data block to be evicted from the local cache.

[0012] Optionally, the extraction of time features and space features from the current access address sequence after adding includes:

[0013] Based on the timestamp information carried by each address in the current access address sequence after adding, extract time features for each address to obtain the extracted time features.

[0014] Use a convolutional layer with a preset dimension to extract space features for each address in the current access address sequence after adding to obtain the extracted space features.

[0015] Optionally, determining the fusion feature based on the extracted time features and space features includes:

[0016] Determine a query vector, a key vector, and a value vector based on the extracted time features, and determine a time attention weight matrix according to the query vector and the key vector;

[0017] Construct a spatial association matrix based on the physical location information of each address in the current access address sequence after adding;

[0018] Determine a spatial attention weight matrix according to the spatial association matrix and the extracted space features;

[0019] Perform weighted fusion on the value vector and the time attention weight matrix, the extracted space features and the spatial attention weight matrix to determine the fusion feature.

[0020] Optionally, constructing a spatial association matrix based on the physical location information of each address in the current access address sequence after adding includes:

[0021] Based on the physical location information of any two addresses in the current access address sequence after addition, determine the spatial proximity of the data blocks corresponding to the two addresses in the disk array, and construct a spatial association matrix according to the spatial proximity.

[0022] Optionally, based on the actual data blocks to be accessed and the access probability distribution, determine the data blocks to be added and the data blocks to be eliminated from the disk array, including:

[0023] Determine the first data blocks whose data has not been loaded into the local cache from the disk array, and determine the second data blocks whose data has been loaded into the local cache from the disk array;

[0024] Using the access probability distribution, determine the data blocks in the first data blocks whose access probability is greater than the first preset probability to obtain the first target data blocks;

[0025] Using the access probability distribution, determine the data blocks in the first data blocks whose access probability is not greater than the first preset probability and is greater than the second preset probability to obtain the first initial data blocks;

[0026] Based on the physical location information of the first initial data blocks and the actual data blocks to be accessed in the disk array respectively, determine the spatial proximity between the first initial data blocks and the actual data blocks to be accessed, and when the spatial proximity between the first initial data blocks and the actual data blocks to be accessed is greater than the first preset proximity, determine the first initial data blocks as the second target data blocks;

[0027] Determine the data blocks to be added according to the first target data blocks and the second target data blocks;

[0028] Using the access probability distribution, determine the data blocks in the second data blocks whose access probability is less than the third preset probability to obtain the second initial data blocks;

[0029] Based on the physical location information of the second initial data blocks and the actual data blocks to be accessed in the disk array respectively, determine the spatial proximity between the second initial data blocks and the actual data blocks to be accessed, and when the spatial proximity between the second initial data blocks and the actual data blocks to be accessed is less than the second preset proximity, determine the second initial data blocks as the data blocks to be eliminated.

[0030] Optionally, before inputting the current access address sequence into the pre-trained probability distribution prediction model, it further includes:

[0031] Obtain the data block access addresses within a preset time period to obtain the original access address sequence;

[0032] Based on the window size and the sliding step, perform window sliding on the original access address sequence to divide the original access address sequence into several subsequences;

[0033] Use the last address in each subsequence as the output, and use the other addresses in each subsequence except the last address as the input to construct a training set;

[0034] Use the training set to train the initial probability distribution prediction model to obtain a pre-trained probability distribution prediction model; the initial probability distribution prediction model is a model constructed based on a long short-term memory network.

[0035] Optionally, before using the last address in each subsequence as the output, it further includes:

[0036] Determine the similarity between any two subsequences among a number of subsequences;

[0037] When the similarity is greater than a preset similarity, perform deduplication processing on any two subsequences to delete one of the two subsequences from any two subsequences to obtain a number of subsequences after deduplication processing.

[0038] In a second aspect, the present invention provides a data access device applied to a disk array card, including:

[0039] An address sequence determination module for determining a current access address sequence based on the current data block access address and the historical data block access address;

[0040] A probability distribution prediction module for inputting the current access address sequence into the pre-trained probability distribution prediction model to add corresponding physical location information to each address in the current access address sequence, and extract time features and spatial features from the current access address sequence after adding, determine a fusion feature based on the extracted time features and spatial features, and predict the access probability distribution of each data block in the disk array next time according to the fusion feature; the physical location information is the physical location information of the data block corresponding to each address in the disk array;

[0041] A data block determination module for, after accessing the data of the actual data block to be accessed from the local cache using the current data block access address, determining a data block to be added and a data block to be eliminated from the disk array based on the actual data block to be accessed and the access probability distribution;

[0042] A cache update module for loading the data of the data block to be added into the local cache and deleting the data of the data block to be eliminated from the local cache.

[0043] In a third aspect, the present invention provides an electronic device, including:

[0044] A memory for storing a computer program;

[0045] A processor for executing a computer program to implement the steps of the foregoing data access method.

[0046] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the foregoing data access method are implemented.

[0047] In the present invention, a disk array card determines a current access address sequence based on a current data block access address and a historical data block access address; inputs the current access address sequence into a pre-trained probability distribution prediction model to add corresponding physical location information to each address in the current access address sequence, and extracts time features and spatial features from the current access address sequence after adding, determines a fusion feature based on the extracted time features and spatial features, and predicts the access probability distribution of each data block in the disk array next time according to the fusion feature; the physical location information is the physical location information of the data block corresponding to each address in the disk array; after accessing the actual data block to be accessed from the local cache using the current data block access address, determines a data block to be added and a data block to be eliminated from the disk array based on the actual data block to be accessed and the access probability distribution; loads the data of the data block to be added into the local cache, and deletes the data of the data block to be eliminated from the local cache.

[0048] Beneficial effects: By inputting the access address sequence into a pre-trained probability distribution prediction model, the present invention predicts the access probability distribution of each data block in the disk array next time, and based on the access probability distribution, loads the data of the data block most likely to be accessed next time in the disk array into the local cache, and eliminates the data of the data block least likely to be accessed next time in the disk array from the local cache, so as to prefetch the data into the local cache, so that when the data block access address is obtained next time, the data of the corresponding data block can be directly accessed from the local cache. Compared with accessing the data of the corresponding data block from the disk array, by reducing the request and response interaction with the disk array, not only the performance impact on the disk array is reduced, but also the data access efficiency can be improved to achieve efficient data access in the face of a large number of concurrent requests. Further, by adopting an access address sequence including a current data block access address and a historical data block access address, and predicting the access probability distribution of each data block in the disk array next time based on the spatio-temporal fusion feature of the access address sequence, the prediction accuracy of the access probability distribution is improved. Description of the Drawings

[0049] To more clearly illustrate the embodiments of the present invention, the following will briefly introduce the drawings required for use in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0050] Figure 1 It is a flowchart of a data access method provided by an embodiment of the present invention;

[0051] Figure 2 It is an architecture diagram of a data access provided by an embodiment of the present invention;

[0052] Figure 3 It is a flowchart of a data access provided by an embodiment of the present invention;

[0053] Figure 4 It is a schematic structural diagram of a data access device provided by an embodiment of the present invention;

[0054] Figure 5 It is a structural diagram of an electronic device provided by an embodiment of the present invention. Specific Embodiments

[0055] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.

[0056] The terms "including" and "having" in the specification of the present invention and any deformations related to "including" and "having" are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but may include steps or units not listed.

[0057] To enable those skilled in the art of this technology to better understand the solution of the present invention, the following will further elaborate on the present invention in conjunction with the drawings and specific embodiments.

[0058] In an environment with high-intensity workloads or specific data access patterns, even a storage system integrated with redundant arrays of independent disks and solid-state drives may encounter performance bottlenecks. The root cause of this phenomenon lies in the fact that traditional data access mechanisms mainly rely on the request-response mode, that is, accessing data from the solid-state drives of disk arrays based on requests for response. When faced with a large number of concurrent requests, frequent request and response interactions may lead to a significant reduction in the performance of solid-state drives, thereby increasing data access latency. To this end, the present invention provides a data access method that pre-fetches data into the local cache to achieve efficient data access when faced with a large number of concurrent requests.

[0059] See Figure 1 As shown, an embodiment of the present invention provides a data access method applied to a disk array card, including:

[0060] Step S11: Determine the current access address sequence based on the current data block access address and the historical data block access address.

[0061] In an embodiment of the present invention, the disk array card uses the data block access address obtained at the current moment as the current data block access address, and combines the current data block access address and the historical data block access address into the current access address sequence. It should be noted that the disk array card is equivalent to an array controller and is used to manage and control the disk array.

[0062] Step S12: Input the current access address sequence into a pre-trained probability distribution prediction model to add corresponding physical location information to each address in the current access address sequence, extract time features and spatial features from the added current access address sequence, determine the fusion features based on the extracted time features and spatial features, and predict the access probability distribution of each data block in the disk array according to the fusion features; the physical location information is the physical location information of the data block corresponding to each address in the disk array.

[0063] In an embodiment of the present invention, after determining the current access address sequence, the disk array card inputs the current access address sequence into a pre-trained probability distribution prediction model to determine the physical location information of the data block corresponding to each address in the current access address sequence, thereby adding corresponding physical location information to each address in the current access address sequence, extracting time features and spatial features from the added current access address sequence, then determining the fusion features based on the extracted time features and spatial features, and predicting the access probability distribution of each data block in the disk array according to the fusion features.

[0064] It should be noted that the probability distribution prediction model adopts a long short-term memory network (LSTM, Long Short-Term Memory). Among them, the long short-term memory network belongs to a type of recurrent neural network (RNN, Recurrent Neural Network), which is specifically designed for efficient processing of sequence data. The long short-term memory network can identify and capture long-term and short-term dependencies in sequence data, and maintain and update information through its unique memory cell mechanism to ensure the effective transmission of sequence features during processing. Since data access essentially contains an access address sequence organized in chronological order, the long short-term memory network can be selected to analyze and process the access address sequence, thereby predicting the probability distribution of the next access to each data block in the disk array and improving the prediction accuracy of the access probability distribution.

[0065] Among them, the physical location information of the data block corresponding to each address in the current access address sequence in the disk array can refer to the SSD channel or the storage granule (Die) number of the data block in the disk array, etc.

[0066] In the process of extracting time features and spatial features from the added current access address sequence, based on the timestamp information carried by each address in the added current access address sequence, time features are extracted for each address to obtain the extracted time features; a convolutional layer with a preset dimension is used to extract spatial features for each address in the added current access address sequence to obtain the extracted spatial features.

[0067] That is, the added current access address sequence can be expressed as , where each s includes the data block access address and the physical location information of the data block corresponding to this address in the disk array. The probability distribution prediction model obtains the timestamp information carried by each address in the added current access address sequence, and extracts time features for each address based on the timestamp information to obtain the extracted time features ; a convolutional layer with a preset dimension is used to obtain the physical location information of each address in the added current access address sequence, and spatial features are extracted for each address in the added current access address sequence based on the physical location information to obtain the extracted spatial features . Among them, the convolutional layer with a preset dimension can adopt a one-dimensional convolutional layer in this embodiment.

[0068] In this way, in this embodiment, the probability distribution prediction model dynamically focuses on the key time nodes of each address in the access address sequence to capture the time correlation of each address in the access address sequence, and then uses the convolutional layer with a preset dimension to capture the spatial correlation of each address in the access address sequence, so as to enhance the subsequent prediction ability of the access probability distribution through the time correlation and the spatial correlation.

[0069] Further, in the process of determining the fusion feature based on the extracted time feature and spatial feature, a query vector, a key vector, and a value vector are determined based on the extracted time feature, and a time attention weight matrix is determined according to the query vector and the key vector; based on the physical location information of each address in the current access address sequence after addition, a spatial correlation matrix is constructed; according to the spatial correlation matrix and the extracted spatial feature, a spatial attention weight matrix is determined; the value vector, the time attention weight matrix, the extracted spatial feature, and the spatial attention weight matrix are weighted and fused to determine the fusion feature.

[0070] Specifically, after obtaining the extracted time feature, the extracted time feature is calculated based on the model weight parameters corresponding to the query vector, the key vector, and the value vector respectively to obtain the query vector, the key vector, and the value vector. The involved formulas are as follows:

[0071] ;

[0072] ;

[0073] ;

[0074] Among them, represents the query vector, represents the extracted time feature, represents the key vector, represents the value vector, represents the model weight parameter corresponding to the query vector, represents the model weight parameter corresponding to the key vector, represents the model weight parameter corresponding to the value vector respectively.

[0075] And the calculation formula of the time attention weight matrix is: ; Among them, represents the time attention weight matrix, represents the query vector, represents the key vector, T represents the transpose operation of the matrix, represents the dimension of the key vector, and Softmax represents the activation function.

[0076] In the process of constructing a spatial association matrix based on the physical location information of each address in the current access address sequence after addition, based on the physical location information of any two addresses in the current access address sequence after addition, determine the spatial proximity of the data blocks corresponding to the two addresses in the disk array, and construct a spatial association matrix according to the spatial proximity.

[0077] That is, for any two addresses in the current access address sequence after addition, based on the physical location information of the data blocks corresponding to the two addresses in the disk array, determine the spatial proximity of the data blocks corresponding to the two addresses in the disk array, and construct a spatial association matrix according to the spatial proximity between the data blocks corresponding to each address in the current access address sequence after addition.

[0078] Among them, the calculation formula for the spatial proximity of the data blocks corresponding to any two addresses in the disk array is: ; represents the data block corresponding to one of the two addresses and the data block corresponding to the other address the spatial proximity between them, represents an adjustable bandwidth parameter, the Loc function is used to extract the physical location information of the data block in the disk array, and exp represents the natural exponential function.

[0079] After obtaining the spatial association matrix, the calculation formula for the spatial attention weight matrix is: ; where, represents the spatial attention weight matrix, represents the extracted spatial feature, represents the fully connected layer parameter of the model, represents the spatial association matrix.

[0080] Finally, for the determination of the fused feature, first, based on the extracted spatial feature and the model weight parameter corresponding to the spatial feature, determine the spatial feature to be fused, and based on the adaptive weight coefficient, perform weighted fusion on the value vector and the time attention weight matrix, the spatial feature to be fused and the spatial attention weight matrix to determine the fused feature. The formulas involved are as follows:

[0081] ;

[0082] ;

[0083] Among them, represents the extracted spatial feature, represents the model weight parameter corresponding to the spatial feature, represents the spatial feature to be fused, represents the adaptive weight coefficient, represents the value vector, represents the temporal attention weight matrix, represents the spatial attention weight matrix, represents the fused feature.

[0084] It should be noted that for the adaptive weight coefficient , its calculation formula is: ; The function represents the Sigmoid activation function, and the avg function represents the aggregation function for calculating the average value of a set of numerical values, represents the extracted temporal feature, represents the extracted spatial feature, and both represent the model weight parameters.

[0085] Furthermore, after determining the fused feature, the access probability distribution of the next access to each data block in the disk array can be predicted based on the fused feature, that is, ; where represents the fused feature, represents the access probability distribution of the next access to each data block in the disk array, and both represent the parameters of the model output layer.

[0086] Step S13, after accessing the data of the actual data block to be accessed from the local cache using the current data block access address, determine the data block to be added and the data block to be eliminated from the disk array based on the actual data block to be accessed and the access probability distribution.

[0087] In the embodiment of the present invention, when the disk array card obtains the current data block access address, it can access the data of the actual data block to be accessed from the local cache based on the current data block access address, where the actual data block to be accessed is the data block corresponding to the current data block access address.

[0088] Furthermore, after knowing the actual data block to be accessed corresponding to the current data block access address, based on the actual data block to be accessed and the access probability distribution of the next access to each data block in the disk array, the data block to be added and the data block to be eliminated can be determined from the disk array, thereby updating the data in the local cache.

[0089] Specifically, first, determine the first data blocks whose data has not been loaded into the local cache from the disk array, and determine the second data blocks whose data has been loaded into the local cache from the disk array; then, using the access probability distribution, determine the data blocks with an access probability greater than the first preset probability from the first data blocks to obtain the first target data blocks; using the access probability distribution, determine the data blocks with an access probability not greater than the first preset probability and greater than the second preset probability from the first data blocks to obtain the first initial data blocks; based on the physical location information of the first initial data blocks and the actual data blocks to be accessed in the disk array respectively, determine the spatial proximity between the first initial data blocks and the actual data blocks to be accessed, and when the spatial proximity between the first initial data blocks and the actual data blocks to be accessed is greater than the first preset proximity, determine the first initial data blocks as the second target data blocks; determine the data blocks to be added according to the first target data blocks and the second target data blocks. Further, using the access probability distribution, determine the data blocks with an access probability less than the third preset probability from the second data blocks to obtain the second initial data blocks; based on the physical location information of the second initial data blocks and the actual data blocks to be accessed in the disk array respectively, determine the spatial proximity between the second initial data blocks and the actual data blocks to be accessed, and when the spatial proximity between the second initial data blocks and the actual data blocks to be accessed is less than the second preset proximity, determine the second initial data blocks as the data blocks to be eliminated.

[0090] It should be noted that the first preset probability is greater than the second preset probability, and the second preset probability is greater than the third preset probability. Moreover, the first preset proximity is greater than the second preset proximity; the spatial proximity reflects the distance between the physical positions of two data blocks. The greater the spatial proximity, the closer the physical positions of the two data blocks are. On the contrary, the smaller the spatial proximity, the farther the physical positions of the two data blocks are.

[0091] It should also be noted that the reason for using the spatial proximity in the embodiments of the present invention is that in an actual storage system, adjacent data blocks (such as data in the same stripe) are often continuously accessed due to spatial locality. Therefore, when the currently accessed data block is known, the data blocks at a relatively close distance from the currently accessed data block have a high probability of being accessed next time.

[0092] Step S14: Load the data of the data blocks to be added into the local cache, and delete the data of the data blocks to be eliminated from the local cache.

[0093] After determining the data blocks to be added and the data blocks to be eliminated from the disk array, load the data of the data blocks to be added into the local cache, and delete the data of the data blocks to be eliminated from the local cache.

[0094] In an embodiment of the present invention, by prefetching the data of the data block that is most likely to be accessed next time into the local cache, when the data block access address is obtained next time, the data of the corresponding data block can be directly accessed from the local cache instead of accessing the data of the corresponding data block from the disk array, thereby reducing the request and response interactions with the disk array, reducing the performance impact on the disk array, and improving the data access efficiency to achieve efficient data access in the face of a large number of concurrent requests. Further, in an embodiment of the present invention, by removing the data of the data block that is least likely to be accessed next time from the local cache, the waste of storage resources of the local cache is reduced, and the reasonable utilization of the local cache is realized.

[0095] The training of the probability distribution prediction model in the embodiment of the present invention specifically may include: obtaining the data block access addresses within a preset time period to obtain the original access address sequence; based on the window size and the sliding step length, performing window sliding on the original access address sequence to divide the original access address sequence into several subsequences; constructing a training set by using the last address in each subsequence as the output and the other addresses in each subsequence except the last address as the input; and then training the initial probability distribution prediction model by using the training set to obtain a pre-trained probability distribution prediction model. Among them, the initial probability distribution prediction model is a model constructed based on a long short-term memory network.

[0096] Taking the window size as W, the sliding step length as s, and the length of the original access address sequence as L as an example, based on the window size W and the sliding step length s, perform window sliding on the original access address sequence to divide the original access address sequence into several subsequences with a length of W; and the total number of subsequences is [(L - W) / s] + 1.

[0097] It should be noted that if [(L - W) / s] + 1 is not an integer, it means that when the original access address sequence is segmented by using the preset window size and sliding step length, the size of the remaining data during the last sliding is less than the preset window size, that is, the remaining data during the last sliding is not enough to fill the entire window. At this time, the window size during the last sliding can be dynamically adjusted, so that the size of the last window is allowed to be less than the preset window size, and the last subsequence of the original access address sequence is segmented based on the dynamically adjusted window size and the preset sliding step length. In addition, the sliding step length during the last sliding can also be dynamically adjusted, and the last subsequence of the original access address sequence is segmented based on the preset window size and the dynamically adjusted sliding step length. In this way, through the above two dynamic adjustment methods, the original access address sequence can be completely segmented, and the full utilization of the original access address sequence is realized.

[0098] Exemplarily, if the window size is 2, the sliding step is 2, and the length of the original access address sequence is 4, then the original access address sequence can be split into 2 subsequences. If the window size is 3, the sliding step is 2, and the length of the original access address sequence is 4, then the original access address sequence can still be split into 2 subsequences. The length of the first subsequence is 3, and the length of the second subsequence is 2 when the window size is dynamically adjusted, and is 3 when the sliding step is dynamically adjusted.

[0099] Considering that among the several split subsequences, there may be many subsequences with high similarity or even complete overlap, it is necessary to determine the similarity between any two subsequences among the several subsequences. When the similarity is greater than the preset similarity, duplicate removal processing is performed on any two subsequences to delete one of the two subsequences from any two subsequences, so as to obtain several subsequences after duplicate removal processing.

[0100] It should be noted that for the similarity between any two subsequences, the Tanimoto coefficient can be used for calculation. Moreover, the Tanimoto coefficient takes values between 0 and 1. 0 indicates that the two subsequences do not overlap at all (i.e., no similarity), and 1 indicates that the two subsequences are exactly the same (i.e., completely similar); that is, the Tanimoto coefficient is positively correlated with the similarity. Among them, the calculation formula for the similarity between any two subsequences is as follows:

[0101] ;

[0102] x represents one of the two subsequences, y represents the other of the two subsequences, and T(x, y) represents the similarity between the two subsequences.

[0103] Furthermore, in the process of training the initial probability distribution prediction model using the training set, the cross-entropy loss between the predicted output and the actual output of the probability distribution prediction model can be determined to adjust each model parameter in the probability distribution prediction model, so as to accelerate the convergence of the probability distribution prediction model and improve the prediction accuracy of the probability distribution prediction model.

[0104] Beneficial effects: By inputting the access address sequence into a pre-trained probability distribution prediction model, the present invention predicts the access probability distribution of each data block in the disk array next time, and based on the access probability distribution, loads the data of the data block most likely to be accessed next time in the disk array into the local cache, and eliminates the data of the data block least likely to be accessed next time from the local cache, so as to prefetch the data into the local cache. Thus, when the data block access address is obtained next time, the data of the corresponding data block can be directly accessed from the local cache. Compared with accessing the data of the corresponding data block from the disk array, by reducing the request and response interaction with the disk array, not only the performance impact on the disk array is reduced, but also the data access efficiency can be improved to achieve efficient data access in the face of a large number of concurrent requests. Further, by adopting an access address sequence including the current data block access address and the historical data block access address, and predicting the access probability distribution of each data block in the disk array next time based on the spatio-temporal fusion features of the access address sequence, the prediction accuracy of the access probability distribution is improved.

[0105] See Figure 2 and Figure 3 As shown, an embodiment of the present invention provides a data access method applied to a disk array card, including:

[0106] Obtain the current data block access address sent by the host, access the data of the actually to-be-accessed data block from the local cache using the current data block access address, and determine the current access address sequence based on the current data block access address and the historical data block access address.

[0107] Input the current access address sequence into a pre-trained probability distribution prediction model to determine the physical location information of the data block corresponding to each address in the current access address sequence in the disk array, add the corresponding physical location information to each address in the current access address sequence, then extract the time feature and space feature of the added current access address sequence, determine the fusion feature based on the extracted time feature and space feature, and predict the access probability distribution of each data block in the disk array next time according to the fusion feature.

[0108] Based on the actually to-be-accessed data block and the access probability distribution, determine the data block to be added and the data block to be eliminated from the disk array, load the data of the data block to be added into the local cache, and delete the data of the data block to be eliminated from the local cache, so as to prefetch the data of the data block most likely to be accessed next time into the local cache, and eliminate the data of the data block least likely to be accessed next time from the local cache, so that when the data block access address is obtained next time, the data of the corresponding data block can be directly accessed from the local cache.

[0109] Beneficial effects: By inputting the access address sequence into a pre-trained probability distribution prediction model, the present invention predicts the access probability distribution of each data block in the disk array next time, and based on the access probability distribution, loads the data of the data block most likely to be accessed next time in the disk array into the local cache, and eliminates the data of the data block least likely to be accessed next time from the local cache, so as to prefetch the data into the local cache. Thus, when the data block access address is obtained next time, the data of the corresponding data block can be directly accessed from the local cache. Compared with accessing the data of the corresponding data block from the disk array, by reducing the request and response interaction with the disk array, not only the performance impact on the disk array is reduced, but also the data access efficiency can be improved, so as to achieve efficient data access in the face of a large number of concurrent requests. Further, by adopting an access address sequence including the current data block access address and the historical data block access address, and predicting the access probability distribution of each data block in the disk array next time based on the spatio-temporal fusion feature of the access address sequence, the prediction accuracy of the access probability distribution is improved.

[0110] See Figure 4 As shown, an embodiment of the present invention provides a data access device, which is applied to a disk array card and includes:

[0111] An address sequence determination module 11, configured to determine a current access address sequence based on the current data block access address and the historical data block access address;

[0112] A probability distribution prediction module 12, configured to input the current access address sequence into a pre-trained probability distribution prediction model, add corresponding physical location information to each address in the current access address sequence, extract time features and space features from the added current access address sequence, determine a fusion feature based on the extracted time features and space features, and predict the access probability distribution of each data block in the disk array next time according to the fusion feature; the physical location information is the physical location information of the data block corresponding to each address in the disk array;

[0113] A data block determination module 13, configured to, after accessing the data of the actually to-be-accessed data block from the local cache using the current data block access address, determine a to-be-added data block and a to-be-eliminated data block from the disk array based on the actually to-be-accessed data block and the access probability distribution;

[0114] A cache update module 14, configured to load the data of the to-be-added data block into the local cache and delete the data of the to-be-eliminated data block from the local cache.

[0115] Since the embodiments of the device part correspond to the above embodiments, the embodiments of the device part are described with reference to the embodiments of the above method part and will not be elaborated here.

[0116] Advantageous effects: By inputting the access address sequence into a pre-trained probability distribution prediction model, the present invention predicts the access probability distribution of each data block in the disk array next time, and loads the data of the data block most likely to be accessed next time in the disk array into the local cache, and eliminates the data of the data block least likely to be accessed next time in the disk array from the local cache, so as to prefetch the data into the local cache. Thus, when the data block access address is obtained next time, the data of the corresponding data block can be directly accessed from the local cache. Compared with accessing the data of the corresponding data block from the disk array, by reducing the request and response interaction with the disk array, not only the performance impact on the disk array is reduced, but also the data access efficiency can be improved to achieve efficient data access in the face of a large number of concurrent requests. Further, by adopting an access address sequence including the current data block access address and the historical data block access address, and predicting the access probability distribution of each data block in the disk array next time based on the spatio-temporal fusion feature of the access address sequence, the prediction accuracy of the access probability distribution is improved.

[0117] Further, an embodiment of the present application also discloses an electronic device. Figure 5 It is a structural diagram of an electronic device shown according to an exemplary embodiment, and the content in the figure cannot be considered as any limitation to the scope of use of the present application. The electronic device may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. Among them, the memory 22 is used to store a computer program, and the computer program is loaded and executed by the processor 21 to implement the relevant steps in the data access method disclosed in any of the foregoing embodiments. In addition, the electronic device in this embodiment may specifically be an electronic computer.

[0118] In this embodiment, the power supply 23 is used to provide a working voltage for each hardware device on the electronic device; the communication interface 24 can create a data transmission channel between the electronic device and external devices, and the communication protocol it follows is any communication protocol applicable to the technical solution of the present application, and no specific limitation is made here; the input / output interface 25 is used to obtain external input data or output data to the outside, and its specific interface type can be selected according to specific application needs, and no specific limitation is made here.

[0119] In addition, the memory 22, as a carrier for storing resources, can be a read-only memory, a random access memory, a magnetic disk, an optical disc, etc. The resources stored thereon can include an operating system 221, a computer program 222, etc. The storage method can be transient storage or permanent storage.

[0120] Among them, the operating system 221 is used to manage and control each hardware device and the computer program 222 on the electronic device, and it can be Windows Server, Netware, Unix, Linux, etc. In addition to the computer program that can be used to complete the data access method executed by the electronic device disclosed in any of the foregoing embodiments, the computer program 222 can further include a computer program that can be used to complete other specific tasks.

[0121] Furthermore, the present application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the foregoing disclosed data access method is implemented. For the specific steps of this method, reference can be made to the corresponding content disclosed in the foregoing embodiments, and details will not be repeated here.

[0122] Furthermore, the present application also discloses a computer program product, including a computer program / instructions; wherein, when the computer program / instructions are executed by a processor, the foregoing disclosed data access method is implemented. For the specific steps of this method, reference can be made to the corresponding content disclosed in the foregoing embodiments, and details will not be repeated here.

[0123] In this specification, the various embodiments are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. For the same or similar parts among the various embodiments, reference can be made to each other. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and reference can be made to the description in the method part for related parts.

[0124] Those skilled in the art can further realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the components and steps of the examples have been generally described according to their functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0125] The steps of the methods or algorithms described in connection with the embodiments disclosed in this specification may be implemented directly in hardware, in software modules executed by a processor, or in a combination thereof. The software modules may be placed in a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium well known in the art.

[0126] Finally, it should also be noted that in this specification, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variation thereof is intended to cover a non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or device comprising the element.

[0127] The technical solutions provided in this application have been introduced in detail above. Specific examples are used in this specification to elaborate on the principles and implementation manners of this application. The description of the above embodiments is only used to help understand the method and its core idea of this application. At the same time, for those of ordinary skill in the art, based on the idea of this application, there will be changes in the specific implementation manners and application scopes. In summary, the content of this specification should not be construed as a limitation to this application.

Claims

1. A data access method, characterized in that, Applied to a disk array card, including: Determine the current access address sequence based on the current data block access address and the historical data block access address; Input the current access address sequence into a pre-trained probability distribution prediction model to add corresponding physical location information to each address in the current access address sequence, extract temporal features and spatial features from the current access address sequence after adding, determine the fusion feature based on the extracted temporal features and spatial features, and predict the access probability distribution of each data block in the disk array according to the fusion feature; the physical location information is the physical location information of the data block corresponding to each address in the disk array; After accessing the actual data block to be accessed from the local cache using the current data block access address, determine the data block to be added and the data block to be eliminated from the disk array based on the actual data block to be accessed and the access probability distribution; Load the data of the data block to be added into the local cache and delete the data of the data block to be eliminated from the local cache.

2. The data access method according to claim 1, wherein The extraction of temporal features and spatial features from the current access address sequence after adding includes: Extract temporal features from each address based on the timestamp information carried by each address in the current access address sequence after adding to obtain the extracted temporal features; Extract spatial features from each address in the current access address sequence after adding using a convolutional layer with a preset dimension to obtain the extracted spatial features.

3. The data access method according to claim 1, characterized in that The determination of the fusion feature based on the extracted temporal features and spatial features includes: Determine the query vector, key vector, and value vector based on the extracted temporal features, and determine the temporal attention weight matrix according to the query vector and the key vector; Construct a spatial association matrix based on the physical location information of each address in the current access address sequence after adding; Determine the spatial attention weight matrix according to the spatial association matrix and the extracted spatial features; Perform weighted fusion on the value vector, the temporal attention weight matrix, the extracted spatial features, and the spatial attention weight matrix to determine the fusion feature.

4. The data access method according to claim 3, wherein The construction of the spatial association matrix based on the physical location information of each address in the current access address sequence after adding includes: Determine the spatial proximity of the data blocks corresponding to any two addresses in the disk array based on the physical location information of any two addresses in the current access address sequence after adding, and construct a spatial association matrix according to the spatial proximity.

5. The data access method according to claim 1, wherein The determination of the data block to be added and the data block to be eliminated from the disk array based on the actual data block to be accessed and the access probability distribution includes: Determine the first data block whose data is not loaded into the local cache from the disk array, and determine the second data block whose data is loaded into the local cache from the disk array; Using the access probability distribution, determine the data blocks in the first data block whose access probability is greater than a first preset probability to obtain first target data blocks; Using the access probability distribution, determine the data blocks in the first data block whose access probability is not greater than the first preset probability and is greater than a second preset probability to obtain first initial data blocks; Based on the physical location information of the first initial data block and the actual data block to be accessed in the disk array, determine the spatial proximity between the first initial data block and the actual data block to be accessed, and when the spatial proximity between the first initial data block and the actual data block to be accessed is greater than a first preset proximity, determine the first initial data block as a second target data block; Determine the data block to be added according to the first target data block and the second target data block; Using the access probability distribution, determine the data blocks in the second data block whose access probability is less than a third preset probability to obtain second initial data blocks; Based on the physical location information of the second initial data block and the actual data block to be accessed in the disk array, determine the spatial proximity between the second initial data block and the actual data block to be accessed, and when the spatial proximity between the second initial data block and the actual data block to be accessed is less than a second preset proximity, determine the second initial data block as the data block to be eliminated.

6. The data access method according to claim 1, wherein Before inputting the current access address sequence into the pre-trained probability distribution prediction model, it further includes: Obtain the data block access addresses within a preset time period to obtain an original access address sequence; Based on the window size and the sliding step, perform window sliding on the original access address sequence to divide the original access address sequence into several subsequences; Use the last address in each subsequence as the output, and use the other addresses in each subsequence except the last address as the input to construct a training set; Use the training set to train an initial probability distribution prediction model to obtain a pre-trained probability distribution prediction model; the initial probability distribution prediction model is a model constructed based on a long short-term memory network.

7. The data access method according to claim 6, wherein Before using the last address in each subsequence as the output, it further includes: Determine the similarity between any two of the several subsequences; When the similarity is greater than a preset similarity, perform deduplication processing on the any two subsequences to delete one of the any two subsequences from the any two subsequences to obtain the several subsequences after deduplication processing.

8. A data access device, characterized in that, Applied to a disk array card, it includes: An address sequence determination module, configured to determine a current access address sequence based on a current data block access address and a historical data block access address; A probability distribution prediction module, configured to input the current access address sequence into a pre-trained probability distribution prediction model, so as to add corresponding physical location information to each address in the current access address sequence, extract time features and spatial features from the current access address sequence after the addition, determine a fusion feature based on the extracted time features and spatial features, and predict the access probability distribution of each data block in the disk array next time according to the fusion feature; the physical location information is the physical location information of the data block corresponding to each address in the disk array; A data block determination module, configured to, after accessing the actual data block to be accessed from the local cache using the current data block access address, determine a data block to be added and a data block to be eliminated from the disk array based on the actual data block to be accessed and the access probability distribution; A cache update module, configured to load the data of the data block to be added into the local cache, and delete the data of the data block to be eliminated from the local cache.

9. An electronic device, characterized in that, Comprising: A memory, configured to store a computer program; A processor, configured to execute the computer program to implement the steps of the data access method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the data access method according to any one of claims 1 to 7 are implemented.

Images