A data storage allocation method and apparatus

Abstract

The application provides a data storage allocation method and device. The method comprises: receiving a write request, the write request comprising to-be-written data; obtaining a request feature of the to-be-written data and state features of a plurality of storage units; inputting the request feature and the state feature of each storage unit into a scoring model to obtain a score of each storage unit output by the scoring model; wherein the scoring model determines a plurality of feature weights based on the request feature and the state feature of each storage unit, and determines the score of each storage unit based on the plurality of feature weights and the state feature of each storage unit; and selecting a target storage unit from the plurality of storage units according to the score, and writing the to-be-written data into the target storage unit. The application realizes adaptive storage allocation.

Description

Technical Field

[0001] This application relates to the field of computer storage technology, and in particular to a data storage and allocation method and apparatus. Background Technology

[0002] In the field of data storage, especially in storage media such as NAND flash memory, the write request allocation strategy directly affects the overall performance and lifespan of the storage media. Traditional write allocation methods mostly adopt static rules, such as simple polling mechanisms or wear leveling algorithms based on fixed thresholds. These methods often cannot make flexible and adaptive adjustments when dealing with dynamically changing workloads, leading to premature aging of local cells or unstable response speeds in the storage media. Summary of the Invention

[0003] This application provides a data storage and allocation method and apparatus to at least solve the above-mentioned technical problems existing in the prior art.

[0004] According to a first aspect of this application, a data storage allocation method is provided, the method comprising:

[0005] Receive a write request, the write request including data to be written; Obtain the request characteristics of the data to be written and the status characteristics of multiple storage units; The request features and the state features of each storage unit are input into the scoring model to obtain the score of each storage unit output by the scoring model; wherein, the scoring model determines multiple feature weights based on the request features and the state features of each storage unit, and determines the score of each storage unit based on the multiple feature weights and the state features of each storage unit. Based on the score, a target storage unit is selected from the plurality of storage units, and the data to be written is written to the target storage unit.

[0006] In one possible implementation, the request features include at least one of the following: logical address, data type, access frequency, and request priority; The state characteristics include at least one of the following: number of erase / write cycles, bit error rate, data dwell time, and access delay.

[0007] In one possible implementation, the scoring model is trained through the following operations: Acquire historical write data, which includes historical request characteristics, historical status characteristics, and performance parameters of each storage unit after each write. Based on the performance parameters, determine the corresponding storage unit for each write operation, and use the corresponding storage unit as the training label; The historical request features and historical state features are used as training inputs, and the training labels are used as training targets to train the initial model, thereby obtaining the scoring model.

[0008] In one possible implementation, the performance parameters include at least one of the following: write latency, erase / write increment, and bit error rate increment.

[0009] In one embodiment, the scoring model includes a first branch network and a second branch network; The scoring model determines multiple feature weights based on the request features and the state features of each storage unit, and determines a score for each storage unit based on the multiple feature weights and the state features of each storage unit, including: Based on the request characteristics and the state characteristics of each storage unit, the spatial topology characteristics of the storage unit are extracted using the first branch network; Based on the request features and the state features of each storage unit, the timing access features corresponding to the request features are extracted using the second branch network; The spatial topology features and the temporal access features are fused to obtain the fused features; Multiple feature weights are determined based on the fusion features; The score for each storage unit is determined based on the weights of the multiple features and the state characteristics of each storage unit.

[0010] In one possible implementation, the plurality of feature weights include a first weight corresponding to the number of erase / write cycles, a second weight corresponding to the access frequency, and a third weight corresponding to the bit error rate; accordingly, The scoring model determines the score of each storage unit based on the weights of the multiple features and the state characteristics of each storage unit, including: The percentage of erase / write cycles for each storage cell is determined based on the number of erase / write cycles in the state characteristics of each storage cell and the maximum allowed number of erase / write cycles for each storage cell. The score for each storage cell is determined based on the percentage of erase / write cycles for each storage cell and the first weight, the access frequency in the state characteristics of each storage cell and the second weight, and the bit error rate in the state characteristics of each storage cell and the third weight.

[0011] In one possible implementation, the method further includes: Under preset conditions, the scoring model is fine-tuned through backpropagation.

[0012] In one possible implementation, the preset conditions include at least one of the following: The cumulative number of erase / write cycles has reached the preset threshold. The time elapsed since the last fine-tuning of the scoring model has reached a preset time threshold; The average bit error rate of multiple storage units has increased beyond a preset proportional threshold.

[0013] In one possible implementation, the fine-tuning of the scoring model via backpropagation includes: Extract relevant data from multiple historical writes in the circular buffer as training data; The target score is determined based on the performance parameters after each historical write in the training data. Based on the scoring model and the relevant data for each historical write, determine the predicted score for each historical write. Calculate the loss function based on the difference between the predicted score and the target score for each historical write; The scoring model is updated via backpropagation based on the loss function.

[0014] According to a second aspect of this application, a data storage allocation apparatus is provided, the apparatus comprising: A receiving module is used to receive a write request, wherein the write request includes data to be written; The acquisition module is used to acquire the request characteristics of the data to be written and the status characteristics of multiple storage units; The scoring module is used to input the request features and the state features of each storage unit into the scoring model to obtain the score of each storage unit output by the scoring model; wherein, the scoring model determines multiple feature weights based on the request features and the state features of each storage unit, and determines the score of each storage unit based on the multiple feature weights and the state features of each storage unit. The write module is used to select a target storage unit from the plurality of storage units according to the score, and write the data to be written into the target storage unit.

[0015] According to a third aspect of this application, an electronic device is provided, comprising: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method described in this application.

[0016] According to a fourth aspect of this application, a non-transitory computer-readable storage medium is provided storing computer instructions for causing the computer to perform the methods described in this application.

[0017] This application discloses a data storage allocation method and apparatus that receives a write request, the write request including data to be written; acquires request features of the data to be written and state features of multiple storage units; inputs the request features and state features of each storage unit into a scoring model to obtain a score for each storage unit output by the scoring model; wherein the scoring model determines multiple feature weights based on the request features and the state features of each storage unit, and determines a score for each storage unit based on the multiple feature weights and the state features of each storage unit; selects a target storage unit from the multiple storage units according to the score, and writes the data to be written to the target storage unit. This achieves flexible and adaptive adjustment to dynamic workloads, ensuring that write requests with different characteristics can be allocated to the most suitable storage unit. It overcomes the limitations of traditional static rules or fixed threshold methods, effectively solving the technical problem that traditional methods cannot make flexible and adaptive adjustments when handling dynamically changing workloads, leading to premature aging of local units or unstable response speeds in the storage medium. This achieves the effect of extending the overall lifespan of the storage medium and ensuring response stability.

[0018] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this application, nor is it intended to limit the scope of this application. Other features of this application will become readily apparent from the following description. Attached Figure Description

[0019] The above and other objects, features, and advantages of exemplary embodiments of this application will become readily apparent from the following detailed description taken in conjunction with the accompanying drawings. Several embodiments of this application are illustrated in the drawings by way of example and not limitation, in which: In the accompanying drawings, the same or corresponding reference numerals indicate the same or corresponding parts.

[0020] Figure 1 A schematic diagram illustrating the implementation flow of the data storage allocation method provided in an embodiment of this application is shown; Figure 2 This paper illustrates an exemplary structural diagram of the scoring model provided in an embodiment of this application; Figure 3 This illustration shows a schematic diagram of the implementation process of the model fine-tuning operation of the data storage allocation method provided in the embodiments of this application; Figure 4 A schematic diagram of the composition structure of the data storage and distribution device provided in an embodiment of this application is shown; Figure 5 A schematic diagram of the composition structure of an electronic device according to an embodiment of this application is shown. Detailed Implementation

[0021] To make the objectives, features, and advantages of this application more apparent and understandable, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0022] First, the application scenarios of the embodiments of this application are described. With the explosive growth of data volume, NAND flash memory has become the mainstream storage medium due to its high density and low power consumption. However, traditional NAND data allocation strategies have the following problems: static allocation uses fixed rules (such as polling and static wear leveling), which cannot adapt to dynamic workloads; local optima ignore data access popularity and lifecycle correlation, resulting in uneven erase / write operations; in terms of latency sensitivity, high-priority data (such as metadata) fails to respond quickly, affecting the Quality of Service (QoS); in terms of lifespan loss, frequent writing to hot areas accelerates cell aging and reduces the overall lifespan. Related improvement schemes, such as rule-based dynamic wear leveling and greedy algorithms, still rely on human experience and are difficult to handle multi-dimensional nonlinear relationships. Some studies have attempted to use machine learning to predict wear, but they only focus on a single dimension (such as the number of erase / write cycles) and do not integrate key factors such as data features and access patterns, resulting in limited generalization ability.

[0023] Therefore, to solve the above-mentioned technical problems, this application provides a data storage allocation method, apparatus, device and storage medium. By dynamically fusing multi-dimensional features such as data type, access frequency, erase / write count and cell health through deep learning, the optimal physical block allocation strategy is generated in real time to achieve: balanced lifetime and dynamic distribution of write hotspots; performance optimization, allocating high-priority data to low-latency cells; and improved reliability, avoiding high-risk blocks.

[0024] Figure 1 A schematic diagram illustrating the implementation flow of the data storage allocation method provided in an embodiment of this application is shown.

[0025] refer to Figure 1 This application provides a data storage allocation method, which includes: Operation 101: Receive a write request, which includes the data to be written.

[0026] The data storage allocation method provided in this application can be executed by a storage controller, a storage management device, or any electronic device with storage allocation processing capabilities. It is primarily used to write data requiring persistent storage to a storage medium. The storage medium includes, but is not limited to, non-volatile storage media such as NAND flash memory, NOR flash memory, storage arrays in solid-state drives (SSDs), storage-class memory (SCM), or other physical carriers with data storage capabilities.

[0027] Specifically, the executing entity receives a write request initiated by an external device (such as a host system, application, or upper-layer controller). This write request represents an instruction to write data to the storage medium, informing the executing entity that a data write operation needs to be performed. The write request includes at least the data to be written, which represents the core data that needs to be actually stored in the storage medium. The data to be written can be of any type, including but not limited to user data, system data, metadata, log data, etc. It should be noted that in this embodiment, the storage medium is preferably NAND flash memory, but the technical solution of this application is not limited to this; other types of non-volatile storage media are also applicable to the method of this application.

[0028] Operation 102: Obtain the request characteristics of the data to be written and the status characteristics of multiple storage units.

[0029] In response to a received write request, the system acquires the request characteristics corresponding to the data to be written, and simultaneously acquires the status characteristics of multiple storage units currently available for data storage. A storage unit can be a physical block, page, or other independently allocable smallest unit of storage in the storage medium, depending on the organization of the storage medium. Status characteristics can be obtained by the storage unit itself or by the storage controller through monitoring and feedback mechanisms, such as health status information provided by the NAND flash memory chip.

[0030] In one embodiment of this application, the request features include at least one of the following: logical address, data type, access frequency, and request priority; the status features include at least one of the following: number of erase / write cycles, bit error rate, data dwell time, and access delay.

[0031] Logical address refers to the address used to identify the data storage location from the perspective of the host or file system, such as the Logical Block Address (LBA). Logical addresses can be used to map to physical storage locations and also to identify data access patterns or relationships. Data type refers to the attribute category of the data to be written. For example, data can be categorized by access frequency (cold data, low access frequency) and hot data (high access frequency), or by importance and purpose (metadata, user data, log data, etc.). Data type reflects the data's lifecycle and access characteristics. Request priority refers to the processing level of this write request, used to distinguish the importance and latency sensitivity of the write operation. For example, it reflects the quality of service (QoS) level of the write request; high-priority requests typically need to be allocated to higher-performance storage units to ensure a fast response.

[0032] Write / erase cycles refer to the number of program / erase (PE) operations a memory cell has undergone, and are a key indicator of the wear and tear of the memory cell. The closer the write / erase cycles are to the cell's maximum allowable value, the closer the cell is to the end of its lifespan. Bit error rate (RBER) refers to the raw bit error rate (RBER) that occurs when reading data from a memory cell, reflecting the cell's data retention capability and health status. An increased RBER usually indicates a decrease in cell reliability. Retention time (RET) refers to the time elapsed after data is written to the memory cell. A longer retention time may lead to an increased RBER due to charge leakage, especially for NAND flash memory. Access latency refers to the average time required to perform a read or write operation on a memory cell, usually measured in microseconds (μs). Access latency reflects the performance status of the memory cell and can be affected by factors such as cell wear and interference.

[0033] Operation 103 inputs the request features and the state features of each storage unit into the scoring model to obtain the score of each storage unit output by the scoring model; wherein, the scoring model determines multiple feature weights based on the request features and the state features of each storage unit, and determines the score of each storage unit based on the multiple feature weights and the state features of each storage unit.

[0034] To achieve accurate selection of storage units, this application embodiment pre-configures a trained scoring model. This model can quantitatively evaluate the suitability of each storage unit based on data write requirements and storage unit status. Specifically, request features and the status features of each storage unit are input into the scoring model. Upon receiving the input, the scoring model adaptively determines multiple feature weights based on the correlation between the request features and the status features of each storage unit. These feature weights reflect the importance of different status features (such as erase / write cycles, bit error rate, access latency, etc.) for selecting the target storage unit under the current write request. For example, when the write request has high priority, the model may assign a higher weight to access latency, tending to select storage units with lower latency; when the data to be written is cold data, the model may assign a higher weight to erase / write cycles, prioritizing units with less wear. Subsequently, the model uses these feature weights and the status features of each storage unit to calculate and obtain a score for each storage unit.

[0035] Operation 104: Select the target storage unit from multiple storage units based on the score, and write the data to be written to the target storage unit.

[0036] After obtaining the scores of each storage unit, a target storage unit is selected based on the scores. For example, the storage unit with the highest score can be selected as the target storage unit, or a storage unit with a score exceeding a preset threshold can be selected, or one of the top N units can be selected according to the score ranking. In this embodiment, the storage unit with the highest score is preferably selected to ensure that the current write request obtains the optimal allocation result. After the target storage unit is determined, the data to be written is written to that unit, completing the data storage allocation process.

[0037] Thus, this embodiment dynamically calculates the fitness score of each storage unit based on the real-time characteristics of the write request and the current state of the storage unit, and selects the optimal unit for writing. This achieves flexible and adaptive adjustment to dynamic workloads, ensuring that write requests with different characteristics can be allocated to the most suitable storage unit. It overcomes the limitations of traditional static rules or fixed threshold methods, effectively solving the technical problem that traditional methods cannot make flexible and adaptive adjustments when handling dynamically changing workloads, leading to premature aging of local units or unstable response speeds in the storage medium. This achieves the effect of extending the overall lifespan of the storage medium and ensuring response stability.

[0038] In one embodiment of this application, the scoring model is trained through the following operations: acquiring historical write data, which includes historical request features, historical state features, and performance parameters of each storage unit after each write; determining the corresponding storage unit for each write based on the performance parameters, and using the corresponding storage unit as the training label; training the initial model with the historical request features and historical state features as training inputs and the training label as the training target to obtain the scoring model.

[0039] In one embodiment of this application, the performance parameters include at least one of the following: write latency, erase / write increment, and bit error rate increment.

[0040] Specifically, the training process of the scoring model is as follows: 1) Collect historical write data generated by the storage medium during actual operation. This data includes historical request characteristics (such as logical address, data type, access frequency, request priority, etc.) corresponding to each write request, historical state characteristics of each storage unit during the write (such as erase / write count, bit error rate, data dwell time, access latency, etc.), and performance parameters of each storage unit after each write operation. Performance parameters are used to measure the actual effect of the write operation, such as write latency, erase / write count increment, bit error rate increment, etc., which can reflect the impact of the write on the storage unit and the unit's performance after the write.

[0041] 2) Clean and normalize the collected historical write data. First, clean the data to remove outliers and missing values. For example, storage cell data with abnormally high bit error rates (e.g., RBER > 1e-3) or write cycles exceeding the maximum allowable value (e.g., PE > 100k) can be considered outliers and removed to avoid interfering with model training. For samples with missing values, mean imputation, interpolation, or direct deletion can be used. Subsequently, normalize the cleaned data to scale each feature value to a uniform numerical range (e.g., [0, 1] or [-1, 1]) to eliminate the influence of different units on model training and improve training efficiency and accuracy.

[0042] 3) Based on the performance parameters after each write operation, determine the appropriate storage unit for that write operation, i.e., the theoretically optimal write target. For example, for a single write operation, compare the performance of all candidate storage units after the write (e.g., lowest write latency, smallest increment of erase / write cycles, smallest increment of bit error rate, etc.), select the best-performing unit as the appropriate storage unit for that write operation, and use it as the training label. Alternatively, a comprehensive score can be constructed by combining multiple performance metrics, and the unit with the highest score can be selected as the label. In this way, a supervision signal is generated for each historical write data, indicating which unit is the optimal choice given the request characteristics and storage unit status at that time.

[0043] 4) Use the historical request features and corresponding historical state features of each storage unit in each historical write as training input, and the corresponding storage unit for that write as the training target (label). Supervised learning training is then performed on the initial model (e.g., deep neural network, decision tree). During training, the model predicts the score of each storage unit based on the input features and measures the difference between the prediction result and the label using a loss function (e.g., cross-entropy loss, mean squared error). The backpropagation algorithm continuously adjusts the model's internal parameters (including feature weights), allowing the model to gradually learn the complex mapping relationship between input features and the optimal storage unit. After multiple rounds of iterative training, the model converges, resulting in the trained scoring model. The input data can be a multi-dimensional feature vector or matrix, and the label can be the identifier of the storage unit or an encoded vector.

[0044] Through the above training process, the scoring model can effectively integrate multi-source features and autonomously learn the importance weights of different state features under different scenarios, thereby generating an accurate storage unit score for each write request in subsequent practical applications.

[0045] In one embodiment of this application, the scoring model includes a first branch network and a second branch network; the above operation 103, in which the scoring model determines multiple feature weights based on request features and state features of each storage unit, and determines the score of each storage unit based on the multiple feature weights and state features of each storage unit, includes: extracting spatial topology features of the storage unit using the first branch network based on request features and state features of each storage unit; extracting temporal access features corresponding to the request features using the second branch network based on request features and state features of each storage unit; fusing the spatial topology features and temporal access features to obtain fused features; determining multiple feature weights based on the fused features; and determining the score of each storage unit based on the multiple feature weights and state features of each storage unit.

[0046] Specifically, the scoring model in this embodiment adopts a dual-branch network structure, where the two branches share the same input data, namely, the feature vector or feature matrix composed of the request features of the data to be written and the state features of all current candidate storage units. This input data includes multi-source information in both the request and storage unit dimensions, such as logical address, data type, access frequency, and request priority in the request features, as well as the number of erase / write operations, bit error rate, data dwell time, and access latency for each storage unit. As an example, the input data can be a 7-dimensional feature vector, whose dimensions may include, but are not limited to, logical address, data type encoding, access frequency, average number of erase / write operations, and recent bit error rate. However, this application does not limit the specific dimensions and composition of the input features.

[0047] The first branch network can be a Convolutional Neural Network (CNN) or a variant thereof, primarily used to extract spatial topological features between memory cells from input data. Through its network structure and the characteristics of convolutional operations, the first branch network can focus on the spatial relationships between memory cell state features. For example, when input data is organized according to the physical location of memory cells (such as an array composed of word lines and bit lines), the first branch network, by sliding the convolutional kernel in the spatial dimension, can capture the similarity between adjacent memory cells, hotspots of wear distribution, and the mutual influence relationships between physically adjacent cells, thereby extracting topological features reflecting the spatial distribution characteristics of memory cells.

[0048] The second branch network can be a Recurrent Neural Network (RNN), a Long Short-Term Memory (LSTM) network, or a variant thereof. It is primarily used to extract temporal access features implicit in request characteristics from input data. Through its recurrent structure and memory units, the second branch network can focus on the evolution of request characteristics over time. For example, for multiple consecutive write requests, the second branch network can learn whether there is periodicity in the access patterns of the same logical address, and whether changes in request priority and data type at different times reflect the dynamic evolution of the workload, thereby extracting temporal access features that reflect request characteristics.

[0049] After obtaining the spatial topology features from the first branch network output and the temporal access features from the second branch network output, these two types of features are fused. The fusion method can be feature concatenation, which joins the two feature vectors end-to-end to form a new high-dimensional feature vector; it can also be feature addition or weighted summation; or it can be a non-linear combination through a fully connected layer. The fused features simultaneously contain the physical spatial information of the storage units and the temporal dynamic information of the requests, enabling a more comprehensive characterization of the multi-dimensional relationships in the current write context. For example, the fused features may reflect that storage units within a certain physical region have recently been frequently accessed by high-priority data, or that write requests within a certain time period are correlated with specific types of storage units.

[0050] To provide a more intuitive explanation of the structure of this application's scoring model, a specific example is provided below. (Reference) Figure 2 , Figure 2This diagram illustrates an exemplary structural schematic of the scoring model provided in this application. In this example, the input feature is a 7-dimensional vector, which may include the request feature and several state features of the storage unit (e.g., logical address encoding, data type encoding, access frequency, erase / write count, bit error rate, data dwell time, access latency). The input is simultaneously fed into two branch networks: the first branch network is a CNN branch, which contains at least one convolutional layer for extracting the spatial topological features of the storage unit; the second branch network is an LSTM branch for extracting the temporal access features of the request feature. In the CNN branch, a pooling layer can be connected after the convolutional layer to reduce the dimensionality of the features and retain the main spatial information. To prevent overfitting, a Dropout layer (e.g., a dropout probability of 0.2) can be set after the pooling layer or before the branch output. The outputs of the two branch networks are fused in a feature fusion layer to obtain a fused feature. This fused feature will be used for subsequent feature weight generation and score calculation (not shown in the figure). It should be noted that... Figure 3 The specific network parameters and structures shown (such as 7-dimensional input, Dropout ratio, etc.) are merely illustrative examples and do not constitute a limitation on the scope of protection of this application.

[0051] Based on the fused features, the scoring model further generates multiple feature weights. These feature weights are not pre-set fixed values, but are dynamically calculated based on the current input request features and storage unit state features. Specifically, the fused features are input into one or more fully connected layers (or other types of weight generation networks), and after nonlinear transformation, a set of weight coefficients are output. Each weight coefficient corresponds to a certain dimension in the storage unit state features, such as the number of erase / write cycles, access frequency, etc. These weights reflect the importance of different state features for selecting the target storage unit under the current write request and the overall condition of the current storage unit.

[0052] After obtaining the dynamically generated feature weights, the scoring model combines these weights with the state features of each storage unit to calculate a comprehensive score for each storage unit. A higher score indicates that the storage unit is more suitable for writing the data to be written. The calculation method can be a linear weighted sum or a more complex non-linear combination. The final output score can be a numerical value or a normalized probability value, such as a probability matrix representing the suitability probability of each storage unit: Block#1: 0.85, Block#2: 0.12, Block#3: 0.03, used for subsequent target storage unit selection. For example, for the three candidate storage units, their scores are 0.85, 0.12, and 0.03 respectively, indicating that Block#1 is the optimal write target.

[0053] Thus, this embodiment of the application uses a dual-branch network structure to extract spatial topology features and temporal access features based on the same input data, and then merges the two to dynamically generate feature weights, thereby calculating the score of each storage unit. This mechanism enables the scoring model to simultaneously consider physical spatial distribution and temporal evolution, more comprehensively capturing the complex relationship between write requests and storage units, thereby achieving more accurate adaptive allocation under dynamically changing workloads.

[0054] In one embodiment of this application, the multiple feature weights include a first weight corresponding to the number of erase / write operations, a second weight corresponding to the access frequency, and a third weight corresponding to the bit error rate. The above operation 204 determines the score of each storage unit based on the multiple feature weights and the state characteristics of each storage unit, including: determining the percentage of erase / write operations for each storage unit based on the number of erase / write operations in the state characteristics of each storage unit and the maximum allowed number of erase / write operations for each storage unit; and determining the score of each storage unit based on the percentage of erase / write operations for each storage unit and the first weight, the access frequency in the state characteristics of each storage unit and the second weight, and the bit error rate in the state characteristics of each storage unit and the third weight.

[0055] Specifically, after obtaining the dynamically generated feature weights, the scoring model needs to combine these weights with the state characteristics of each memory cell to calculate a comprehensive score for each memory cell. In this embodiment, the feature weights specifically include three dimensions: the first weight corresponds to the number of erase / write operations, the second weight corresponds to the access frequency, and the third weight corresponds to the bit error rate (BER). Based on this, for each candidate memory cell, the erase / write ratio (PE_ratio) is first calculated based on the current erase / write operations in its state characteristics and the maximum allowed erase / write operations for that memory cell. For example, PE_ratio = current erase / write operations / maximum allowed erase / write operations. This percentage reflects the wear and tear of the memory cell; a higher percentage indicates that the cell is closer to the end of its lifespan. Subsequently, the score of the memory cell is determined by combining this erase / write ratio with the first weight, the access frequency of the memory cell with the second weight, and the BER of the memory cell with the third weight. The specific calculation method can be linear weighted summation, for example, calculated using the following scoring function: S = α × (1 - PE_ratio) + β × Access_freq + γ × (1 - RBER). Wherein, PE_ratio is the ratio of the current number of erase / write operations to the maximum allowed number of erase / write operations for the storage unit, i.e., the percentage of erase / write operations; Access_freq is the access frequency of the storage unit; RBER is the raw bit error rate of the storage unit; α, β, and γ are the feature weights corresponding to the number of erase / write operations, access frequency, and bit error rate, respectively, i.e., the first weight, the second weight, and the third weight mentioned above.

[0056] In one embodiment of this application, the scoring model is further fine-tuned through backpropagation when preset conditions are met.

[0057] Specifically, during actual deployment and application, the scoring model may drift over time and with changes in workload, resulting in variations in the state characteristics of storage units and the patterns of write requests. To maintain the accuracy and adaptability of the scoring model, this application introduces an online fine-tuning mechanism. When certain preset conditions are met, the scoring model is updated using recently generated write data through backpropagation, enabling the model to continuously learn new data distributions and load characteristics.

[0058] The preset conditions can be trigger thresholds set based on various factors such as time, events, or performance metrics. For example, a fixed time period (e.g., every 24 hours) can be set to trigger fine-tuning; a certain number of cumulative write operations (e.g., 1000 writes) can be set to trigger it; or health metrics such as the average bit error rate or erase / write cycles of the storage unit can be monitored, and fine-tuning can be triggered when the changes in these metrics exceed a preset proportion. By setting reasonable trigger conditions, fine-tuning can be initiated in a timely manner when model performance deteriorates or the environment changes, avoiding model aging that leads to deterioration in allocation performance.

[0059] When preset conditions are met, recently generated write data (such as request characteristics, status characteristics, and post-write performance parameters of the most recent write operations) is used as training samples. These new training samples are then used to train the scoring model via backpropagation. This involves calculating the model output through forward propagation, comparing it with the expected output (such as the target score or target storage unit determined based on actual write performance) to calculate the loss function, and then using the backpropagation algorithm to update the model's internal weight parameters. In this way, the model can gradually adapt to new data distributions and load patterns while maintaining its scoring accuracy.

[0060] In one embodiment of this application, the preset conditions include at least one of the following: the cumulative number of erase / write operations reaches a preset number threshold; the time elapsed since the last fine-tuning of the scoring model reaches a preset time threshold; and the average bit error rate of multiple storage units increases beyond a preset proportion threshold.

[0061] Specifically, the number of erase / write cycles is an important indicator for measuring the wear and tear of storage cells. When the cumulative number of erase / write cycles for the entire storage medium or a specific area reaches a certain threshold, it indicates that the storage cell has undergone a certain degree of aging, and its state characteristic distribution may change significantly. Triggering fine-tuning at this point allows the model to learn the behavioral characteristics of the worn-out cells, avoiding allocation decision biases caused by cell aging.

[0062] Workload patterns may change slowly over time, such as different access pressures during the day and night, or a shift in data popularity distribution after long-term operation. By setting time periods to trigger fine-tuning, the model can be periodically adapted to these gradual changes, preventing it from becoming outdated. The duration threshold can be set according to business characteristics, such as 24 hours, a week, etc.

[0063] Bit error rate (BER) directly reflects the reliability and data retention capability of storage units. A significant increase in the overall BER may indicate aging of the storage medium, increased interference, or environmental changes (such as rising temperature). In such cases, fine-tuning the scoring model can make it place greater emphasis on reliability indicators such as BER, proactively avoid high-risk units, and improve the reliability of data writing.

[0064] Figure 3 This illustration shows a schematic diagram of the implementation process of the model fine-tuning operation of the data storage allocation method provided in the embodiments of this application.

[0065] In one embodiment of this application, fine-tuning the scoring model through backpropagation includes: extracting relevant data corresponding to multiple historical writes from a circular buffer as training data; determining a target score based on the performance parameters after each historical write in the training data; determining a predicted score corresponding to each historical write based on the scoring model and the relevant data corresponding to each historical write; calculating a loss function based on the difference between the predicted score and the target score for each historical write; and updating the scoring model through backpropagation based on the loss function.

[0066] Specifically, after the scoring model is fine-tuned due to preset conditions, this embodiment of the application uses the following steps to update the model online, the process of which can be referred to... Figure 3 .

[0067] 1) When fine-tuning is triggered, extract multiple sets of recently generated historical write data from the circular buffer. The circular buffer is a fixed-size circular queue used to store historical data from recent periods or a number of recent write operations in real time. As an example, relevant data from the most recent 1000 write operations can be extracted as training samples. This historical data includes the request characteristics of each write request, the state characteristics of each storage unit at the time of the write, and the performance parameters collected after the write is completed (such as write latency, erase / write increment, bit error rate increment, etc.).

[0068] 2) For each historical write in the training data, determine the theoretically optimal allocation result for that write based on its actual performance parameters after the write, and generate a target score accordingly. For example, the performance of all candidate memory cells after the write can be compared, and the cell with the lowest write latency, the smallest increment of erase / write cycles, or the smallest increment of bit error rate can be selected as the target cell, with a higher target score (e.g., 1) for that cell and lower scores (e.g., 0) for other cells. Alternatively, a comprehensive scoring function can be constructed based on multiple performance indicators to calculate the actual performance score for each memory cell. This actual performance score serves as a supervision signal for subsequent calculations of the difference between model predictions and actual results.

[0069] 3) Input the request features of each historical write and the state features of each storage unit at that time from the training data into the current scoring model. Through forward propagation, the model calculates the predicted score for each storage unit. This predicted score reflects how well the model believes each storage unit is suitable for the current write operation under the current model parameters. Then, calculate the loss function based on the difference between the predicted score and the target score for each historical write. Compare the predicted score output by the model with the determined actual performance score, and calculate the difference as the loss value. The loss function can be the Mean Squared Error (MSE), which is the average of the squared differences between the predicted and actual scores. The smaller the loss value, the closer the model's prediction is to the optimal allocation result reflected in the actual write performance.

[0070] 4) Using the calculated loss value, calculate the gradient of the loss relative to the parameters of each layer of the model through the backpropagation algorithm, and update the weight parameters inside the model along the gradient descent direction. During the update process, a learning rate (e.g., 1e-4) can be set to control the step size of each parameter update. Through multiple rounds of iterative training (the number of iterations or early stopping conditions can be set as needed), the model gradually converges, completing this fine-tuning.

[0071] 5) After the model update is completed, model verification and replacement are required. The updated model must be verified to ensure that its performance is no less than that of the original model. After successful verification, the old model in operation is replaced with the new model using a hot-swap method, thereby completing the online model update without interrupting service. In this way, the scoring model can continuously optimize itself based on feedback from actual write performance, adapt to changes in workload and the evolution of storage unit states in a timely manner, and maintain the optimality of the data allocation strategy in the long term.

[0072] Figure 4 A schematic diagram of the composition structure of the data storage and distribution device provided in the embodiments of this application is shown.

[0073] refer to Figure 4This application also provides a data storage and distribution apparatus, the apparatus comprising: The receiving module 401 is used to receive a write request, which includes data to be written. The acquisition module 402 is used to acquire the request characteristics of the data to be written and the status characteristics of multiple storage units; The scoring module 403 is used to input the request features and the state features of each storage unit into the scoring model to obtain the score of each storage unit output by the scoring model; wherein, the scoring model determines multiple feature weights based on the request features and the state features of each storage unit, and determines the score of each storage unit based on the multiple feature weights and the state features of each storage unit. The write module 404 is used to select a target storage unit from multiple storage units based on the score and write the data to be written to the target storage unit.

[0074] It should be noted that the description of the apparatus in this application embodiment is similar to the description of the method embodiment above, and has similar beneficial effects as the method embodiment; therefore, it will not be repeated. For any technical details not covered in the data storage and distribution apparatus provided in this application embodiment, please refer to... Figures 1 to 3 The meaning is understood in accordance with the description of any of the accompanying drawings.

[0075] According to embodiments of this application, this application also provides an electronic device and a readable storage medium.

[0076] Figure 5 A schematic block diagram of an example electronic device 500 that can be used to implement embodiments of this application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the application described and / or claimed herein.

[0077] like Figure 5As shown, the electronic device 500 includes a computing unit 501, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 502 or a computer program loaded from a storage unit 508 into a random access memory (RAM) 503. The RAM 503 may also store various programs and data required for the operation of the electronic device 500. The computing unit 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0078] Multiple components in electronic device 500 are connected to I / O interface 505, including: input unit 506, such as keyboard, mouse, etc.; output unit 507, such as various types of monitors, speakers, etc.; storage unit 508, such as disk, optical disk, etc.; and communication unit 509, such as network card, modem, wireless transceiver, etc. Communication unit 509 allows electronic device 500 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0079] The computing unit 501 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 501 performs the various methods and processes described above, such as data storage allocation methods. For example, in some embodiments, the data storage allocation method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 508. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 500 via ROM 502 and / or communication unit 509. When the computer program is loaded into RAM 503 and executed by the computing unit 501, one or more steps of the data storage allocation method described above may be performed. Alternatively, in other embodiments, the computing unit 501 may be configured to perform the data storage allocation method by any other suitable means (e.g., by means of firmware).

[0080] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transferring data and instructions to the storage system, the at least one input device, and the at least one output device.

[0081] The program code used to implement the methods of this application may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0082] In the context of this application, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0083] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0084] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0085] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0086] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this application can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this application can be achieved, and this is not limited herein.

[0087] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

[0088] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A data storage allocation method, characterized in that, The method includes: Receive a write request, the write request including data to be written; Obtain the request characteristics of the data to be written and the status characteristics of multiple storage units; The request features and the state features of each storage unit are input into the scoring model to obtain the score of each storage unit output by the scoring model; wherein, the scoring model determines multiple feature weights based on the request features and the state features of each storage unit, and determines the score of each storage unit based on the multiple feature weights and the state features of each storage unit. Based on the score, a target storage unit is selected from the plurality of storage units, and the data to be written is written to the target storage unit.

2. The method according to claim 1, characterized in that, The request characteristics include at least one of the following: logical address, data type, access frequency, and request priority; The state characteristics include at least one of the following: number of erase / write cycles, bit error rate, data dwell time, and access delay.

3. The method according to claim 1, characterized in that, The scoring model is trained through the following operations: Acquire historical write data, which includes historical request characteristics, historical status characteristics, and performance parameters of each storage unit after each write. Based on the performance parameters, determine the corresponding storage unit for each write operation, and use the corresponding storage unit as the training label; The historical request features and historical state features are used as training inputs, and the training labels are used as training targets to train the initial model, thereby obtaining the scoring model.

4. The method according to claim 3, characterized in that, The performance parameters include at least one of the following: write latency, erase / write increment, and bit error rate increment.

5. The method according to claim 1, characterized in that, The scoring model includes a first branch network and a second branch network; The scoring model determines multiple feature weights based on the request features and the state features of each storage unit, and determines a score for each storage unit based on the multiple feature weights and the state features of each storage unit, including: Based on the request characteristics and the state characteristics of each storage unit, the spatial topology characteristics of the storage unit are extracted using the first branch network; Based on the request features and the state features of each storage unit, the timing access features corresponding to the request features are extracted using the second branch network; The spatial topology features and the temporal access features are fused to obtain the fused features; Multiple feature weights are determined based on the fusion features; The score for each storage unit is determined based on the weights of the multiple features and the state characteristics of each storage unit.

6. The method according to claim 5, characterized in that, The multiple feature weights include a first weight corresponding to the number of erase / write cycles, a second weight corresponding to the access frequency, and a third weight corresponding to the bit error rate; correspondingly The scoring model determines the score of each storage unit based on the weights of the multiple features and the state characteristics of each storage unit, including: The percentage of erase / write cycles for each storage cell is determined based on the number of erase / write cycles in the state characteristics of each storage cell and the maximum allowed number of erase / write cycles for each storage cell. The score for each storage cell is determined based on the percentage of erase / write cycles for each storage cell and the first weight, the access frequency in the state characteristics of each storage cell and the second weight, and the bit error rate in the state characteristics of each storage cell and the third weight.

7. The method according to claim 1, characterized in that, The method further includes: Under preset conditions, the scoring model is fine-tuned through backpropagation.

8. The method according to claim 7, characterized in that, The preset conditions include at least one of the following: The cumulative number of erase / write cycles has reached the preset threshold. The time elapsed since the last fine-tuning of the scoring model has reached a preset time threshold; The average bit error rate of multiple storage units has increased beyond a preset proportional threshold.

9. The method according to claim 7, characterized in that, The fine-tuning of the scoring model through backpropagation includes: Extract relevant data from multiple historical writes in the circular buffer as training data; The target score is determined based on the performance parameters after each historical write in the training data. Based on the scoring model and the relevant data for each historical write, determine the predicted score for each historical write. Calculate the loss function based on the difference between the predicted score and the target score for each historical write; The scoring model is updated via backpropagation based on the loss function.

10. A data storage and distribution device, characterized in that, The device includes: A receiving module is used to receive a write request, wherein the write request includes data to be written; The acquisition module is used to acquire the request characteristics of the data to be written and the status characteristics of multiple storage units; The scoring module is used to input the request features and the state features of each storage unit into the scoring model to obtain the score of each storage unit output by the scoring model; wherein, the scoring model determines multiple feature weights based on the request features and the state features of each storage unit, and determines the score of each storage unit based on the multiple feature weights and the state features of each storage unit. The write module is used to select a target storage unit from the plurality of storage units according to the score, and write the data to be written into the target storage unit.

Images