Solid state drive read disturb compensation method and device based on deep reinforcement learning

Abstract

This invention provides a method and apparatus for solid-state drive (SSD) read interference compensation based on deep reinforcement learning. The method involves acquiring read operation event stream records; constructing a read interference propagation impact map by combining the physical block topology and historical compensation records, generating a read interference evolution state representation sequence; inputting this sequence into a recurrent state inference network to generate implicit representations of the interference propagation state; then inputting this sequence into an action policy network to jointly generate a compensation adjustment instruction sequence including voltage threshold adjustment and read operation scheduling offset; executing the compensation adjustment, collecting response data, generating a reward evaluation signal, and storing the experience samples in an experience replay memory. This method can adaptively learn the optimal compensation strategy, effectively suppressing the accumulation of voltage offsets caused by read interference, and improving the read operation reliability and lifespan of the SSD while reducing the cost of compensation operations.

Description

Technical Field

[0001] This invention relates to the fields of deep learning and hard disk optimization technology, and in particular to a method and apparatus for compensating for read interference in solid-state drives based on deep reinforcement learning. Background Technology

[0002] Solid-state drives (SSDs), as core data storage devices, are susceptible to read interference between adjacent physical blocks during read operations. This interference can cause unexpected shifts in the voltage threshold of storage cells, leading to data read errors when the voltage threshold shift accumulates to a certain level. Current compensation methods for SSD read interference typically involve adjusting the voltage threshold uniformly for a physical block after the number of read operations on that block reaches a preset threshold, or making independent compensation decisions based on the historical read operation frequency statistics of individual physical blocks. However, these compensation methods face the following problems in practical applications: Interference propagates between different physical blocks within an SSD due to their proximity and differences in read operation frequencies. Relying solely on the number of read operations on a single physical block or historical frequency statistics is insufficient to accurately characterize the spatial propagation characteristics and temporal evolution of interference between physical blocks. This makes it difficult to match the timing and magnitude of compensation with the actual cumulative interference, thus limiting the effectiveness of the compensation. Summary of the Invention

[0003] In view of this, the present invention provides a method and apparatus for compensating for read interference in solid-state drives based on deep reinforcement learning. The technical solution of the embodiments of the present invention is implemented as follows:

[0004] On one hand, embodiments of the present invention provide a solid-state drive read interference compensation method based on deep reinforcement learning, comprising:

[0005] Acquire read operation event stream records of multiple physical blocks in a solid-state drive during a continuous read operation cycle. The read operation event stream records include the physical block identifier, read operation timestamp, and read operation voltage threshold offset record for each read operation.

[0006] Based on the read operation event stream records and the physical block topology of the solid-state drive, a read interference propagation impact map is constructed. Based on the read interference propagation impact map and historical compensation operation records, a read interference evolution state characterization sequence for each physical block is generated. The read interference evolution state characterization sequence includes the voltage offset prediction trajectory of each physical block in the uncompensated state and the voltage offset response feedback information after historical compensation operations.

[0007] The sequence of read interference evolution state representations is input into the recurrent state inference network of the deep reinforcement learning model. The temporal correlation model of the read interference evolution state representation sequence of each physical block is performed through the gated state update mechanism of the recurrent state inference network to generate the hidden representation of the interference propagation state of each physical block at the current time.

[0008] The implicit representation of the interference propagation state is input into the action policy network of the deep reinforcement learning model. The multi-branch decision structure of the action policy network performs joint decision-making processing on the compensation actions of each physical block group, generating a compensation adjustment instruction sequence that includes voltage threshold adjustment instructions and read operation scheduling offset instructions.

[0009] The read operation compensation and adjustment operation of the solid-state drive is executed according to the compensation and adjustment instruction sequence. The voltage threshold offset response observation data of each physical block after compensation and adjustment in the subsequent read operation cycle is collected. The reward evaluation signal is generated based on the difference between the voltage threshold offset response observation data and the predicted trajectory in the read interference evolution state characterization sequence. The reward evaluation signal, the read interference evolution state characterization sequence and the compensation and adjustment instruction sequence are combined into a new experience sample and stored in the experience playback memory bank.

[0010] On the other hand, embodiments of the present invention provide a solid-state drive read interference compensation device, comprising:

[0011] The record acquisition module is used to acquire read operation event stream records of multiple physical blocks in the solid-state drive during a continuous read operation cycle. The read operation event stream records include the physical block identifier, read operation timestamp, and read operation voltage threshold offset record for each read operation.

[0012] The graph construction module is used to construct a read interference propagation impact graph based on the read operation event stream record and the physical block topology of the solid-state drive. Based on the read interference propagation impact graph and historical compensation operation record, it generates a read interference evolution state characterization sequence for each physical block. The read interference evolution state characterization sequence includes the voltage offset prediction trajectory of each physical block in the uncompensated state and the voltage offset response feedback information after historical compensation operation.

[0013] The temporal modeling module is used to input the read interference evolution state representation sequence into the recurrent state inference network of the deep reinforcement learning model. Through the gated state update mechanism of the recurrent state inference network, the temporal correlation model of the read interference evolution state representation sequence of each physical block is performed to generate the implicit representation of the interference propagation state of each physical block at the current moment.

[0014] The compensation decision module is used to input the implicit representation of the interference propagation state into the action policy network of the deep reinforcement learning model. Through the multi-branch decision structure of the action policy network, it performs joint decision processing on the compensation actions of each physical block group, and generates a compensation adjustment instruction sequence that includes voltage threshold adjustment instructions and read operation scheduling offset instructions.

[0015] The compensation and adjustment module is used to perform read operation compensation and adjustment operations on the solid-state drive according to the compensation and adjustment instruction sequence, collect voltage threshold offset response observation data of each physical block in subsequent read operation cycles after compensation and adjustment, generate reward evaluation signal based on the difference between the voltage threshold offset response observation data and the predicted trajectory in the read interference evolution state characterization sequence, and combine the reward evaluation signal, the read interference evolution state characterization sequence and the compensation and adjustment instruction sequence into a new experience sample and store it in the experience playback memory bank.

[0016] This invention constructs a read interference propagation impact map by acquiring read operation event stream records and combining them with the physical block topology. Simultaneously, it generates read interference evolution state representation sequences for each physical block using historical compensation operation records. This transforms the complex interference propagation mechanism within the solid-state drive (SSD) into a structured graph representation and state sequence representation, providing a complete panoramic information on interference evolution for subsequent compensation decisions. The read interference evolution state representation sequences are input into the recurrent state inference network of a deep reinforcement learning model. A gated state update mechanism models the temporal correlation of each physical block, generating implicit representations of the interference propagation state of each physical block at the current moment. This achieves accurate characterization of the dynamic evolution of interference propagation without relying on manually preset interference propagation models. The implicit representations of the interference propagation state are then input into an action policy network. The network uses a multi-branch decision structure to jointly process compensation actions for each physical block group, generating a compensation adjustment instruction sequence that includes voltage threshold adjustment instructions and read operation scheduling offset instructions. This achieves coordinated compensation for read interference from both the voltage and scheduling domains. Finally, the network executes read operation compensation adjustment operations according to the compensation adjustment instruction sequence, collects the voltage threshold offset response observation data after compensation adjustment, and generates a reward evaluation signal based on the difference between the response observation data and the predicted trajectory in the read interference evolution state characterization sequence. This reward evaluation signal, along with the read interference evolution state characterization sequence and the compensation adjustment instruction sequence, is combined into a new experience sample and stored in the experience playback memory bank. Through autonomous exploration and experience accumulation, the compensation strategy is continuously optimized, effectively suppressing the accumulation of voltage threshold offset caused by read interference. Attached Figure Description

[0017] Figure 1 This is a schematic diagram illustrating the implementation process of a solid-state drive read interference compensation method based on deep reinforcement learning, provided in an embodiment of the present invention.

[0018] Figure 2 This is a schematic diagram of the composition structure of a solid-state drive read interference compensation device provided in an embodiment of the present invention.

[0019] Figure 3 This is a schematic diagram of the hardware entity of a computer system provided in an embodiment of the present invention. Detailed Implementation

[0020] This invention provides a method for compensating for read interference in solid-state drives based on deep reinforcement learning. This method can be executed by a processor of a computer system. The computer system can refer to a device with data processing capabilities, such as a server, laptop, tablet, desktop computer, or mobile device (e.g., mobile phone, portable video player, personal digital assistant, dedicated messaging device, portable gaming device).

[0021] Figure 1 This is a schematic diagram illustrating the implementation process of a solid-state drive read interference compensation method based on deep reinforcement learning, as provided in an embodiment of the present invention. Figure 1 As shown, the method includes:

[0022] Step S100: Obtain read operation event stream records of multiple physical blocks in the solid-state drive during a continuous read operation cycle. The read operation event stream records include the physical block identifier, read operation timestamp, and read operation voltage threshold offset record corresponding to each read operation.

[0023] Solid-state drives (SSDs) consist of multiple physical blocks, each of which is the smallest unit for data erasure operations and contains multiple physical pages. The continuous read operation cycle is a preset monitoring time window, which can be a fixed-length time interval or a sliding time window dynamically adjusted according to the read operation frequency. The read operation event stream record is a data set that records relevant information for each read operation in chronological order within a continuous read operation cycle for selected physical blocks in the SSD. Physical block identifiers uniquely identify each physical block in the SSD and are encoded using physical block numbering. The read operation timestamp records the specific time of the read operation, marked by the internal clock counter of the SSD controller chip. The read operation voltage threshold offset record is the offset value between the read reference voltage applied to correctly read the data stored in the storage cell and the standard read reference voltage during a read operation. The sign and magnitude of this offset directly reflect the drift direction and degree of the storage cell threshold voltage caused by accumulated read interference. When performing a read operation, the solid-state drive (SSD) controller chip reads the voltage threshold offset compensation value of the current physical block through the voltage control unit. This compensation value, along with the physical block identifier of the read operation and the read operation timestamp obtained from the hardware timer, is assembled into a read operation event record, which is then written to the controller chip's internal cache queue. After a continuous read operation cycle ends, the controller chip retrieves these records in batches from the cache queue, sorts them according to the order of their timestamps, and obtains a structured read operation event stream record.

[0024] Step S200: Construct a read interference propagation impact map based on the read operation event stream record and the physical block topology of the solid-state drive, and generate a read interference evolution state characterization sequence for each physical block based on the read interference propagation impact map and historical compensation operation records. The read interference evolution state characterization sequence includes the voltage offset prediction trajectory of each physical block in the uncompensated state and the voltage offset response feedback information after historical compensation operations.

[0025] The read interference propagation impact map is a directed graph structure describing the propagation relationship of interference between different physical blocks in a solid-state drive (SSD) due to mutual influence from read operations. The physical block topology is the spatial arrangement of physical blocks within the SSD chip and their physical proximity relationships; this structure is obtained from the SSD's physical address mapping table. The historical compensation operation log is a record of a series of compensation actions performed by the SSD during past operation to suppress read interference. It includes the target physical block applied to each compensation operation, the compensation time, the voltage threshold adjustment magnitude, and the observed voltage offset changes after compensation.

[0026] The read disturbance evolution state characterization sequence is a data chain that arranges the states of each physical block in a continuous time window in a temporal sequence. Each state node in this sequence contains the value of the voltage offset prediction trajectory of that physical block without any compensation, as well as the voltage offset response feedback information observed after the actual compensation operation. The voltage offset prediction trajectory is a trend curve of the physical block voltage threshold offset changing over time, fitted by temporal analysis of the historical read operation event stream. The construction of this trajectory needs to consider the nonlinear relationship between the read operation frequency and the accumulation of voltage offset.

[0027] Voltage offset response feedback information records the deviation between the actual voltage offset of a physical block and the predicted trajectory after a preset compensation operation is applied. The statistical distribution of this deviation is used to evaluate the effectiveness of different compensation strategies. Timestamp sequences of all physical blocks are extracted from the read operation event stream records. The read operation density of each physical block is analyzed, and the spatial proximity between physical blocks is determined by combining this with the physical block adjacency table read from the SSD firmware. The mutual influence coefficient of read operation frequencies is calculated. Historical compensation operation records are read from the SSD's log storage area. Each record includes the compensation time point, the identifier of the compensated physical block, the adjusted voltage threshold amplitude, and the measured voltage threshold offset values ​​collected over multiple consecutive read operation cycles after compensation. These historical data are aligned with the propagation correlation in the read interference propagation impact map to generate a continuous read interference evolution state characterization sequence for each physical block.

[0028] In one implementation, step S200 may specifically include the following steps S210 to S260:

[0029] Step S210: Parse the read operation timestamp sequence of each physical block in the read operation event stream record, identify the dense read operation intervals and sparse read operation intervals in the read operation timestamp sequence, and generate a read operation burst mode descriptor for each physical block based on the alternating distribution pattern of dense and sparse read operation intervals.

[0030] In one implementation, step S210 may specifically include the following steps S211 to S216:

[0031] Step S211: Extract the read operation timestamp sequence from the read operation event stream record of each physical block, perform differential processing on the read operation timestamp sequence to obtain the time interval sequence between adjacent read operations, and each interval value in the time interval sequence corresponds to the time difference between two adjacent read operations.

[0032] For a given physical block's read operation timestamp sequence, performing a first-order difference operation yields a time interval sequence, where each interval is the time difference between two adjacent read operations. This sequence transforms consecutive time points into a set of time difference values ​​representing the density of read operations. The statistical distribution characteristics of the time interval sequence, such as skewness, kurtosis, and quantile distribution, can reflect the temporal clustering characteristics of read operations on that physical block. If the time interval sequence contains a large number of small interval values ​​and a small number of large interval values, it indicates that the read operations exhibit obvious burst characteristics; if the time interval sequence is relatively uniformly distributed, it indicates that the read operations exhibit stationary characteristics.

[0033] Step S212: Determine the threshold for dense read operations and the threshold for sparse read operations based on the statistical distribution characteristics of the time interval sequence. Divide adjacent read operations with a time interval value not greater than the threshold for dense read operations into consecutive read operation pairs within the dense read operation interval. Mark the idle time period between adjacent read operations with a time interval value not less than the threshold for sparse read operations as the sparse read operation interval.

[0034] Statistical analysis is performed on the time interval sequence to calculate the median, quartiles, and standard deviation. The median is multiplied by a first preset coefficient (less than 1) to obtain a dense read operation threshold, ensuring the dense threshold is below average. The median is then multiplied by a second preset coefficient (greater than 1) to obtain a sparse read operation threshold, ensuring the sparse threshold is above average. This adaptive threshold determination method based on statistical distribution adapts to the read operation characteristics of different physical blocks under varying load conditions, avoiding misjudgments caused by using a fixed threshold. Each interval value in the time interval sequence is iterated over. When the interval value is not greater than the dense read operation threshold, the two adjacent read operations corresponding to that interval value are marked as a dense read operation pair. When the interval value is not less than the sparse read operation threshold, the idle period between the two adjacent read operations corresponding to that interval value is marked as a sparse read operation interval.

[0035] Step S213: Perform connectivity merging processing on consecutive read operation pairs within the dense read operation interval, merge multiple consecutive read operations with adjacent time intervals satisfying the dense condition into one dense read operation interval, and generate the start read operation timestamp, end read operation timestamp, and number of read operations within the interval as dense read operation interval attributes.

[0036] Connectivity merging aggregates temporally consecutive read operation pairs that all satisfy the density condition. Starting from the first read operation marked as a dense read operation pair, it traverses backward along the time axis. If the interval between adjacent read operation pairs also satisfies the density condition, the read operations belonging to these pairs are merged into the same interval. When an interval that does not meet the density condition is encountered, the current dense interval ends. The start timestamp of the dense interval is recorded as the time of the first read operation within the interval, and the end timestamp is recorded as the time of the last read operation within the interval. The number of read operations within the interval is the total number of merged read operations. This merging process can integrate scattered dense read operation pairs into burst intervals with clear boundaries, so that the number of read operations within the interval can directly reflect the cumulative intensity of read interference within the burst interval.

[0037] Step S214: Identify the time interval between adjacent dense read operation intervals, divide the time interval between adjacent dense read operation intervals that is greater than the sparse read operation threshold into sparse read operation intervals, and generate the start idle timestamp, end idle timestamp and interval duration of each sparse read operation interval as sparse read operation interval attributes.

[0038] Iterate through all dense read operation intervals and calculate the time interval between two adjacent dense read operation intervals. This time interval is the difference between the end timestamp of the previous dense interval and the start timestamp of the next dense interval. When this difference is greater than the sparse read operation threshold, the time interval is divided into a sparse read operation interval. The start idle timestamp of the sparse read operation interval is the end timestamp of the previous dense interval, and the end idle timestamp is the start timestamp of the next dense interval. The duration of the interval is the difference between the end idle timestamp and the start idle timestamp. The duration of the sparse read operation interval reflects the recovery window length obtained by the physical block between two read operation bursts. This window length can help determine whether the accumulated read interference can be mitigated through natural recovery.

[0039] Step S215: Based on the dense read operation interval attributes and the sparse read operation interval attributes, the dense read operation intervals and the sparse read operation intervals are arranged alternately in chronological order to generate a read operation burst mode descriptor for each physical block. The read operation burst mode descriptor contains an alternating sequence of the distribution positions of the dense read operation intervals and the sparse read operation intervals.

[0040] The dense read operation intervals generated in step S213 and the sparse read operation intervals generated in step S214 are sorted according to their starting timestamps to obtain an alternating sequence. Each element in this sequence carries its type identifier (dense or sparse) and the corresponding interval attribute. This alternating sequence fully characterizes the burst-idle alternation pattern of physical block read operations on the time axis. The distribution of dense intervals reveals the time periods and frequencies of high read interference, while the distribution of sparse intervals reveals the time windows in which accumulated read interference may be mitigated. This pattern descriptor serves as the basic input for subsequent analysis of the synchronicity and propagation correlation of read operations between physical blocks.

[0041] Step S216: Input the read operation burst pattern descriptor of each physical block into the burst pattern clusterer. Cluster the read operation burst pattern descriptors of different physical blocks according to the distribution of the start timestamps of the dense read operation intervals and the similarity of the number of read operations within the intervals. Generate physical block group identifiers with similar read operation burst patterns and store the physical block group identifiers in association with the read operation burst pattern descriptors of each physical block.

[0042] The burst pattern clusterer can employ the density-based clustering algorithm DBSCAN. For the burst pattern descriptor of read operations in each physical block, the start timestamps of all dense read operation intervals are extracted to form a start timestamp vector, and the number of read operations within each dense read operation interval is extracted to form a count vector. These two vectors are concatenated to form the feature vector of the physical block. The clustering algorithm uses the Euclidean distance between feature vectors as a similarity measure, setting neighborhood radius and minimum sample size parameters. The algorithm traverses the feature vectors of all physical blocks, identifies core points, and constructs connected components, grouping physical blocks belonging to the same connected component into the same group. A unique group identifier is generated for each group, and this identifier is associated and stored in the corresponding physical block's burst pattern descriptor. Through clustering, physical blocks with similar read operation burst patterns are grouped together, and these physical blocks may exhibit similar propagation behaviors in subsequent interference propagation analysis.

[0043] Step S220: Extract the physical adjacency relationship between different physical blocks in the physical block topology of the solid-state drive. Determine the mutual influence coefficient of read operation frequency between physical blocks based on the physical adjacency relationship and the read operation burst mode descriptor of each physical block. The mutual influence coefficient of read operation frequency includes the influence direction identifier and the influence intensity level identifier.

[0044] The physical block topology records the coordinates of each physical block on the chip. Physical adjacency is determined by calculating the distances between two physical blocks in the horizontal, vertical, and diagonal directions. For each pair of physical blocks, the Manhattan distance is first calculated based on their coordinates in the topology. When the Manhattan distance is less than a preset adjacency radius, the pair of physical blocks is considered to have a physical adjacency relationship. The existence of physical adjacency is the spatial basis for read interference propagation, as the impact of read interference on neighboring physical blocks mainly stems from the capacitive coupling effect on word lines and bit lines, as well as the conduction of substrate noise. Read operation burst mode descriptors for the source and target physical blocks are obtained, and the timing distribution information of dense read operation intervals is extracted from the descriptors. The overlap ratio between the dense read operation intervals of the source and target physical blocks on the time axis is calculated. A higher overlap ratio indicates a stronger interference effect of the read operation burst of the source physical block on the target physical block. This is because when the source physical block is in a high-frequency read operation state, the interference signal it generates will propagate to neighboring physical blocks through the physical coupling path. If the target physical block is also performing a read operation at the same time, the accumulation rate of its voltage threshold offset will be significantly accelerated. The overlap ratio is mapped to a preset influence intensity level to generate an influence intensity level identifier. The influence direction identifier is determined based on the relative orientation of the physical blocks in the topology, with the unidirectional propagation direction from the source physical block to the target physical block serving as the influence direction identifier.

[0045] Step S230: Construct a directed graph structure with physical blocks as nodes and the mutual influence coefficient of read operation frequency as edge weights based on the mutual influence coefficient of read operation frequency and the physical adjacency relationship between physical blocks.

[0046] In one implementation, step S230 may specifically include the following steps S231 to S236:

[0047] Step S231: Determine the spatial proximity level between physical blocks based on the physical adjacency relationship between physical blocks, and mark physical block pairs with physical proximity levels exceeding a preset proximity threshold as candidate propagation physical block pairs. The candidate propagation physical block pairs include source physical block identifiers and target physical block identifiers.

[0048] The coordinates of physical blocks on the chip are extracted from the physical block topology. For any two physical blocks, their Euclidean distance can be calculated and mapped to a preset spatial proximity level. The smaller the distance, the higher the spatial proximity level. The spatial proximity level reflects the magnitude of the coupling capacitance between physical blocks and the impedance of the substrate noise propagation path. The higher the proximity, the stronger the physical basis for interference propagation. A proximity threshold is set, and physical block pairs with a spatial proximity level greater than the threshold are marked as candidate propagation physical block pairs, indicating that there is a potential possibility of interference propagation between these physical block pairs. In each candidate propagation physical block pair, the source physical block and the target physical block are determined according to the sorting order of the physical block identifiers or according to the relative orientation of their coordinate positions.

[0049] Step S232: For each candidate propagation physical block pair, obtain the read operation burst mode descriptor of the source physical block and the read operation burst mode descriptor of the target physical block, and calculate the intensity benchmark value of the read operation frequency mutual influence coefficient based on the degree of overlap between the dense read operation interval of the source physical block and the dense read operation interval of the target physical block on the time axis.

[0050] Extract the source physical block identifier and target physical block identifier from the candidate propagation physical block pairs, and read the corresponding read operation burst mode descriptors from the storage structure. Traverse all dense read operation intervals of the source physical block. For each interval, search the dense read operation interval list of the target physical block for intervals that overlap with that interval on the time axis. When calculating the overlap degree, divide the duration of the overlapping portion of the two intervals by the total duration of that dense read operation interval of the source physical block. This ratio reflects the overlap ratio between the read operation burst periods of the source physical block and the target physical block. The higher the overlap ratio, the greater the likelihood that the target physical block will simultaneously experience read operations within the time window of interference from the source physical block. Therefore, the higher the contribution of the source physical block read operations to the voltage threshold offset of the target physical block. Sum the overlap degrees of all dense read operation intervals using a weighted average, with the weight being the proportion of the number of read operations in each dense read operation interval to the total number of read operations in the source physical block. This weighted average overlap degree is then used as the intensity benchmark value.

[0051] Step S233: Based on the relative positions of the source physical block and the target physical block in the physical topology, determine the influence direction indicator of the mutual influence coefficient of the read operation frequency. The influence direction indicator is used to indicate the unidirectional propagation direction of the interference from the source physical block to the target physical block.

[0052] The center coordinates of the source and target physical blocks are obtained from the physical block topology. A vector pointing from the center of the source physical block to the center of the target physical block is calculated, and the direction angle of this vector is quantized to eight basic directions. The influence direction identifier is encoded as the direction code value corresponding to that direction, clearly indicating the unidirectional path of interference propagation. In the physical structure of solid-state drives, due to the asymmetry between the word line drive direction and the bit line read direction, interference propagation is not completely symmetrical. Therefore, the introduction of direction identifiers can more accurately reflect the actual physical propagation characteristics.

[0053] Step S234: Combine the intensity reference value with the influence direction identifier to generate a directional read operation frequency mutual influence coefficient for each candidate propagation physical block pair. The magnitude of the read operation frequency mutual influence coefficient reflects the degree of influence of the source physical block read operation on the target physical block voltage threshold offset.

[0054] The intensity benchmark value is used as the numerical part of the mutual influence coefficient of the read operation frequency, and the influence direction identifier is used as the directional attribute part of the coefficient. The two are combined to form a composite coefficient with directional attributes. This coefficient is numerically equal to the intensity benchmark value and semantically includes propagation direction information.

[0055] Step S235: Using all physical blocks as the node set, candidate propagation physical block pairs as the directed edge set, and the mutual influence coefficient of read operation frequency as the weight value of the directed edges, construct a directed graph structure as the read interference propagation influence map.

[0056] Specifically, an empty directed graph data structure is initialized. All physical blocks are traversed, and each physical block is added as a node to the directed graph. All candidate propagation physical block pairs are traversed. For each candidate propagation physical block pair, a directed edge is added between the source physical block node and the target physical block node, and the numerical part of the mutual influence coefficient of read operation frequencies is set as the weight of this directed edge. After adding all nodes and edges, a complete directed graph structure is obtained. For example, in this directed graph structure, a directed edge from node A to node B indicates that a read operation of physical block A will have an interfering effect on physical block B, and the edge weight represents the strength of the influence.

[0057] Step S236: Perform hierarchical clustering on the nodes in the directed graph structure, divide the physical blocks into different interference propagation levels according to the weight distribution of directed edges between nodes, generate interference propagation direction identifiers and inter-level transmission delay parameters between each level, and store the inter-level transmission delay parameters as additional attributes of directed edges in the interference propagation influence map.

[0058] Specifically, the directed graph structure is topologically sorted. Starting with nodes with an in-degree of zero, nodes with an in-degree of zero are removed layer by layer, and the layer number is recorded. Nodes removed in the same round are assigned to the same interference propagation layer. This hierarchical clustering process in this embodiment of the invention reveals the hierarchical structure of interference propagation: physical blocks at upstream layers are the main sources of interference, and their read operations affect physical blocks at multiple downstream layers; physical blocks at downstream layers are the victims of interference, and their voltage threshold offsets are cumulatively affected by multiple upstream physical blocks. The interference propagation direction identifier between layers is determined according to the layer number, with layers with smaller numbers pointing to layers with larger numbers. The inter-layer propagation delay parameter is determined by analyzing the difference in read operation timestamps between pairs of physical blocks belonging to different layers: physical blocks belonging to the source layer and physical blocks belonging to the target layer are selected, and the average time difference between the read operation timestamp of the source physical block and the moment of voltage offset change of the target physical block is calculated. This average time difference is used as the inter-layer propagation delay parameter. The physical significance of this delay parameter lies in the fact that the interference signal needs to undergo physical processes such as word line coupling and substrate noise conduction to propagate from the source physical block to the target physical block. This process involves a certain time delay. Incorporating this delay into the graph as an additional attribute of directed edges can provide a more accurate time reference for subsequent timing alignment and state modeling. This parameter is stored as an additional attribute of directed edges in the read interference propagation impact graph, and the additional attribute is associated with the corresponding directed edge.

[0059] Step S240: Extract the measured voltage threshold offset values ​​of each physical block at multiple historical moments and the corresponding compensation operation records from the historical compensation operation records, and associate and store the measured voltage threshold offset values ​​with the corresponding compensation operation records to form a set of historical compensation experience entries.

[0060] Historical compensation operation records are stored in the reserved storage area of ​​the solid-state drive. Each record includes the execution time of the compensation operation, the target physical block identifier, the measured voltage threshold offset before compensation, the measured voltage threshold offset after compensation, the compensation action type, and the compensation magnitude. The historical compensation operation records are traversed, and the measured voltage threshold offset (including before and after compensation) and the corresponding compensation operation record are extracted from each record. The measured values ​​before and after compensation for the same physical block at adjacent time points are correlated to obtain a historical compensation experience entry. This entry records the change in the physical block's voltage offset under the set compensation operation. All historical compensation experience entries for all physical blocks are aggregated to form a historical compensation experience entry set. This set provides a supervisory signal for subsequent compensation strategy learning. By analyzing the differences in voltage offset response under different compensation operations, the optimal compensation strategy can be learned.

[0061] Step S250: Based on the directed graph structure in the read interference propagation impact map, select historical compensation experience entries of other physical blocks that have a propagation relationship with the current physical block from the historical compensation experience entry set. Align the selected historical compensation experience entries with the historical compensation experience entries of the current physical block in time sequence to generate a read interference evolution state representation sequence for each physical block in a continuous time window. Each state node in the read interference evolution state representation sequence contains the measured value of the voltage offset of the current physical block, the voltage offset impact value propagated from the adjacent physical blocks, and the corresponding compensation operation record.

[0062] For the current physical block, all directed edges pointing to it are obtained from the read interference propagation impact map. The source physical blocks of these directed edges are other physical blocks that have a propagation correlation with the current physical block. The existence of propagation correlation means that read operations of these source physical blocks will have a cumulative effect on the voltage threshold offset of the current physical block. Therefore, the state information of these source physical blocks must be included when constructing the state representation of the current physical block. Historical compensation experience entries of these source physical blocks are selected from the historical compensation experience entry set, and historical compensation experience entries of the current physical block itself are also selected. The selected entries are sorted by timestamp, and a linear interpolation method is used to align the time points of all entries to a unified time grid.

[0063] At each time grid point, the measured voltage offset of the current physical block is directly obtained from the historical compensation experience entries of the current physical block; the influence value of the voltage offset propagated from adjacent physical blocks is obtained by multiplying the measured voltage offset of the source physical block at that time point by the weight coefficient of the directed edge. This calculation method is based on the linear superposition assumption of interference propagation, that is, the voltage offset of the source physical block contributes to the total offset of the target physical block in a certain proportion through the propagation path; the corresponding compensation operation record extracts the compensation operation information executed at that time point from the historical compensation experience entries of the current physical block. These three parts of information at each time grid point are combined into a state node, and the state nodes at all time grid points are arranged in chronological order to form a read interference evolution state representation sequence.

[0064] Step S260: Perform time-series smoothing on the read interference evolution state characterization sequence. Based on the propagation correlation in the directed graph structure, realign the voltage offset influence values ​​in the state nodes of adjacent physical blocks according to the propagation delay time to generate an aligned state node sequence with a unified time reference. Update the confidence weight of the voltage offset response feedback information in the read interference evolution state characterization sequence of each physical block according to the ratio of the measured voltage offset value of each state node in the aligned state node sequence to the propagated voltage offset influence value.

[0065] The inter-level propagation delay parameter associated with the directed edges related to the current physical block is extracted from the interference propagation impact map. For each propagation-related source physical block, its voltage offset impact value is shifted forward on the time axis by the amount of the propagation delay parameter, aligning the occurrence time of the impact value with the actual arrival time of the current physical block. This realignment operation resolves the timing misalignment problem caused by physical propagation delay, ensuring that at the same time grid point, the measured voltage offset value of the current physical block and the impact value from the source physical block are causally consistent in time. After shifting, the voltage offset impact value sequence is smoothed using a sliding window averaging method, with the window size set to an integer multiple of the propagation delay parameter to eliminate the effects of measurement noise and random fluctuations. After alignment and smoothing, an aligned state node sequence with a unified time reference is obtained. Each state node in the aligned state node sequence is traversed, and the ratio of the measured voltage offset value to the propagated voltage offset impact value is calculated. This ratio reflects the relative magnitude between the voltage offset generated by the current physical block itself and the offset propagated from adjacent physical blocks at the current time point. The current ratio is compared with the historical average ratio. If the deviation between the current ratio and the historical average ratio exceeds a preset threshold, it indicates that the voltage offset response feedback information in this state node may be abnormal (e.g., due to measurement errors or uncertainties in compensation operations). Therefore, the confidence weight of the voltage offset response feedback information in this node is reduced. If the deviation is within the threshold range, the confidence weight is maintained or increased. This confidence weight update mechanism allows subsequent deep reinforcement learning models to focus more on state nodes with higher reliability during training, improving the model's learning efficiency and stability. The updated confidence weights are associated with and stored with the corresponding state nodes, completing the optimization of the read disturbance evolution state representation sequence.

[0066] Step S300: Input the read interference evolution state representation sequence into the recurrent state inference network of the deep reinforcement learning model, and perform temporal correlation modeling on the read interference evolution state representation sequence of each physical block through the gated state update mechanism of the recurrent state inference network to generate the hidden representation of the interference propagation state of each physical block at the current time.

[0067] The deep reinforcement learning model comprises two main components: a recurrent state inference network and an action policy network. The recurrent state inference network employs a gated recurrent unit network structure, which includes an input layer, a gated state update unit, an output projection layer, and a feedback adjustment unit. The read interference evolution state representation sequence is fed into the input layer of the recurrent state inference network as input data. The gated state update mechanism controls the fusion ratio of historical and current information through reset and update gates, thereby modeling temporal dependencies. In the application scenario of solid-state drive read interference compensation, the voltage offset of a physical block depends not only on the current read operation frequency but also on the long-term impact of historically accumulated interference. Furthermore, the interference propagation between different physical blocks exhibits complex temporal dependencies. The gate mechanism of the recurrent state inference network effectively captures this long-distance temporal dependency by selectively forgetting historical states and selectively remembering current inputs, constructing a hidden state representation that reflects the dynamics of interference accumulation and propagation. The network processes the input sequence step-by-step in the temporal dimension, and the output hidden state at each time step aggregates all historical information from the beginning of the sequence to the current moment. The hidden state output at the last time step is transformed nonlinearly by the output projection layer to generate a latent representation of the interference propagation state of each physical block at the current time. This latent representation is a high-dimensional vector that encodes the degree of interference accumulation, propagation impact state, and compensation response characteristics of the physical block at the current time.

[0068] In one implementation, step S300 may specifically include the following steps S310 to S360:

[0069] Step S310: Arrange the state nodes of each physical block in the read interference evolution state characterization sequence in the continuous time window according to the time order to obtain the state node time sequence chain of each physical block. Each state node in the state node time sequence chain contains the measured value of the voltage offset of the current physical block, the influence value of the voltage offset propagated from the adjacent physical blocks, and the corresponding compensation operation record.

[0070] For each physical block, all state nodes of that physical block are extracted from the read disturbance evolution state representation sequence and arranged in ascending order of timestamps to obtain a state node time series chain. The length of each time series chain is equal to the number of time grid points within a continuous time window, with each grid point corresponding to a state node. The construction of the state node time series chain organizes the scattered discrete state nodes into a continuous time series, providing a standardized input format for subsequent recurrent neural network processing.

[0071] Step S320: Input the temporal chain of the state nodes of each physical block into the input layer of the cyclic state inference network in parallel. The input layer allocates an independent input channel for each physical block, and the parameters are shared between the input channels to extract the common temporal evolution law of different types of physical blocks.

[0072] The input layer of the recurrent state inference network is designed as a multi-channel parallel input structure, with the number of physical blocks determining the number of input channels. Each input channel corresponds to a state node timing chain of a physical block and independently receives the input data of that physical block. All input channels share the same network parameters; that is, the weight matrix and bias terms of the gated state update unit are the same across channels of different physical blocks. This parameter sharing mechanism enables the network to learn the common patterns of different physical blocks during the interference evolution process, such as the basic growth pattern of voltage offset accumulation during readout operations and the recovery decay pattern after compensation operations. Simultaneously, because each physical block has an independent input channel, the network can maintain the ability to process the independent timing of each physical block, preventing the temporal characteristics of different physical blocks from being mixed due to parameter sharing.

[0073] Step S330: The gating state update unit of the cyclic state inference network performs time-step recursive calculation of the state node time sequence chain of each physical block. At each time step, the gating state update unit calculates the reset gating signal and update gating signal based on the input state node of the current time step and the hidden state output of the previous time step.

[0074] In one implementation, step S330 may specifically include the following steps S331 to S336:

[0075] Step S331: Concatenate the input state node of the current time step with the hidden state output of the previous time step to generate a concatenated state vector. Input the concatenated state vector into the reset gate calculation unit. The reset gate calculation unit performs a linear transformation on the concatenated state vector and generates a reset gate signal through an activation function. The reset gate signal is used to control the proportion of forgotten information in the hidden state of the previous time step.

[0076] The reset gate computation unit contains a fully connected layer. The input dimension of this fully connected layer equals the sum of the input state node dimension and the hidden state dimension, and the output dimension equals the hidden state dimension. The fully connected layer performs a linear transformation on the concatenated state vector, i.e., multiplying it by the weight matrix and adding a bias term, to obtain an intermediate vector. This intermediate vector is then input into a sigmoid activation function, which maps the value of each element to between 0 and 1, resulting in the reset gate signal vector. The closer each element value in the reset gate signal is to 0, the higher the proportion of historical information in the corresponding dimension that has been forgotten. In the solid-state drive read interference compensation scenario, the learning mechanism of the reset gate signal can automatically identify the differences in the importance of historical information in different dimensions to the current state prediction.

[0077] Step S332: Input the spliced ​​state vector into the update gate calculation unit, perform a linear transformation on the spliced ​​state vector, and generate an update gate signal through an activation function. The update gate signal is used to control the fusion ratio of the current input information and historical information during state update.

[0078] The structure of the update gate computation unit is similar to that of the reset gate computation unit, containing a fully connected layer and a sigmoid activation function. The concatenated state vector is input to the fully connected layer for linear transformation. The transformed vector is then mapped to the 0-1 interval by the sigmoid activation function, generating the update gate signal vector. The value of the update gate signal determines the weight of the candidate hidden state in the final hidden state; a larger value indicates a greater contribution of the current input information to the state update. In interference propagation modeling, when a physical block is in a dense read operation region and receives strong propagation influences from multiple upstream physical blocks, the update gate signal increases, making the current input information dominate the state update.

[0079] Step S333: Reset the hidden state output from the previous time step according to the reset gating signal, concatenate the reset hidden state with the input state node of the current time step, input it into the candidate state calculation unit, perform a linear transformation on the concatenated vector, and generate candidate hidden states through the activation function.

[0080] The reset process involves element-wise multiplying the hidden state vector from the previous time step with the reset gating signal vector to obtain the reset hidden state. In the reset hidden state, information in dimensions suppressed by the reset gating signal is discarded; the historical features corresponding to these dimensions are considered irrelevant to the current interference state prediction. The reset hidden state is then concatenated with the input state node of the current time step to obtain a new concatenated vector. This vector is input into the candidate state computation unit, which contains a fully connected layer and a hyperbolic tangent activation function. The fully connected layer performs a linear transformation on the concatenated vector, and the transformed vector is mapped to the interval between -1 and 1 using the hyperbolic tangent activation function, generating a candidate hidden state vector. The candidate hidden state represents a new state candidate value formed based on the current input and selectively retained historical information. In interference propagation modeling, the candidate hidden state is a preliminary estimate of the physical block interference state at the current time step, including the impact of the current read operation, the propagation impact from neighboring physical blocks, and the historical cumulative impact after reset filtering.

[0081] Step S334: Based on the updated gating signal, perform a weighted summation on the hidden state and candidate hidden states output at the previous time step to generate the hidden state output at the current time step. In the weighted summation, the updated gating signal is used as a weight coefficient to control the contribution of the candidate hidden states.

[0082] The hidden state output at the current time step is calculated as follows: The hidden state from the previous time step is multiplied element-wise by the difference between 1 and the update gating signal to obtain the historical information retention part; the candidate hidden state is multiplied element-wise by the update gating signal to obtain the new information integration part; the two parts are then added together to obtain the final hidden state at the current time step. The value of the update gating signal determines the contribution of new information to the state update; a larger value indicates a higher proportion of new information. This weighted summation mechanism allows the network to smoothly integrate historical and current information, avoiding gradient explosion or gradient vanishing problems caused by simple linear accumulation. In interference propagation modeling, this mechanism enables the network to dynamically adjust the fusion ratio of historical and current information according to the actual situation of interference accumulation, thereby more accurately capturing the nonlinear characteristics of interference evolution.

[0083] Step S335: Pass the hidden state output at the current time step to the next time step as the input to the gating state update unit at the next time step, and output the hidden state output at the current time step to the output layer of the recurrent state inference network for subsequent processing.

[0084] After the state recursion calculation is completed at each time step, the hidden state output at the current time step is stored in two paths: one path serves as the input for the next time step, passing it to the gated state update unit for calculation in subsequent time steps, thus realizing the continuous transmission of temporal information; the other path outputs to the output layer of the recurrent state inference network, where it is aggregated with the hidden states of other physical blocks at that time step. This dual-path design ensures the continuous transmission of temporal information while providing intermediate representations for each time step for subsequent global state aggregation.

[0085] Step S336: Group the hidden states output by each physical block at the current time step according to the propagation hierarchy of the physical blocks in the read interference propagation influence map, perform aggregation operation on the hidden states of all physical blocks within the same propagation hierarchy to generate a hierarchical aggregation state vector, and broadcast the hierarchical aggregation state vector back to each physical block of the corresponding propagation hierarchy as additional hidden state input.

[0086] The interference propagation impact map is used to obtain the interference propagation level number of each physical block. At the same time step, the hidden state vectors of physical blocks belonging to the same level are aggregated using average pooling, which calculates the average value for each dimension of the hidden state vectors of all physical blocks within the same level to obtain the level aggregated state vector. The level aggregated state vector gathers the interference state information of all physical blocks within that level, reflecting the overall interference distribution characteristics of that level. This level aggregated state vector is copied multiple times and broadcast back to each physical block in that level as additional input information for that physical block in the next time step calculation. It is then concatenated with the hidden state of that physical block before participating in subsequent calculations. This mechanism enables information sharing among physical blocks within the same propagation level, allowing them to perceive each other's interference states and collaboratively consider the impact within the level during compensation decisions. Furthermore, since the aggregation operation is independent between different levels, there is no information confusion between physical blocks at different levels, maintaining the directionality of interference propagation.

[0087] Step S340: Based on the reset gating signal and the update gating signal, selectively forget the hidden state output at the previous time step and fuse it with the current input information to generate the hidden state output at the current time step as the intermediate state representation of the physical block at the current moment.

[0088] The specific implementation process of this step has been described in detail in steps S331 to S335. Through the synergistic effect of resetting the gating signal and updating the gating signal, the gating state update unit achieves selective forgetting of historical information and selective fusion of current input information. The resetting gating signal determines which information to discard from the historical state, and the updating gating signal determines which new information to introduce from the current input. The combined effect of the two enables the cyclic state inference network to adaptively adjust the magnitude and direction of the state update. The generated current time step output hidden state is a compressed representation of the temporal information of the physical block from the start time to the current time, serving as an intermediate state representation of the physical block at the current time. This intermediate state representation includes the physical block's own interference accumulation history, the propagation influence history from neighboring physical blocks, and the response history of the compensation operation.

[0089] Step S350: Concatenate the hidden states output by each physical block at the last time step to obtain a state tensor with uniform dimension. Input the state tensor into the output projection layer of the recurrent state inference network. Perform nonlinear transformation on the state tensor through the output projection layer to generate the hidden representation of the interference propagation state of each physical block at the current time.

[0090] In one implementation, step S350 may specifically include the following steps S351 to S356:

[0091] Step S351: Extract the hidden state output by the last time step of the state node timing chain for each physical block. The hidden state output by the last time step gathers all the timing evolution information of the physical block from the historical start time to the current time.

[0092] The hidden state vector of each physical block at the last time step is read from the state storage buffer of the recurrent state inference network. Due to the recursive computation characteristics of the gated state update unit, the hidden state at the last time step not only contains the input information of that time step, but also retains relevant information from all earlier time steps in the sequence through the selective memory of the gating mechanism. This ability to capture long-range dependencies is the core advantage of recurrent neural networks when processing time-series data.

[0093] Step S352: Concatenate the hidden states output by all physical blocks at the last time step according to the spatial arrangement order of the physical blocks in the read interference propagation influence map to generate an initial state tensor with spatial and feature dimensions. Each spatial position in the initial state tensor corresponds to the hidden state vector of a physical block.

[0094] Physical blocks are arranged in rows and columns on the chip, forming a two-dimensional spatial grid. The spatial coordinates of each physical block are obtained from the interference propagation effect map. The hidden state vectors of all physical blocks are placed into their corresponding grid positions, and positions where no physical block exists are filled with zero vectors, resulting in a three-dimensional tensor of shape equal to the number of rows multiplied by the number of columns multiplied by the feature dimension, i.e., the initial state tensor. This spatial arrangement organizes the originally independent hidden states of physical blocks into a data form with spatial structure.

[0095] Step S353: Input the initial state tensor into the spatial feature extraction unit of the output projection layer, extract the spatial neighborhood association features between physical blocks through convolution operation, and generate the spatially enhanced state tensor.

[0096] The spatial feature extraction unit employs a two-dimensional convolutional neural network structure containing multiple convolutional layers. Each convolutional layer uses multiple convolutional kernels; for example, the kernel size is 3×3 with a stride of 1, and the padding is set to uniform to maintain the spatial size of the output tensor. The initial state tensor is input into the first convolutional layer. The convolutional kernel slides along the spatial dimension of the tensor, performing a weighted summation of the hidden state vectors of physical blocks within each local neighborhood and adding a bias. After passing through an activation function, a feature map is output. The 3×3 kernel size allows each output position to aggregate the hidden state information of itself and its eight surrounding neighborhood positions, which corresponds precisely to the local adjacency relationship between physical blocks and their surrounding physical blocks in a solid-state drive. Multiple convolutional layers are stacked to progressively expand the receptive field, enabling each output position to aggregate information from physical blocks within a larger spatial range. For example, after two convolutional layers, each output position can aggregate information from physical blocks within a surrounding 5×5 area, which matches the actual spatial range of read interference propagation. The feature map output by the last convolutional layer is the spatially augmented state tensor. The feature vector of each spatial location not only contains the temporal state information of the physical block itself, but also incorporates the state information of its neighboring physical blocks, reflecting the interference propagation pattern in the local area.

[0097] Step S354: Input the spatially enhanced state tensor into the nonlinear mapping unit of the output projection layer, and perform channel-by-channel feature transformation on the spatially enhanced state tensor through a multi-layer fully connected structure to generate the implicit representation of the intermediate interference propagation state.

[0098] The nonlinear mapping unit employs a multilayer perceptron structure containing multiple fully connected layers. The spatially augmented state tensor is first flattened along the feature dimension, while preserving the independence of the spatial dimension, meaning that the feature vector at each spatial location is processed independently. Each fully connected layer performs a linear transformation on the input feature vector, and then performs feature mapping layer by layer through a nonlinear activation function. Multiple fully connected layers are stacked, progressively mapping the original feature space to a higher-level abstract feature space. This layer-by-layer nonlinear transformation can extract higher-order interaction relationships between features, such as the complex nonlinear combination relationship between the accumulation of interference within a physical block and its neighborhood propagation influence. The set of feature vectors output by the last fully connected layer forms the implicit representation of the intermediate interference propagation state. The spatial structure of this representation is the same as the input tensor, and the feature dimension is determined by the design of the output projection layer.

[0099] Step S355: Input the implicit representation of the intermediate interference propagation state into the temporal smoothing unit of the projection layer, and perform a weighted moving average processing based on the implicit representation of the intermediate interference propagation state at the current time and the implicit representation of the interference propagation state output at the previous time to generate the implicit representation of the interference propagation state of each physical block at the current time.

[0100] The temporal smoothing unit maintains a historical state buffer, storing the implicit representation of the interference propagation state output at the previous time step. For the intermediate implicit representation of the interference propagation state at the current time step, a weighted moving average is calculated between it and the implicit representation of the interference propagation state at the previous time step. The weighting coefficient is determined by the smoothing factor. The smoothing factor ranges from 0 to 1. The larger the smoothing factor, the higher the proportion of historical information in the current time step representation, and the smoother the temporal change. This smoothing process can suppress state abrupt changes introduced by measurement noise or transient response of compensation operations, making the changes in the implicit representation of the interference propagation state more consistent with the continuous characteristics of the physical process. At the same time, the smoothing process is also equivalent to a low-pass filter, preserving the main trend of the interference state evolution and filtering out high-frequency noise components.

[0101] Step S356: Set the output dimension of the interference propagation state implicit representation to match the input dimension of the action policy network, so that the interference propagation state implicit representation can be directly used as input data for the input layer of the action policy network.

[0102] During the design phase of the output projection layer, the output dimension of the last fully connected layer of the nonlinear mapping unit is set to be the same as the expected input dimension of the action policy network's input layer. This dimension-matching design eliminates the additional dimension transformation operations required for data transmission between the two networks, simplifying the integration of the network architecture. Simultaneously, dimension matching ensures that the information capacity of the implicit representation of the interference propagation state matches the representational capability of the action policy network, preventing the loss of key information due to too low a dimension and avoiding the introduction of redundant information that leads to training difficulties due to too high a dimension.

[0103] Step S360: Input the implicit representation of the interference propagation state into the feedback adjustment unit of the cyclic state inference network. The feedback adjustment unit generates a state change trend vector based on the change between the implicit representation of the interference propagation state of each physical block at the current time and the implicit representation of the interference propagation state output at the previous time. The state change trend vector is then fused with the implicit representation of the interference propagation state of each physical block at the current time and output.

[0104] Specifically, the feedback adjustment unit receives the implicit representation of the interference propagation state at the current moment and the implicit representation of the interference propagation state at the previous moment, calculates the difference between the two, and obtains a change tensor. This change tensor reflects the amplitude and direction of the interference state change per unit time, and is a direct representation of the dynamics of interference evolution. A convolution operation is performed on the change tensor to extract the spatiotemporal pattern of state change, generating a state change trend vector. The spatial receptive field of the convolution operation can capture the spatial correlation of the change, such as the synchronicity or propagation pattern of state changes of physical blocks in a certain area. This state change trend vector is then fused element-wise with the implicit representation of the interference propagation state at the current moment, and the fused result is used as the final output of the feedback adjustment unit. This fusion operation is equivalent to superimposing dynamic change information on the static state representation, so that the output representation includes both the current snapshot of the interference state and the trend information of state change. In compensation decision-making, the state change trend information is of great value for predicting the future direction of interference evolution and taking preventive compensation measures in advance.

[0105] Step S400: Input the implicit representation of the interference propagation state into the action policy network of the deep reinforcement learning model, and perform joint decision-making processing of compensation actions for each physical block group through the multi-branch decision structure of the action policy network to generate a compensation adjustment instruction sequence containing voltage threshold adjustment instructions and read operation scheduling offset instructions.

[0106] The action policy network employs a multi-branch decision structure, comprising a shared feature extraction backbone network, a voltage adjustment branch network, and a scheduling offset branch network. The implicit representation of the interference propagation state is first input to the shared feature extraction backbone network, which extracts spatial distribution patterns and inter-physical block correlation patterns to generate a shared feature representation. This shared feature representation is simultaneously input to the voltage adjustment branch network and the scheduling offset branch network, enabling parallel decision-making for different types of compensation actions. The voltage adjustment branch network generates a suggested voltage threshold adjustment range for each physical block, resulting in a set of voltage threshold adjustment instructions. The scheduling offset branch network generates a global scheduling offset policy, including the priority offset of each physical block in the read operation queue and instructions for rearranging the read operation execution order. The outputs of the two branches are fused to generate compensation adjustment instruction entries for each physical block, which are then arranged according to the propagation hierarchy to form a compensation adjustment instruction sequence. This multi-branch decision structure allows for joint optimization of two different types of compensation actions—voltage threshold adjustment and read operation scheduling—based on the same set of shared features, avoiding policy conflicts that might result from separate decision-making.

[0107] In one implementation, step S400 may specifically include the following steps S410 to S460:

[0108] Step S410: Input the hidden representation of the interference propagation state into the shared feature extraction backbone network of the action policy network. The shared feature extraction backbone network contains multiple convolutional layers and attention layers, which are used to extract the spatial distribution pattern and the physical block association pattern in the hidden representation of the interference propagation state to generate shared feature representation.

[0109] The shared feature extraction backbone network employs a residual convolutional network structure, comprising multiple residual convolutional blocks and a spatial attention module. The latent representation of the perturbation propagation state serves as the input to the backbone network as a 3D tensor. The first residual convolutional block contains two convolutional layers and a skip connection. For example, the convolutional layers use 3×3 kernels with a stride of 1 and uniform padding. Each convolutional layer is followed by a batch normalization layer and an activation function. The skip connection directly adds the input to the output of the convolutional block, mitigating the vanishing gradient problem and allowing the network to stack deeper layers to extract higher-level abstract features. Multiple residual convolutional blocks are stacked, progressively extracting higher-level spatial features. The spatial resolution of the feature map output by each residual convolutional block remains constant, but the number of feature channels gradually increases, enabling the network to express more complex spatial patterns at deeper levels. The spatial attention module calculates attention weights on the feature map in the spatial dimension. Specifically, it performs global average pooling and global max pooling on the feature map, concatenates the two pooling results, generates a spatial attention weight map through convolutional layers, and then multiplies the attention weight map element-wise with the original feature map. The spatial attention module enables the network to focus on physical block regions where interference propagation has a significant impact, i.e., those areas with high voltage offsets and large propagation effects, thereby allocating more attention resources in subsequent compensation decisions.

[0110] Step S420: Input the shared feature representation into the voltage adjustment branch network of the action policy network, generate a voltage threshold adjustment range proposal value according to the interference propagation state of each physical block in the shared feature representation, and associate the voltage threshold adjustment range proposal value with the physical block identifier to form a voltage threshold adjustment instruction set.

[0111] In one implementation, step S420 may specifically include the following steps S421 to S425:

[0112] Step S421: The physical block-level feature extraction layer of the shared feature representation input voltage adjustment branch network is used to extract features independently based on the feature regions corresponding to each physical block in the shared feature representation, generating a feature vector for each physical block. The feature vector contains the current interference propagation state encoding information of the physical block.

[0113] The shared feature representation preserves the spatial arrangement structure of the physical blocks. The physical block-level feature extraction layer traverses each physical block location in the spatial dimension, extracting the feature vector corresponding to that location. For each location in the spatial grid, the values ​​of all feature channels at that location are extracted from the shared feature representation to obtain a feature vector. This feature vector directly encodes the interference propagation state information of the physical block at the current moment, including the voltage offset state of the physical block itself, the influence state propagated from neighboring physical blocks, and the role of the physical block in the propagation spectrum.

[0114] Step S422: Configure an independent action output head for the feature vector of each physical block. The action output head contains a multilayer perceptron structure. The multilayer perceptron performs a layer-by-layer nonlinear mapping on the input feature vector to generate a probability distribution on the candidate set of voltage threshold adjustment amplitude for the physical block.

[0115] Each physical block corresponds to an independent action output head. All action output heads share the same multilayer perceptron structure but have independent parameters. The multilayer perceptron contains two fully connected layers. The first layer maps the input feature vector to an intermediate dimension through an activation function. The second layer maps the intermediate dimension to the output dimension, which is equal to the number of candidate values ​​in the candidate set for voltage threshold adjustment magnitude. The output of the second layer is converted into a probability distribution through a normalized exponential function, with each candidate value corresponding to a probability value. The design of independent action output heads allows different physical blocks to make different voltage threshold adjustment decisions based on their respective feature vectors, reflecting the differences in compensation strategies.

[0116] Step S423: Sample the voltage threshold adjustment range candidate set of each physical block according to the probability value to obtain the voltage threshold adjustment range recommendation value of each physical block, and convert the voltage threshold adjustment range recommendation value into a voltage adjustment level code that can be recognized by the voltage control unit.

[0117] A random sampling strategy is employed, selecting a candidate value from the candidate set based on a probability distribution as the suggested voltage threshold adjustment range for the physical block. Compared to deterministic selection (such as choosing the candidate value with the highest probability), random sampling introduces exploratory elements, allowing the deep reinforcement learning model to explore different compensation actions during training and avoiding premature convergence to local optima. The voltage control unit uses level codes to control the voltage adjustment range, with each level corresponding to a specific voltage adjustment amount. The sampled suggested value is mapped to the closest level code to generate an instruction format recognizable by the voltage control unit.

[0118] Step S424: Based on the propagation hierarchy in the read interference propagation impact map of the physical blocks, the physical blocks are divided into multiple physical block groups. After generating a voltage threshold adjustment range recommendation value for each physical block in the physical block group, the propagation impact consistency adjustment is performed on the voltage threshold adjustment range recommendation values ​​within the same physical block group. If the propagation impact consistency adjustment finds that the voltage threshold adjustment range recommendation value of the upstream physical block is greater than the voltage threshold adjustment range recommendation value of the downstream physical block within the same physical block group, the voltage threshold adjustment range recommendation values ​​are redistributed according to the propagation direction so that the voltage threshold adjustment range recommendation value of the upstream physical block is not greater than the voltage threshold adjustment range recommendation value of the downstream physical block.

[0119] The propagation level number of each physical block is obtained from the read interference propagation impact map, and physical blocks with the same level number are grouped into the same physical block group. The propagation level reflects the upstream and downstream relationship of interference propagation. Physical blocks with smaller level numbers are upstream, and the interference generated by their read operations will propagate to downstream physical blocks with larger level numbers. Physical blocks with larger level numbers are downstream, and their voltage threshold offset is affected by the cumulative interference from multiple upstream physical blocks. In the design of the voltage threshold adjustment strategy, this matching relationship between the propagation direction and the compensation intensity needs to be considered. As the source of interference, the frequency and intensity of the read operations of upstream physical blocks directly affect the degree of interference accumulation in downstream physical blocks. Therefore, when applying voltage threshold adjustment to upstream physical blocks, the adjustment range should be relatively mild to avoid the side effects of overcompensation. Downstream physical blocks, due to the cumulative interference from multiple upstream physical blocks, have more severe voltage threshold drift and require a larger voltage threshold adjustment for compensation. Therefore, in the consistency adjustment, the recommended value of the voltage threshold adjustment range of upstream physical blocks should not be greater than the recommended value of the adjustment range of downstream physical blocks, that is, the compensation intensity increases along the propagation direction.

[0120] Iterate through each physical block group and obtain the suggested voltage threshold adjustment range for all physical blocks within the group. Following the propagation direction, upstream physical blocks refer to those with smaller layer numbers, and downstream physical blocks refer to those with larger layer numbers. Check if there are any cases where the suggested value of an upstream physical block is greater than that of a downstream physical block, i.e., check if the compensation intensity is reversed with the propagation direction. If such a case exists, redistribute the suggested voltage threshold adjustment range according to the propagation direction. The specific redistribution method is as follows: starting from the downstream physical block, adjust step-by-step towards the upstream physical block. For each pair of adjacent layers of physical blocks, if the suggested value of the upstream physical block is greater than that of the downstream physical block, reduce the portion of the upstream physical block that exceeds the downstream physical block, and increase the suggested value of the downstream physical block by the same amount, so that the suggested value of the upstream physical block is adjusted to be no greater than that of the downstream physical block. Repeat this process until the constraint that the upstream suggested value is no greater than the downstream suggested value is satisfied between all adjacent layers of physical blocks. This adjustment ensures that the allocation of compensation resources is consistent with the direction of interference propagation. Upstream physical blocks use relatively mild compensation to avoid introducing new interference, while downstream physical blocks use relatively strong compensation to offset the effects of accumulated interference, thereby improving the overall effectiveness of the compensation strategy.

[0121] Step S425: Store the suggested voltage threshold adjustment range of each physical block after the consistency adjustment of propagation effect according to the physical block identifier, and generate a voltage threshold adjustment instruction set.

[0122] The recommended voltage threshold adjustment range for each physical block is associated with its corresponding physical block identifier to obtain key-value pair instruction entries. All instruction entries for all physical blocks form a voltage threshold adjustment instruction set, which will be used to subsequently fuse with scheduling offset instructions to generate the final compensation adjustment instruction sequence.

[0123] Step S430: Input the shared feature representation into the scheduling offset branch network of the action policy network, and generate a global scheduling offset policy based on the interference propagation state distribution of all physical blocks in the shared feature representation. The global scheduling offset policy includes the priority offset of each physical block in the read operation queue and the read operation execution order reordering instruction.

[0124] In one implementation, step S430 may specifically include the following steps S431 to S436:

[0125] Step S431: Input the shared feature representation into the global feature aggregation layer of the scheduling offset branch network, perform global average pooling on the shared feature representation in the spatial dimension to generate a global feature vector, which gathers the global distribution information of the interference propagation state of all physical blocks.

[0126] The global feature aggregation layer performs average pooling on the spatial dimension of the shared feature representation. For each feature channel in the feature map, the average value of all spatial locations in that channel is calculated to obtain the global feature value for that channel. The global feature values ​​of all channels are concatenated to obtain a one-dimensional global feature vector. This vector summarizes the interference propagation state information of all physical blocks on the entire chip, including the overall voltage offset level, the distribution of active interference propagation regions, and the historical effects of compensation operations. Due to the use of average pooling, the global feature vector is robust to state fluctuations in local regions, reflecting overall statistical characteristics rather than local details.

[0127] Step S432: Input the global feature vector into the policy generation network of the scheduling offset branch network. The policy generation network contains a multi-layer fully connected structure, which performs a layer-by-layer nonlinear transformation on the global feature vector to generate the original output vector of the global scheduling offset policy.

[0128] For example, the policy generation network employs a multilayer perceptron structure containing three fully connected layers. The global feature vector is input to the first fully connected layer, undergoes a linear transformation, and is then passed through an activation function to output an intermediate vector. This intermediate vector is input to the second fully connected layer, undergoes another linear transformation and activation, and outputs another intermediate vector. This intermediate vector is then input to the third fully connected layer, undergoes a linear transformation, and outputs the original output vector. The dimension of the original output vector is equal to the number of physical blocks multiplied by the number of bits in the priority offset encoding plus the number of bits in the order reordering indicator encoding. This nonlinear mapping capability of the fully connected structure enables the network to learn the complex mapping relationship between the global interference distribution and the scheduling policy.

[0129] Step S433: Decompose the original output vector into a priority offset quantum vector and a reordering indicator subvector. Each element in the priority offset quantum vector corresponds to the read operation scheduling priority offset of a physical block. The reordering indicator subvector contains the reordering order identifier of the read commands to be processed in the read operation queue.

[0130] According to the preset vector segmentation rules, the original output vector is divided into two sub-vectors. The first sub-vector is a priority offset quantum vector, the length of which is equal to the number of physical blocks multiplied by the number of bits encoded for each priority offset. After decoding, each physical block corresponds to a priority offset value. The second sub-vector is a reordering indicator sub-vector, the length of which is determined by the maximum length of the read operation queue. After decoding, it forms a reordering sequence. The priority offset determines the relative priority of each physical block in the read operation queue, while the reordering indicator provides finer-grained command-level scheduling control.

[0131] Step S434: Normalize the priority offset quantum vector so that the priority offset of each physical block is distributed within a preset offset range, and generate the priority offset of each physical block in the read operation queue.

[0132] Each element in the priority offset quantum vector is normalized. A min-max normalization method can be used to map the element value to a preset offset range, the specific range of which is not limited. The mapped integer is the read operation scheduling priority offset for that physical block; a positive value indicates an increase in priority, and a negative value indicates a decrease in priority. This normalization process ensures the priority offset remains within a reasonable range, preventing extreme values ​​from causing abnormal scheduler behavior.

[0133] Step S435: Input the order reordering indicator subvector into the reordering instruction encoder, convert the order reordering indicator subvector into a reordering instruction format that can be parsed by the read operation scheduler, and generate a read operation execution order reordering indicator.

[0134] The reordering instruction encoder parses the sequence reordering instruction subvector into reordering instructions in the read operation queue. Reordering instructions are presented in a target-source pairing format, where each pair represents moving a read operation command from one position in the queue to another. The encoder converts the values ​​in the subvector into multiple position pairs according to a preset format, resulting in the read operation execution order reordering instruction. This reordering instruction provides more flexible scheduling control compared to simple priority adjustment, allowing for precise adjustment of the execution order of multiple read operation commands.

[0135] Step S436: Input the priority offset and read operation execution order rearrangement instruction into the conflict resolution unit of the scheduling offset branch network, and check whether the priority offset causes multiple physical blocks in the read operation queue to have the same priority offset. If the same occurs, fine-tune the priority offset according to the propagation level depth of the physical block in the read interference propagation influence map, so that the physical block with a smaller propagation level depth gets a higher read operation scheduling priority.

[0136] The conflict resolution unit traverses the priority offsets of all physical blocks, checking for duplicate values. Identical priority offsets prevent the scheduler from distinguishing the read operation priorities of these physical blocks, potentially causing ambiguity in the scheduling order and hindering the accurate communication of the deep reinforcement learning model's fine-grained control over scheduling priorities. If physical blocks with the same priority offset exist, their propagation depth is determined. Propagation depth refers to the longest path length from the source level of the interference propagation impact map to the level of the physical block, reflecting its position in the interference propagation chain. Physical blocks with smaller propagation depths are closer to the source of interference propagation, belonging to the upstream level. Read operations from these upstream physical blocks will interfere with physical blocks at multiple downstream levels. Therefore, read operations from upstream physical blocks should receive higher scheduling priority to execute them as quickly as possible, reducing the accumulation time of interference propagation downstream. Physical blocks with larger propagation depths are closer to the end of interference propagation, primarily bearing the impact of upstream interference. Their own read operations have less impact on other physical blocks, thus their scheduling priority is relatively lower. Physical blocks with the same priority offset are sorted according to their propagation depth, with smaller blocks receiving higher scheduling priority. A fine-tuning increment is added to the priority offset of smaller blocks to further increase their priority; conversely, a fine-tuning increment is removed from the priority offset of larger blocks to relatively decrease their priority. This ensures that the priority offsets of all physical blocks are distinct, and that propagation depth and scheduling priority are negatively correlated, meaning upstream physical blocks receive higher read operation scheduling priority. This fine-tuning mechanism, by prioritizing read operations from upstream physical blocks, reduces the duration of interference propagation downstream, mitigating the cumulative effect of read interference at its source.

[0137] Step S440: Perform instruction fusion processing on the voltage threshold adjustment instruction set and the global scheduling offset strategy. Pair the voltage threshold adjustment instructions and the corresponding read operation scheduling priority offset according to the physical block identifier to generate compensation adjustment instruction entries for each physical block.

[0138] The process iterates through all physical block identifiers. For each physical block, it extracts the suggested voltage threshold adjustment range from the voltage threshold adjustment instruction set and the read operation scheduling priority offset from the global scheduling offset strategy. These two pieces of information are then combined with the physical block identifier to form a compensation adjustment instruction entry. The data structure of the compensation adjustment instruction entry includes the target physical block identifier, the voltage threshold adjustment range, and the read operation scheduling priority offset. Instruction fusion processing integrates two different types of compensation actions into the same instruction set, facilitating subsequent unified execution.

[0139] Step S450: Arrange the compensation adjustment instruction entries of all physical blocks according to the propagation hierarchy of the physical blocks in the read interference propagation effect map to generate a compensation adjustment instruction sequence. Each entry in the compensation adjustment instruction sequence includes the target physical block identifier, the voltage threshold adjustment magnitude value, and the read operation scheduling priority offset.

[0140] The propagation level number of each physical block is obtained from the interference propagation impact map. All compensation adjustment command entries are sorted in ascending order of propagation level number, with entries within the same level arranged according to the physical block identifier. This sorting forms a compensation adjustment command sequence, whose execution order is consistent with the interference propagation direction, prioritizing compensation adjustments for upstream physical blocks. This execution order ensures that compensation for upstream physical blocks takes effect promptly, reducing interference propagation to downstream physical blocks and thus improving the overall compensation effect.

[0141] Step S460: Input the compensation adjustment instruction sequence into the instruction conflict detection unit of the action strategy network, and detect whether there is a conflict in the voltage threshold adjustment amplitude value of multiple compensation adjustment instruction entries for the same physical block group in the compensation adjustment instruction sequence. If there is a conflict, the conflicting voltage threshold adjustment amplitude values ​​are weighted and merged according to the propagation influence weight of the physical block in the read interference propagation influence map to generate a conflict-free compensation adjustment instruction sequence.

[0142] The instruction conflict detection unit traverses the compensation adjustment instruction sequence and groups them according to physical block groups. For instruction entries within the same physical block group, it checks for conflicting voltage threshold adjustment values. A conflict occurs when two instruction entries make inconsistent adjustment requirements for the same physical block or for different physical blocks with strong coupling relationships. For example, within the same physical block group, one instruction requires a significant increase in the voltage threshold, while another instruction requires a slight increase or decrease. This inconsistency may cause the compensation effects to cancel each other out. If a conflict is detected, the propagation influence weight of the physical block involved in the conflict is obtained. The propagation influence weight is obtained from the interference propagation influence map and is defined as the sum of the outgoing edge weights of that physical block. The propagation influence weight reflects the degree of interference influence of that physical block on downstream physical blocks; the larger the weight, the greater the impact of the voltage threshold adjustment on the overall interference propagation. The conflicting adjustment values ​​are weighted and averaged. The weighted average adjustment value is used as the merged adjustment value. The original conflicting instruction entries are removed, and the merged new instruction entries are added. After processing all conflicts, a conflict-free compensation adjustment instruction sequence is generated.

[0143] Step S500: Execute the read operation compensation and adjustment operation of the solid-state drive according to the compensation and adjustment instruction sequence, collect the voltage threshold offset response observation data of each physical block in the subsequent read operation cycle after compensation and adjustment, generate a reward evaluation signal based on the difference between the voltage threshold offset response observation data and the predicted trajectory in the read interference evolution state characterization sequence, and combine the reward evaluation signal, the read interference evolution state characterization sequence and the compensation and adjustment instruction sequence into a new experience sample and store it in the experience playback memory bank.

[0144] As one implementation method, in step S500, the read operation compensation and adjustment operation of the solid-state drive is performed according to the compensation and adjustment instruction sequence, and the voltage threshold offset response observation data of each physical block in the subsequent read operation cycle after compensation and adjustment is collected. Specifically, it may include the following steps S510~S560:

[0145] Step S510: Parse each compensation adjustment instruction entry in the compensation adjustment instruction sequence, extract the target physical block identifier, voltage threshold adjustment magnitude value and read operation scheduling priority offset from each compensation adjustment instruction entry, convert the voltage threshold adjustment magnitude value into the register configuration value of the solid-state drive voltage control unit, and convert the read operation scheduling priority offset into the queue priority configuration parameter of the solid-state drive read operation scheduler.

[0146] The main control chip iterates through each entry in the compensation adjustment instruction sequence, parsing out three fields. The voltage threshold adjustment amplitude value uses a level code format, which the main control chip converts into the corresponding bit configuration in the voltage control unit register. Specifically, the voltage control unit maintains a mapping table that maps different level codes to specific reference voltage offset values. After the main control chip writes the level code into the register, the voltage control unit adjusts the reference voltage applied to the storage unit according to the register value in subsequent read operations. The read operation scheduling priority offset is converted into a queue priority configuration parameter, which is directly written into the scheduler's priority configuration table. The scheduler maintains a priority mapping array, where the array index is the physical block identifier and the array value is the priority offset of that physical block. When a new read operation command arrives, the scheduler calculates the scheduling priority of the command based on the priority offset of the target physical block, for example, the base priority plus the priority offset. The base priority can be preset according to the service quality requirements or latency sensitivity of the read operation.

[0147] Step S520: Write the register configuration value into the corresponding register of the voltage control unit. The voltage control unit adjusts the reference voltage level applied to each physical block during the read operation according to the register configuration value.

[0148] The main control chip writes the register configuration values ​​into the register group of the voltage control unit via the internal bus. The register group of the voltage control unit is addressed according to physical block identifiers, with each physical block corresponding to a set of register bits used to store the voltage adjustment level for that physical block. During subsequent read operations, when a read operation is performed on a physical block, the voltage control unit reads the register configuration value corresponding to that physical block from the register group and selects the corresponding reference voltage level to output to the read circuit based on the configuration value. The adjustment of the reference voltage level directly affects the judgment boundary of the storage cell threshold voltage, thereby compensating for threshold voltage drift caused by read interference.

[0149] Step S530: Write the queue priority configuration parameters into the priority configuration table of the read operation scheduler. The read operation scheduler rearranges the order of the read operation commands to be processed in the queue according to the priority offset of each physical block in the priority configuration table, and executes the read operation commands in the rearranged order.

[0150] The main control chip writes the queue priority configuration parameters into the priority configuration table of the read operation scheduler. The read operation scheduler maintains a queue of pending read operation commands, each command associated with a target physical block identifier and the command's arrival timestamp. The scheduler calculates the final priority of the command based on the priority offset of the physical block in the priority configuration table, by adding the priority offset to the command's base priority (which can be dynamically adjusted according to the command's waiting time). The scheduler rearranges the commands in the queue in descending order of final priority, and when final priorities are the same, they are sorted according to the principle of first-come, first-served arrival timestamps. Then, the read operation commands are executed sequentially in the rearranged order. This priority offset-based scheduling mechanism allows deep reinforcement learning models to change the temporal distribution of read interference by adjusting the execution order of read operations on different physical blocks. For example, it can distribute the read operations of heavily interfered physical blocks to avoid the concentrated accumulation of interference.

[0151] Step S540: Within the preset observation time window after executing the compensation adjustment command sequence, continuously collect the voltage threshold offset measurement values ​​of each physical block after executing the read operation command, and associate and store the voltage threshold offset measurement values ​​with the corresponding physical block identifier and measurement timestamp to generate a voltage threshold offset response observation data set.

[0152] After the compensation adjustment command is executed, a timer is started, and the length of the observation time window is set. The length of the observation time window is set according to the read operation frequency of the solid-state drive and the time scale of interference propagation, and is usually set to several read operation cycles to ensure that the steady-state response after the compensation operation can be observed. Within the observation time window, whenever a read operation command is completed, the main control chip reads the voltage threshold offset measurement value corresponding to that read operation from the voltage control unit. This measurement value reflects the voltage offset required when actually reading data after the compensation operation is executed. At the same time, the physical block identifier and the current timestamp of the read operation command are recorded. These three pieces of information are associated and stored in the response observation data buffer to obtain a response observation record. After the observation time window ends, all records in the buffer constitute the voltage threshold offset response observation data set.

[0153] Step S550: Perform time-series processing on the voltage threshold offset response observation data set, arrange the voltage threshold offset measurement values ​​in the order of physical block identifier and measurement timestamp, and generate the voltage threshold offset response observation data sequence for each physical block in the subsequent read operation cycle.

[0154] The voltage threshold offset response observation dataset is traversed, and records are grouped according to physical block identifiers. Within each physical block group, records are sorted in ascending order of measurement timestamps to obtain the voltage threshold offset measurement sequence for that physical block. Since the read operation execution times for each physical block within the observation time window may differ, and the measurement timestamps of different physical blocks may not be perfectly aligned, the sequence length for each physical block may vary. The sequences of all physical blocks constitute the voltage threshold offset response observation data sequence, which records the trajectory of the voltage threshold offset of each physical block over time after the compensation operation.

[0155] Step S560: Group the voltage threshold offset response observation data sequence according to the propagation hierarchy relationship in the reading interference propagation influence map, extract the distribution characteristics of the voltage threshold offset measurement values ​​of all physical blocks within the same propagation hierarchy, generate the voltage offset response distribution statistics for each propagation hierarchy, and store the response distribution statistics as additional data in association with the voltage threshold offset response observation data sequence.

[0156] The propagation level number of each physical block is obtained from the interference propagation impact map. The voltage threshold offset response observation data sequence is grouped according to the level number. For each physical block group at each level, the mean, variance, maximum, and minimum values ​​of the voltage threshold offset measurements for all physical blocks within that level are calculated. The mean reflects the average level of the overall voltage offset at that level, the variance reflects the degree of offset difference between physical blocks within the level, and the maximum and minimum values ​​reflect the fluctuation range of the voltage offset at that level. These distribution statistics can characterize the impact of compensation operations on different propagation levels from a macroscopic perspective. For example, if the voltage offset decreases significantly after compensation at the upstream level, but remains high at the downstream level, it indicates that the compensation effect has not been fully transmitted. These statistics are associated with and stored with the original response observation data sequence as auxiliary information for subsequent analysis of the compensation effect and adjustment of the reward evaluation signal.

[0157] As one implementation method, step S500, generating a reward evaluation signal based on the difference between the voltage threshold offset response observation data and the predicted trajectory in the read disturbance evolution state characterization sequence, may specifically include the following steps S570~S5120:

[0158] Step S570: Extract the voltage offset prediction trajectory from the read interference evolution state characterization sequence before the current time under the uncompensated state. The voltage offset prediction trajectory includes the predicted voltage threshold offset value of each physical block under the uncompensated state and the prediction timestamp corresponding to the predicted value.

[0159] Each state node in the read interference evolution state characterization sequence contains a value on the predicted voltage offset trajectory. This predicted trajectory is a trend curve of the physical block voltage threshold offset over time, fitted by time-series analysis of historical read operation event streams. Its construction process considers the nonlinear relationship between read operation frequency and voltage offset accumulation, as well as the propagation effects between physical blocks. The predicted voltage offset values ​​and their corresponding prediction timestamps are extracted from all state nodes in this sequence and arranged in timestamp order to obtain the predicted voltage offset trajectory for each physical block. This predicted trajectory serves as a benchmark for evaluating the compensation effect, reflecting the natural evolution trend of the physical block voltage offset without compensation operations.

[0160] Step S580: Compare the measured values ​​in the voltage threshold offset response observation data sequence with the predicted values ​​at the corresponding timestamps in the voltage offset prediction trajectory on a physical block basis, and calculate the offset difference between the measured values ​​and the predicted values ​​at the same timestamp for each physical block.

[0161] For each physical block, its voltage threshold offset response observation data sequence and voltage offset prediction trajectory are obtained. Since the measurement timestamps of the response observation data sequence may not be perfectly aligned with the time grid of the prediction trajectory, time alignment processing is required first. A linear interpolation method is used for the prediction trajectory, and the interpolated predicted value is calculated at the time corresponding to the measurement timestamp. Each measured value in the observation data sequence is paired with the interpolated predicted value, and the difference between the measured value and the predicted value is calculated to obtain the offset difference value of the physical block at that timestamp. The sign of this difference value reflects the effect of the compensation operation relative to the uncompensated state: a negative value indicates that the voltage offset after compensation is lower than the predicted value, i.e., the compensation has a suppressive effect; a positive value indicates that the voltage offset after compensation is higher than the predicted value, i.e., the compensation effect is poor or has produced side effects.

[0162] Step S590: Calculate the weighted difference as the original reward score based on the offset difference value of each physical block and the propagation influence weight of each physical block in the read interference propagation influence map. The offset difference value of the physical block with the larger the propagation influence weight contributes more to the original reward score.

[0163] The propagation influence weight of each physical block is obtained from the interference propagation influence map. This weight is defined as the sum of the weights of all outgoing edges of that physical block. The propagation influence weight reflects the degree of interference influence of that physical block on downstream physical blocks. A larger weight means that the voltage offset change of that physical block will affect more downstream physical blocks. Therefore, these physical blocks should be given higher weights when evaluating the compensation effect. The offset difference values ​​of all physical blocks at each timestamp are multiplied by the propagation influence weight of that physical block to obtain the weighted difference value. All weighted difference values ​​are summed to obtain the weighted difference sum. The original reward score is set as the negative of the weighted difference sum, so that the smaller the offset difference value (i.e., the better the compensation effect), the larger the original reward score. If the voltage offset after compensation is generally lower than the predicted value, the weighted difference sum is negative, the original reward score is positive, and the larger the absolute value, the higher the reward; if the voltage offset after compensation is higher than the predicted value, the original reward score is negative, and a penalty is imposed. It should be noted that the adjustment range of the read operation scheduling priority offset in the compensation action cost penalty value is converted into an equivalent voltage offset influence value through a pre-calibrated conversion factor. After being added to the voltage threshold adjustment range value, a unified cost index in voltage dimension is obtained, so that the net compensation benefit in voltage unit is obtained by subtracting the compensation action cost penalty value (voltage dimension) from the original reward score (voltage dimension). This subtraction operation is completely consistent in physical dimension, ensuring that the physical meaning of the reward evaluation signal is clear and comparable. Similarly, in other parts of the embodiments of the present invention, if there are similar dimension issues, those skilled in the art can unify the dimensions based on common knowledge.

[0164] Step S5100: Calculate the compensation action cost penalty value based on the magnitude of the voltage threshold adjustment magnitude and the adjustment magnitude of the read operation scheduling priority offset in the compensation adjustment instruction sequence. The compensation action cost penalty value is positively correlated with the magnitude of the voltage threshold adjustment magnitude and the adjustment magnitude of the read operation scheduling priority offset.

[0165] Each instruction entry in the compensation adjustment instruction sequence is traversed, extracting the voltage threshold adjustment magnitude and the read operation scheduling priority offset. A larger absolute value for the voltage threshold adjustment magnitude indicates a larger adjustment, resulting in higher costs, as significant voltage adjustments may introduce additional read latency or power consumption, and potentially impact the long-term reliability of the memory cells. Similarly, a larger absolute value for the read operation scheduling priority offset indicates a greater change to the original scheduling order, also resulting in higher costs, as large scheduling offsets may affect the response latency of other read operations, leading to a decrease in service quality. For each instruction entry, the weighted sum of the square of the voltage threshold adjustment magnitude and the absolute value of the priority offset yields the cost value for that entry. The use of the square function ensures that the cost of large adjustments increases faster than that of small adjustments, encouraging the model to adopt a more moderate compensation strategy. The cost values ​​of all entries are summed to obtain the compensation action cost penalty value.

[0166] Step S5110: Subtract the compensation action cost penalty value from the original reward score to generate an intermediate reward evaluation signal.

[0167] The intermediate reward evaluation signal is obtained by subtracting the penalty value for the cost of the compensation action from the original reward score. This signal comprehensively reflects the trade-off between the effectiveness of the compensation operation in suppressing voltage offset and the cost incurred in performing the compensation operation. If the compensation operation achieves a good voltage offset suppression effect with a small cost, the intermediate reward evaluation signal is positive and large; if the compensation operation suppresses the voltage offset but the cost is too high, or the suppression effect is poor, the intermediate reward evaluation signal is small or even negative. This reward design guides deep reinforcement learning models to not only pursue the voltage offset suppression effect when exploring compensation strategies, but also to consider the cost of the compensation action itself, ultimately learning the optimal compensation strategy.

[0168] Step S5120: Input the intermediate reward evaluation signal into the reward smoothing unit. The reward smoothing unit performs time-series smoothing on the intermediate reward evaluation signal according to the trend of the historical reward evaluation signal to generate the final reward evaluation signal. The final reward evaluation signal is used to indicate the overall superiority or inferiority of the voltage offset suppression effect and compensation action cost achieved by this compensation adjustment operation relative to the predicted trajectory without compensation.

[0169] The reward smoothing unit maintains a historical reward queue, storing intermediate reward evaluation signals generated from multiple past compensation operations. Trend analysis is performed on the signals in the historical reward queue, calculating the moving average of the most recent signals. The current intermediate reward evaluation signal is then weighted and averaged with the moving average; the smoothing factor determines the degree of influence of historical information. The weighted average result is output as the final reward evaluation signal. Temporal smoothing can suppress drastic fluctuations in the reward signal caused by measurement noise, transient responses to compensation operations, or environmental changes, making the reward signal more stable and reliable. Simultaneously, smoothing ensures that the reward signal reflects the long-term effect of the compensation strategy rather than the accidental performance of a single operation, helping deep reinforcement learning models learn stable policies.

[0170] As one implementation method, in step S500, the reward evaluation signal, the read interference evolution state representation sequence, and the compensation adjustment instruction sequence are combined into a new experience sample and stored in the experience playback memory bank. Specifically, this may include the following steps S5130~S5180:

[0171] Step S5130: Use the starting state node in the read disturbance evolution state representation sequence as the initial state representation of the experience sample, use the compensation adjustment instruction sequence as the executed action representation of the experience sample, and use the reward evaluation signal as the immediate benefit representation of the experience sample.

[0172] The first state node at a given time point, the initial state node, is extracted from the read interference evolution state representation sequence. This state node corresponds to the interference state before the compensation adjustment operation is performed. The measured voltage offset value, the voltage offset influence value propagated from adjacent physical blocks, and the corresponding compensation operation record in this node are used as the initial state representation. The initial state representation fully describes the interference state of each physical block of the solid-state drive before the compensation operation is performed, including the current voltage offset level, the accumulated propagation influence, and the recently performed compensation operations. The compensation adjustment instruction sequence generated in step S500 is used as the executed action representation, which records the specific compensation actions taken in the current state. The final reward evaluation signal generated in step S5120 is used as the immediate benefit representation, which reflects the benefit obtained after performing the compensation action.

[0173] Step S5140: Extract the subsequent state nodes formed after executing the compensation and adjustment instruction sequence from the read interference evolution state representation sequence as the termination state representation of the empirical sample. The subsequent state nodes include the measured voltage offset values ​​of each physical block after executing the compensation and adjustment instruction sequence, the voltage offset influence values ​​propagated from adjacent physical blocks, and the corresponding compensation operation records.

[0174] The state node at the last time point is extracted from the sequence of disturbance evolution state representations, i.e., the state node at the end of the observation time window after the execution of the compensation adjustment command sequence. This state node records the new disturbance state of each physical block after the compensation operation takes effect, including the measured value of the voltage offset after compensation, the influence value of the voltage offset propagated from adjacent physical blocks after compensation adjustment, and the record of the compensation operation executed during this time period. The termination state representation and the initial state representation form a closed loop of state transition, fully describing the impact of the compensation action on the disturbance state.

[0175] Step S5150: Combine the initial state representation, the executed action representation, the immediate benefit representation, and the final state representation in the form of a quadruple to generate a new experience sample. The quadruple form is used to represent the complete experience segment from the initial state to the final state after executing an action to obtain immediate benefit.

[0176] The initial state representation, the executed action representation, the immediate benefit representation, and the final state representation are sequentially combined into a four-tuple data structure. This four-tuple completely records the closed-loop process of a complete compensation decision and execution. The four-tuple organization is a classic data format for experience replay mechanisms in deep reinforcement learning, which facilitates batch processing and model updates when sampling from the experience replay memory.

[0177] Step S5160: Store the generated new experience samples into the storage queue of the experience replay memory. If the storage queue of the experience replay memory is full, remove the experience sample with the lowest priority weight from the storage queue according to the sample priority weight.

[0178] The experience replay memory maintains a fixed-capacity queue to store recently generated experience samples. When a new experience sample is generated, it is inserted at the end of the queue. If the queue is full before insertion, the sample with the lowest priority is retrieved and removed before the new sample is inserted, maintaining the queue capacity. This queue management method ensures that the experience replay memory always stores the most recently generated experience samples, while retaining high-value samples through a priority mechanism.

[0179] Step S5170: Generate a sample timestamp identifier and sample priority weight for the new experience sample stored in the experience playback memory. The sample priority weight is determined by the magnitude of the reward evaluation signal in the new experience sample and the degree of difference between the measured voltage offset value sequence in the read interference evolution state characterization sequence corresponding to the new experience sample and the historical voltage offset sequence of the same physical block in the historical experience sample library.

[0180] A globally incrementing timestamp is generated for each newly stored sample to distinguish its freshness and record the order in which it was generated. The sample priority weight is calculated by weighting two parts: the first part is the magnitude of the reward evaluation signal; a larger value indicates a better compensation effect, making the sample more valuable for learning the optimal strategy. The second part is the degree of difference between the measured voltage offset sequence in the sample and the historical voltage offset sequences of the same physical block in the historical experience sample library. This difference is obtained by calculating the root mean square error between the measured sequence and the historical average sequence; a larger error indicates that the interference state corresponding to the sample occurs less frequently in historical data, thus having higher exploration value. The two parts are weighted and summed to obtain the sample priority weight, so that samples with both high reward and high exploration value receive higher sampling priority.

[0181] Step S5180: Associate the sample timestamp identifier and sample priority weight with the new experience sample and store them to obtain the experience sample entry with priority identifier. Store it in the experience replay memory and update the sampling probability distribution of the experience replay memory so that the experience sample entry with higher priority weight has a higher probability of being selected in subsequent sampling.

[0182] The sample timestamp identifier and sample priority weight are stored as metadata associated with new experience samples to obtain complete experience sample entries. The experience replay memory updates the sampling probability distribution according to the priority weight of all samples, adopting a priority experience replay mechanism, where the probability of each sample being selected is proportional to its priority weight. In specific implementation, a piecewise linear sampler based on priority weights can be constructed, mapping the priority weights to the sampling interval length; samples with larger weights occupy a larger proportion in the sampling interval. During subsequent training, the deep reinforcement learning model extracts samples from the experience replay memory according to the updated sampling probability distribution to update the model parameters. This priority sampling mechanism allows the model to learn from high-value samples more frequently, accelerating the convergence process, while retaining sampling opportunities for low-priority samples, avoiding the model getting trapped in local optima. By continuously iterating the execution of compensation operations, collecting response data, generating reward signals, and storing experience samples, the deep reinforcement learning model gradually optimizes its policy network parameters, achieving adaptive compensation control for solid-state drive read interference, minimizing the cost of compensation operations while suppressing voltage offset accumulation.

[0183] Based on the foregoing embodiments, this invention provides a solid-state drive read interference compensation device. The units and modules included in the device can be implemented by a processor in a computer device; of course, they can also be implemented by specific logic circuits. In the implementation process, the processor can be a central processing unit (CPU), a microprocessor unit (MPU), a digital signal processor (DSP), or a field programmable gate array (FPGA), etc.

[0184] Figure 2 This is a schematic diagram of the composition structure of a solid-state drive read interference compensation device provided in an embodiment of the present invention, as shown below. Figure 2 As shown, the solid-state drive read interference compensation device 200 includes:

[0185] The record acquisition module 210 is used to acquire read operation event stream records of multiple physical blocks in the solid-state drive during a continuous read operation cycle. The read operation event stream records include the physical block identifier, read operation timestamp, and read operation voltage threshold offset record for each read operation.

[0186] The graph construction module 220 is used to construct a read interference propagation impact graph based on the read operation event stream record and the physical block topology of the solid-state drive, and generate a read interference evolution state characterization sequence for each physical block based on the read interference propagation impact graph and historical compensation operation record. The read interference evolution state characterization sequence includes the voltage offset prediction trajectory of each physical block in the uncompensated state and the voltage offset response feedback information after historical compensation operation.

[0187] The temporal modeling module 230 is used to input the read interference evolution state representation sequence into the recurrent state inference network of the deep reinforcement learning model. Through the gated state update mechanism of the recurrent state inference network, the temporal correlation model of the read interference evolution state representation sequence of each physical block is performed to generate the implicit representation of the interference propagation state of each physical block at the current moment.

[0188] The compensation decision module 240 is used to input the implicit representation of the interference propagation state into the action policy network of the deep reinforcement learning model, and to perform joint decision processing of compensation actions on each physical block group through the multi-branch decision structure of the action policy network, thereby generating a compensation adjustment instruction sequence that includes voltage threshold adjustment instructions and read operation scheduling offset instructions.

[0189] The compensation adjustment module 250 is used to execute read operation compensation adjustment operations on the solid-state drive according to the compensation adjustment instruction sequence, collect voltage threshold offset response observation data of each physical block in subsequent read operation cycles after compensation adjustment, generate a reward evaluation signal based on the difference between the voltage threshold offset response observation data and the predicted trajectory in the read interference evolution state characterization sequence, and combine the reward evaluation signal, the read interference evolution state characterization sequence, and the compensation adjustment instruction sequence into a new experience sample and store it in the experience playback memory bank. The description of the above device embodiment is similar to the description of the above method embodiment and has similar beneficial effects. In some embodiments, the functions or modules included in the device provided by the present invention can be used to execute the methods described in the above method embodiments. For technical details not disclosed in the device embodiments of the present invention, please refer to the description of the method embodiments of the present invention for understanding.

[0190] Figure 3 A hardware entity diagram of a computer system provided as an embodiment of the present invention, such as... Figure 3 As shown, the hardware entity of the computer system 1000 includes a processor 1001 and a memory 1002, wherein the memory 1002 stores a computer program that can run on the processor 1001, and the processor 1001 executes the program to implement the steps in the method of any of the above embodiments.

[0191] The memory 1002 stores computer programs that can run on the processor. The memory 1002 is configured to store instructions and applications that can be executed by the processor 1001. It can also cache data to be processed or already processed (e.g., image data, audio data, voice communication data, and video communication data) of the processor 1001 and various modules in the computer system 1000. It can be implemented by flash memory or random access memory (RAM).

[0192] When the processor 1001 executes the program, it implements any of the steps of the deep reinforcement learning-based solid-state drive read interference compensation method mentioned above. The processor 1001 typically controls the overall operation of the computer system 1000.

Claims

1. A method for read disturb compensation of a solid state drive based on deep reinforcement learning, characterized in that, include: Acquire read operation event stream records of multiple physical blocks in a solid-state drive during a continuous read operation cycle. The read operation event stream records include the physical block identifier, read operation timestamp, and read operation voltage threshold offset record for each read operation. Constructing a read interference propagation impact map based on the read operation event stream records and the physical block topology of the solid-state drive (SSD), specifically including: parsing the read operation timestamp sequence of each physical block in the read operation event stream records, identifying dense and sparse read operation intervals in the read operation timestamp sequence, generating read operation burst mode descriptors for each physical block based on the alternating distribution pattern of dense and sparse read operation intervals; extracting the physical adjacency relationships between different physical blocks in the physical block topology of the SSD, determining the read operation frequency mutual influence coefficient between physical blocks based on the physical adjacency relationships and the read operation burst mode descriptors of each physical block, wherein the read operation frequency mutual influence coefficient includes an influence direction identifier and an influence intensity level identifier; and constructing a directed graph structure with physical blocks as nodes and the read operation frequency mutual influence coefficient as edge weights as the read interference propagation impact map based on the read operation frequency mutual influence coefficient and the physical adjacency relationships between physical blocks. Based on the read interference propagation impact map and historical compensation operation records, a read interference evolution state characterization sequence for each physical block is generated. The read interference evolution state characterization sequence includes the voltage offset prediction trajectory of each physical block in the uncompensated state and the voltage offset response feedback information after historical compensation operations. The read interference evolution state representation sequence is input into the recurrent state inference network of the deep reinforcement learning model. The read interference evolution state representation sequence of each physical block is time-series correlated and modeled through the gated state update mechanism of the recurrent state inference network to generate the implicit representation of the interference propagation state of each physical block at the current time. The implicit representation of the interference propagation state is input into the action policy network of the deep reinforcement learning model. The multi-branch decision structure of the action policy network performs joint decision-making processing on the compensation actions of each physical block group, generating a compensation adjustment instruction sequence that includes voltage threshold adjustment instructions and read operation scheduling offset instructions. The solid-state drive read operation compensation and adjustment operation is executed according to the compensation and adjustment instruction sequence. Voltage threshold offset response observation data of each physical block in subsequent read operation cycles after compensation and adjustment are collected. A reward evaluation signal is generated based on the difference between the voltage threshold offset response observation data and the predicted trajectory in the read interference evolution state characterization sequence. The reward evaluation signal, the read interference evolution state characterization sequence and the compensation and adjustment instruction sequence are combined into a new experience sample and stored in the experience playback memory bank.

2. The method according to claim 1, characterized in that, The step of generating a sequence representing the evolution state of read interference for each physical block based on the read interference propagation impact map and historical compensation operation records includes: Extract the measured voltage threshold offset values ​​of each physical block at multiple historical moments and the corresponding compensation operation records from the historical compensation operation records. Then, associate and store the measured voltage threshold offset values ​​with the corresponding compensation operation records to form a set of historical compensation experience entries. Based on the directed graph structure in the read interference propagation impact map, historical compensation experience entries of other physical blocks that have a propagation relationship with the current physical block are selected from the historical compensation experience entry set. The selected historical compensation experience entries are time-aligned with the historical compensation experience entries of the current physical block itself to generate a read interference evolution state representation sequence for each physical block in a continuous time window. Each state node in the read interference evolution state representation sequence includes the measured value of the voltage offset of the current physical block, the voltage offset impact value propagated from the adjacent physical blocks, and the corresponding compensation operation record. The read interference evolution state characterization sequence is subjected to time-series smoothing. Based on the propagation correlation in the directed graph structure, the voltage offset influence values ​​in the state nodes of adjacent physical blocks are realigned according to the propagation delay time to generate an aligned state node sequence with a unified time reference. Based on the ratio of the measured voltage offset value of each state node in the aligned state node sequence to the propagated voltage offset influence value, the confidence weight of the voltage offset response feedback information in the read interference evolution state characterization sequence of each physical block is updated.

3. The method according to claim 2, characterized in that, The step of parsing the read operation timestamp sequence of each physical block in the read operation event stream record and identifying dense and sparse read operation intervals in the read operation timestamp sequence includes: Extract the read operation timestamp sequence from the read operation event stream record of each physical block, and perform differential processing on the read operation timestamp sequence to obtain the time interval sequence between adjacent read operations. Each interval value in the time interval sequence corresponds to the time difference between two adjacent read operations. Based on the statistical distribution characteristics of the time interval sequence, the threshold for dense read operations and the threshold for sparse read operations are determined. Adjacent read operations with a time interval value not greater than the threshold for dense read operations are divided into consecutive read operation pairs within the dense read operation interval. The idle time period between adjacent read operations with a time interval value not less than the threshold for sparse read operations is marked as the sparse read operation interval. The connectivity merging process is performed on consecutive read operation pairs within the dense read operation interval. Multiple consecutive read operations with adjacent time intervals satisfying the dense condition are merged into a dense read operation interval. The start read operation timestamp, end read operation timestamp, and number of read operations within the interval are generated as dense read operation interval attributes. The time interval between adjacent dense read operation intervals is identified, and the time interval between adjacent dense read operation intervals with a time interval greater than the sparse read operation threshold is divided into sparse read operation intervals. The start idle timestamp, end idle timestamp, and duration of each sparse read operation interval are generated as sparse read operation interval attributes. Based on the dense read operation interval attributes and sparse read operation interval attributes, the dense read operation intervals and sparse read operation intervals are alternately arranged in chronological order to generate a read operation burst mode descriptor for each physical block. The read operation burst mode descriptor contains an alternating sequence of the distribution positions of the dense read operation intervals and the sparse read operation intervals. The read operation burst pattern descriptor of each physical block is input into the burst pattern clusterer. The read operation burst pattern descriptors of different physical blocks are clustered according to the distribution of the start timestamps of the dense read operation intervals and the similarity of the number of read operations within the intervals. Physical block group identifiers with similar read operation burst patterns are generated and the physical block group identifiers are associated and stored in the read operation burst pattern descriptor of each physical block.

4. The method according to claim 2, characterized in that, The construction of a directed graph structure as a read interference propagation impact map, based on the mutual influence coefficient of read operation frequencies and the physical adjacency relationship between physical blocks, with physical blocks as nodes and the mutual influence coefficient of read operation frequencies as edge weights, includes: The spatial proximity level between physical blocks is determined based on the physical adjacency relationship between physical blocks. Physical block pairs with a physical proximity level exceeding a preset proximity threshold are marked as candidate propagation physical block pairs. The candidate propagation physical block pairs include a source physical block identifier and a target physical block identifier. For each candidate propagation physical block, the read operation burst mode descriptor of the source physical block and the read operation burst mode descriptor of the target physical block are obtained. The intensity benchmark value of the mutual influence coefficient of the read operation frequency is calculated based on the degree of overlap between the dense read operation interval of the source physical block and the dense read operation interval of the target physical block on the time axis. Based on the relative positions of the source physical block and the target physical block in the physical topology, the influence direction indicator of the mutual influence coefficient of the read operation frequency is determined. The influence direction indicator is used to indicate the unidirectional propagation direction of the interference from the source physical block to the target physical block. The intensity reference value is combined with the influence direction identifier to generate a directional read operation frequency mutual influence coefficient for each candidate propagation physical block pair. The magnitude of the read operation frequency mutual influence coefficient reflects the degree of influence of the source physical block read operation on the target physical block voltage threshold offset. Using all physical blocks as the node set, candidate propagation physical block pairs as the directed edge set, and the mutual influence coefficient of read operation frequency as the weight value of the directed edge, a directed graph structure is constructed as the read interference propagation influence map. The nodes in the directed graph structure are subjected to hierarchical clustering. Based on the weight distribution of directed edges between nodes, physical blocks are divided into different interference propagation levels. Interference propagation direction identifiers and inter-level transmission delay parameters are generated between each level. The inter-level transmission delay parameters are stored as additional attributes of directed edges in the interference propagation influence map.

5. The method according to claim 1, characterized in that, The process involves inputting the read interference evolution state representation sequence into the recurrent state inference network of the deep reinforcement learning model. Through the gated state update mechanism of the recurrent state inference network, the read interference evolution state representation sequence of each physical block is temporally correlated and modeled to generate the implicit representation of the interference propagation state of each physical block at the current moment, including: Arrange the state nodes of each physical block in the read interference evolution state characterization sequence in a continuous time window according to time order to obtain the state node time sequence chain of each physical block. Each state node in the state node time sequence chain contains the measured value of the voltage offset of the current physical block, the influence value of the voltage offset propagated from the adjacent physical blocks, and the corresponding compensation operation record. The state node time-series chains of each physical block are input in parallel into the input layer of the cyclic state inference network. The input layer allocates an independent input channel for each physical block, and the input channels share parameters to extract the common temporal evolution rules of different types of physical blocks. The gating state update unit of the cyclic state inference network performs time-step recursive calculation of the state node time sequence chain of each physical block. At each time step, the gating state update unit calculates the reset gating signal and update gating signal based on the input state node of the current time step and the hidden state output of the previous time step. Based on the reset gating signal and the update gating signal, the hidden state of the output of the previous time step is selectively forgotten and fused with the current input information to generate the hidden state of the output of the current time step as the intermediate state representation of the physical block at the current moment. The hidden states output by each physical block at the last time step are concatenated to obtain a state tensor with uniform dimensions. The state tensor is then input into the output projection layer of the recurrent state inference network. The output projection layer performs nonlinear transformation on the state tensor to generate the hidden representation of the interference propagation state of each physical block at the current time. The implicit representation of the interference propagation state is input into the feedback adjustment unit of the cyclic state inference network. The feedback adjustment unit generates a state change trend vector based on the change between the implicit representation of the interference propagation state of each physical block at the current time and the implicit representation of the interference propagation state output at the previous time. The state change trend vector is then fused with the implicit representation of the interference propagation state of each physical block at the current time and output.

6. The method according to claim 5, characterized in that, The gated state update unit of the cyclic state inference network performs time-step state recursion calculation on the time-series chain of the state nodes of each physical block, including: The input state node of the current time step is concatenated with the hidden state output of the previous time step to generate a concatenated state vector. The concatenated state vector is then input into the reset gate calculation unit. The reset gate calculation unit performs a linear transformation on the concatenated state vector and generates a reset gate signal through an activation function. The reset gate signal is used to control the proportion of forgotten information in the hidden state of the previous time step. The spliced ​​state vector is input into the update gate calculation unit. After linear transformation of the spliced ​​state vector, an update gate signal is generated through an activation function. The update gate signal is used to control the fusion ratio of the current input information and historical information during state update. The hidden state output from the previous time step is reset according to the reset gating signal. The reset hidden state is then concatenated with the input state node of the current time step and input into the candidate state calculation unit. After linear transformation of the concatenated vector, the candidate hidden state is generated through the activation function. The hidden state and candidate hidden states output at the previous time step are weighted and summed according to the updated gating signal to generate the hidden state output at the current time step. In the weighted summation, the updated gating signal is used as a weight coefficient to control the contribution of the candidate hidden states. The hidden state output at the current time step is passed to the next time step as the input to the gated state update unit of the next time step, and the hidden state output at the current time step is output to the output layer of the recurrent state inference network for subsequent processing. The hidden states output by each physical block at the current time step are grouped according to the propagation hierarchy of the physical blocks in the read interference propagation influence map. The hidden states of all physical blocks within the same propagation hierarchy are aggregated to generate a hierarchical aggregated state vector. The hierarchical aggregated state vector is then broadcast back to each physical block at the corresponding propagation hierarchy as additional hidden state input.

7. The method according to claim 5, characterized in that, The step of concatenating the hidden states output by each physical block at the last time step to obtain a state tensor with uniform dimensions, and then inputting the state tensor into the output projection layer of the recurrent state inference network, includes: Extract the hidden state output at the last time step of the state node time sequence chain for each physical block. The hidden state output at the last time step gathers all the time sequence evolution information of the physical block from the historical start time to the current time. The hidden states output by all physical blocks at the last time step are spliced ​​together according to the spatial arrangement order of the physical blocks in the read interference propagation influence map to generate an initial state tensor with spatial dimension and feature dimension. Each spatial position in the initial state tensor corresponds to the hidden state vector of a physical block. The initial state tensor is input to the spatial feature extraction unit of the output projection layer, and the spatial neighborhood association features between physical blocks are extracted through convolution operation to generate a spatially enhanced state tensor. The spatially enhanced state tensor is input into the nonlinear mapping unit of the projection layer, and the channel-wise feature transformation of the spatially enhanced state tensor is performed through a multi-layer fully connected structure to generate the implicit representation of the intermediate interference propagation state. The intermediate interference propagation state implicit representation is input to the time-series smoothing unit of the projection layer, and a weighted moving average is performed on the intermediate interference propagation state implicit representation at the current time and the interference propagation state implicit representation output at the previous time to generate the interference propagation state implicit representation of each physical block at the current time. The output dimension of the implicit representation of the interference propagation state is set to match the input dimension of the action policy network, so that the implicit representation of the interference propagation state can be directly used as input data for the input layer of the action policy network.

8. The method according to claim 1, characterized in that, The action policy network of the deep reinforcement learning model inputs the implicit representation of the interference propagation state into the action policy network. Through the multi-branch decision structure of the action policy network, joint decision-making processing of compensation actions for each physical block group is performed, generating a compensation adjustment instruction sequence containing voltage threshold adjustment instructions and read operation scheduling offset instructions, including: The hidden representation of the interference propagation state is input into the shared feature extraction backbone network of the action policy network. The shared feature extraction backbone network contains multiple convolutional layers and attention layers, which are used to extract the spatial distribution pattern and the physical block association pattern in the hidden representation of the interference propagation state to generate a shared feature representation. The shared feature representation is input into the voltage adjustment branch network of the action strategy network. A voltage threshold adjustment range recommendation value is generated according to the interference propagation state of each physical block in the shared feature representation. The voltage threshold adjustment range recommendation value is associated with the physical block identifier and stored to form a voltage threshold adjustment instruction set. The shared feature representation is input into the scheduling offset branch network of the action policy network. A global scheduling offset policy is generated based on the interference propagation state distribution of all physical blocks in the shared feature representation. The global scheduling offset policy includes the priority offset of each physical block in the read operation queue and the read operation execution order reordering instruction. The voltage threshold adjustment instruction set is fused with the global scheduling offset strategy. The voltage threshold adjustment instructions and their corresponding read operation scheduling priority offsets are paired according to the physical block identifier to generate a compensation adjustment instruction entry for each physical block. All physical block compensation adjustment instruction entries are arranged according to the propagation hierarchy of the physical blocks in the read interference propagation impact map to generate a compensation adjustment instruction sequence. Each entry in the compensation adjustment instruction sequence includes the target physical block identifier, voltage threshold adjustment magnitude value, and read operation scheduling priority offset. The compensation adjustment instruction sequence is input into the instruction conflict detection unit of the action strategy network to detect whether there is a conflict in the voltage threshold adjustment amplitude value of multiple compensation adjustment instruction entries for the same physical block group in the compensation adjustment instruction sequence. If there is a conflict, the conflicting voltage threshold adjustment amplitude values ​​are weighted and merged according to the propagation influence weight of the physical block in the read interference propagation influence spectrum to generate a conflict-free compensation adjustment instruction sequence.

9. The method according to claim 8, characterized in that, The voltage adjustment branch network that represents the shared features as input action policy network includes: The physical block-level feature extraction layer of the shared feature representation input voltage adjustment branch network performs independent feature extraction based on the feature regions corresponding to each physical block in the shared feature representation, generating a feature vector for each physical block. The feature vector contains the current interference propagation state encoding information of the physical block. Each physical block is configured with an independent action output head for its feature vector. The action output head contains a multilayer perceptron structure. The multilayer perceptron performs a layer-by-layer nonlinear mapping on the input feature vector to generate a probability distribution on the candidate set of voltage threshold adjustment amplitude for that physical block. Sample the voltage threshold adjustment range candidate set of each physical block according to the probability value to obtain the voltage threshold adjustment range recommendation value of each physical block, and convert the voltage threshold adjustment range recommendation value into a voltage adjustment level code that can be recognized by the voltage control unit; Based on the propagation hierarchy in the read interference propagation impact map of physical blocks, physical blocks are divided into multiple physical block groups. After generating a voltage threshold adjustment range recommendation value for each physical block in a physical block group, the propagation impact consistency adjustment is performed on the voltage threshold adjustment range recommendation values ​​within the same physical block group. If the propagation impact consistency adjustment finds that the voltage threshold adjustment range recommendation value of the upstream physical block is greater than the voltage threshold adjustment range recommendation value of the downstream physical block within the same physical block group, the voltage threshold adjustment range recommendation value is redistributed according to the propagation direction so that the voltage threshold adjustment range recommendation value of the upstream physical block is not greater than the voltage threshold adjustment range recommendation value of the downstream physical block. The recommended voltage threshold adjustment range for each physical block after the consistency adjustment is based on the physical block identifier is stored to generate a set of voltage threshold adjustment instructions.

10. A solid-state drive read interference compensation device, characterized in that, include: The record acquisition module is used to acquire read operation event stream records of multiple physical blocks in the solid-state drive during a continuous read operation cycle. The read operation event stream records include the physical block identifier, read operation timestamp, and read operation voltage threshold offset record for each read operation. The graph construction module is used to construct a read interference propagation impact graph based on the read operation event stream records and the physical block topology of the solid-state drive. Specifically, it includes: parsing the read operation timestamp sequence of each physical block in the read operation event stream records; identifying dense and sparse read operation intervals in the read operation timestamp sequence; generating read operation burst mode descriptors for each physical block based on the alternating distribution pattern of dense and sparse read operation intervals; extracting the physical adjacency relationships between different physical blocks in the physical block topology of the solid-state drive; and determining the relationships between physical blocks based on the physical adjacency relationships and the read operation burst mode descriptors of each physical block. A read operation frequency mutual influence coefficient is generated, which includes an influence direction identifier and an influence intensity level identifier. Based on the read operation frequency mutual influence coefficient and the physical adjacency relationship between physical blocks, a directed graph structure is constructed with physical blocks as nodes and the read operation frequency mutual influence coefficient as edge weights as a read interference propagation influence map. Based on the read interference propagation influence map and historical compensation operation records, a read interference evolution state characterization sequence for each physical block is generated. The read interference evolution state characterization sequence includes the voltage offset prediction trajectory of each physical block in the uncompensated state and the voltage offset response feedback information after historical compensation operations. The temporal modeling module is used to input the read interference evolution state representation sequence into the recurrent state inference network of the deep reinforcement learning model, and to perform temporal correlation modeling on the read interference evolution state representation sequence of each physical block through the gated state update mechanism of the recurrent state inference network, so as to generate the implicit representation of the interference propagation state of each physical block at the current moment. The compensation decision module is used to input the implicit representation of the interference propagation state into the action policy network of the deep reinforcement learning model, and to perform joint decision processing of compensation actions on each physical block group through the multi-branch decision structure of the action policy network, thereby generating a compensation adjustment instruction sequence that includes voltage threshold adjustment instructions and read operation scheduling offset instructions. The compensation adjustment module is used to perform read operation compensation adjustment operations on the solid-state drive according to the compensation adjustment instruction sequence, collect voltage threshold offset response observation data of each physical block in subsequent read operation cycles after compensation adjustment, generate a reward evaluation signal based on the difference between the voltage threshold offset response observation data and the predicted trajectory in the read interference evolution state characterization sequence, and combine the reward evaluation signal, the read interference evolution state characterization sequence and the compensation adjustment instruction sequence into a new experience sample and store it in the experience playback memory bank.

Images