Reading threshold predictions in memory devices using deep neural networks

Abstract

An apparatus, system, and method for improving performance of a memory device are described. An example method includes obtaining a plurality of samples corresponding to a probability distribution of each of a plurality of cell voltage distributions of a memory device, each of the plurality of cell voltage distributions corresponding to a read voltage; determining a number of first deep neural networks (DNNs) based on the samples obtained for the plurality of cell voltage distributions; estimating, for each of the first DNNs, one or more parameters of a respective probability distribution based on the plurality of samples; training each of the first DNNs based on the samples and the respective one or more parameters; and training a second DNN based on the samples and the one or more parameters from each of the first DNNs to enable generation of updated read voltage values for retrieving information from the memory device.

Description

Technical Field

[0001] This patent document generally relates to a non-volatile memory device, and more specifically, to a deep neural network in a non-volatile memory device. Background Technology

[0002] Data integrity is a critical characteristic of any data storage device and data transfer. In solid-state memory (e.g., NAND flash memory) devices, information is stored in cells via varying charge levels. During write and read processes, noise is introduced by programming interference and inter-cell interference charge leakage, causing the voltage level to drop over time. This drop is proportional to the amount of charge stored and the number of program / erase (P / E) cycles the cell undergoes. Taking voltage drop into account when determining the read voltage threshold can increase the lifespan of the memory device. Summary of the Invention

[0003] Embodiments of the disclosed technology relate to a method, system, and apparatus for improving the performance of blocks in a memory device. In the example, the performance of the memory device is improved by using a set of first deep neural networks (DNNs) and a second DNN. Each of the first deep neural networks is configured to estimate parameters for different programming voltage (PV) distributions, and the second DNN is configured to integrate the estimated parameter sets from each of the first DNNs and output an optimal read voltage threshold that takes into account memory cell degradation, thereby enabling reliable retrieval of information from various types of memory devices throughout their lifespan.

[0004] In one example, a method for improving the performance of a memory device is described. The method includes: obtaining a plurality of samples corresponding to a probability distribution of each of a plurality of cell voltage distributions of the memory device, wherein each of the plurality of cell voltage distributions corresponds to a read voltage; determining the number of a plurality of first deep neural networks (DNNs) based on the samples obtained for the plurality of cell voltage distributions; estimating one or more parameters of a corresponding probability distribution for each of the plurality of first DNNs based on the plurality of samples; training each of the plurality of first DNNs based on the samples and the corresponding one or more parameters; and training a second DNN based on the samples and the one or more parameters from each of the plurality of first DNNs, such that an updated read voltage value can be generated for retrieving information from the memory device.

[0005] In another example, a method for improving the performance of a memory device is described. The method includes: obtaining samples corresponding to a plurality of probability distributions, each of which is associated with each of a plurality of cell voltage distributions of the memory device, and each of the plurality of cell voltage distributions corresponding to a read voltage; estimating one or more parameters of a plurality of first deep neural networks (DNNs) associated with the corresponding probability distributions based on the samples, each of the plurality of first DNNs having been trained using the samples obtained from the corresponding probability distributions; determining an updated read voltage based on the output of a second DNN, the input of which includes one or more parameters from each of the plurality of first DNNs, and the second DNN having been trained using the samples and one or more parameters from each of the plurality of first DNNs; and applying the updated read voltage to the memory device to retrieve information from the memory device.

[0006] In another example, an apparatus for improving the performance of a memory device is described. The apparatus includes a memory controller, a plurality of first deep neural networks (DNNs), and a second DNN, each first DNN being communicatively connected to the memory controller. The second DNN includes a plurality of inputs and an output connected to the memory controller, each input being connected to the output of a corresponding first DNN. The memory controller is configured to obtain a plurality of samples corresponding to a probability distribution of each of a plurality of cell voltage distributions of the memory device, each of the plurality of cell voltage distributions corresponding to a read voltage; determine the number of the plurality of first deep neural networks (DNNs) based on the samples obtained for the plurality of cell voltage distributions; estimate one or more parameters of the corresponding probability distribution for each of the plurality of first DNNs based on the plurality of samples; train each of the plurality of first DNNs based on the samples and the corresponding one or more parameters; and train the second DNN based on the samples and one or more parameters from each of the plurality of first DNNs, such that an updated read voltage value can be generated for retrieving information from the memory device.

[0007] In another example, an apparatus for improving the performance of a memory device is described. The apparatus includes: a memory controller, a plurality of first deep neural networks (DNNs), and a second DNN, each first DNN being communicatively connected to the memory controller. The second DNN includes a plurality of inputs and an output connected to the memory controller, each input being connected to the output of a corresponding first DNN. The memory controller is configured to obtain samples corresponding to a plurality of probability distributions, each of which is associated with each of a plurality of cell voltage distributions of the memory device, and each of the plurality of cell voltage distributions corresponds to a read voltage. For each of the plurality of first deep neural networks (DNNs) associated with a corresponding probability distribution, one or more parameters of the corresponding probability distribution are estimated based on the samples, each of the first DNNs having been trained using samples obtained from the corresponding probability distribution. An updated read voltage is determined based on the output of the second DNN, the input of which includes one or more parameters from each of the plurality of first DNNs, and the second DNN having been trained using samples and one or more parameters from each of the plurality of first DNNs. The updated read voltage is then applied to the memory device to retrieve information from the memory device.

[0008] In yet another example, these methods can be implemented as processor-executable instructions and stored on a computer-readable program medium.

[0009] The subject matter described in this patent document can be implemented in a specific manner that provides one or more of the following features. Attached Figure Description

[0010] Figure 1 An example of a memory system is shown.

[0011] Figure 2 This is a diagram illustrating an example of a non-volatile memory device.

[0012] Figure 3 This shows the cell voltage level distribution (V) of a non-volatile memory device. th Example diagram.

[0013] Figure 4 This shows the cell voltage level distribution (V) of a non-volatile memory device. th Another example diagram.

[0014] Figure 5 This shows the cell voltage level distribution (V) of the non-volatile memory device before and after programming interference. th Example diagram.

[0015] Figure 6This shows the cell voltage level distribution (V) of a non-volatile memory device as a function of a reference voltage. th Example diagram.

[0016] Figure 7 An example of the theoretical voltage probability distribution for a three-layer cell (TLC) NAND flash memory block is shown.

[0017] Figure 8 This is an example diagram illustrating a typical architecture of a deep neural network (DNN) used to determine the optimal read voltage.

[0018] Figure 9 This is an example diagram illustrating a DNN architecture for determining the optimal read voltage according to the described embodiment.

[0019] Figure 10 An example is shown that uses the probability density function (PDF) of a skewed normal distribution to determine the optimal read voltage.

[0020] Figure 11A and Figure 11B A flowchart of an example method for improving the performance of a memory device is shown.

[0021] Figure 12 This is an example diagram illustrating a storage device that can be configured to implement the described embodiments. Detailed Implementation

[0022] Semiconductor memory devices can be volatile or non-volatile. Volatile semiconductor memory devices perform read and write operations at high speed, but the contents stored in them may be lost when power is off. Non-volatile semiconductor memory devices retain their contents even when power is off. Non-volatile semiconductor memory devices can be used to store content that must be retained regardless of whether they are powered on or off.

[0023] With the increasing demand for high-capacity memory devices, multi-level cell (MLC) or multi-bit memory devices, where each cell stores multiple bits of data, are becoming increasingly common. However, memory cells in MLC non-volatile memory devices must have threshold voltages corresponding to four or more distinguishable data states within a finite voltage window. To improve data integrity in non-volatile memory devices, the level and distribution of the read voltage used to distinguish data states must be adjusted throughout the entire lifespan of the memory device to achieve optimal values ​​during read operations and / or read attempts.

[0024] Several methods have been developed to generate optimal read voltages in non-volatile memory devices. Some methods employ deep neural networks (DNNs), which are more traditionally used in fields such as computer vision (CV), natural language processing (NLP), robotics, and speech. In one example, embodiments of the disclosed technology involve non-volatile memory systems (e.g., such as...) Figures 1 to 6 The DNN architecture used (as described in the text) advantageously reduces the latency and inference time of DNNs.

[0025] Figures 1 to 6 This paper outlines non-volatile memory systems (e.g., flash memory, NAND flash memory) that can implement embodiments of the disclosed technology.

[0026] Figure 1 This is a block diagram of an example memory system 100 implemented based on some embodiments of the disclosed technology. The memory system 100 includes a memory module 110, which can be used to store information for use by other electronic devices or systems. The memory system 100 can be incorporated into other electronic devices and systems (e.g., located on a circuit board). Alternatively, the memory system 100 can be implemented as an external storage device, such as a USB flash drive and a solid-state drive (SSD).

[0027] The memory module 110 included in the memory system 100 may include memory regions (e.g., memory arrays) 102, 104, 106, and 108. Each of the memory regions 102, 104, 106, and 108 may be included in a single memory die or multiple memory dies. The memory die may be included in an integrated circuit (IC) chip.

[0028] Each of memory regions 102, 104, 106, and 108 includes multiple memory cells. Read operations, programming operations, or erase operations can be performed on a memory unit basis. Therefore, each memory unit can include a predetermined number of memory cells. The memory cells in memory regions 102, 104, 106, and 108 can be included in a single memory die or multiple memory dies.

[0029] Memory cells in each of memory regions 102, 104, 106, and 108 may be arranged in rows and columns within a memory unit. Each memory unit may be a physical unit. For example, a group of multiple memory cells may form a memory unit. Each memory unit may also be a logical unit. For example, a memory unit may be a block or a page that can be identified by a unique address, such as a block address or a page address. As another example, memory regions 102, 104, 106, and 108 may include computer memory, which includes storage banks as logical units of data storage. A memory unit may be a storage bank that can be identified by a storage bank address. During a read or write operation, a unique address associated with a particular memory unit can be used to access that particular memory unit. Based on the unique address, information can be written to or retrieved from one or more memory cells within that particular memory unit.

[0030] Memory cells in memory regions 102, 104, 106, and 108 may include non-volatile memory cells. Examples of non-volatile memory cells include flash memory cells, phase-change random access memory (PRAM) cells, magnetoresistive random access memory (MRAM) cells, or other types of non-volatile memory cells. In an example implementation where the memory cells are configured as NAND flash memory cells, read or write operations can be performed on a page-by-page basis. However, erase operations in NAND flash memory are performed on a block-by-block basis.

[0031] Each of the non-volatile memory cells can be configured as a single-level cell (SLC) or a multi-level memory cell. A single-level cell can store one bit of information per cell. A multi-level memory cell can store more than one bit of information per cell. For example, each of the memory cells in memory regions 102, 104, 106, and 108 can be configured as a multi-level cell (MLC) storing two bits of information per cell, a three-level cell (TLC) storing three bits of information per cell, or a four-level cell (QLC) storing four bits of information per cell. In another example, each of the memory cells in memory regions 102, 104, 106, and 108 can be configured to store at least one bit of information (e.g., one bit or more bits of information), and each of the memory cells in memory regions 102, 104, 106, and 108 can be configured to store more than one bit of information.

[0032] like Figure 1As shown, the memory system 100 includes a controller module 120. The controller module 120 includes a memory interface 121 for communicating with the memory module 110, a host interface 126 for communicating with a host (not shown), a processor 124 for running firmware layer code, and a cache 123 and system memory 122 for temporarily or persistently storing executable firmware / instructions and related information, respectively. In some embodiments, the controller module 120 may include an error correction engine 125 to perform error correction operations on information stored in the memory module 110. The error correction engine 125 may be configured to detect / correct single bit errors or multiple bit errors. In another embodiment, the error correction engine 125 may be located within the memory module 110.

[0033] A host device can be an apparatus or system including one or more processors operated to retrieve data from or store data in or write data to the memory system 100. In some embodiments, examples of a host device may include a personal computer (PC), a portable digital device, a digital camera, a digital multimedia player, a television, and a wireless communication device.

[0034] In some implementations, controller module 120 may further include a host interface 126 for communicating with a host. Host interface 126 may include components conforming to at least one host interface specification, including but not limited to Serial Advanced Technology Attachment (SATA), Serial Small Computer System Interface (SAS) specification, and High Speed ​​Peripheral Component Interconnect (PCIe).

[0035] Figure 2 Examples of memory cell arrays implemented based on some embodiments of the disclosed technology are shown.

[0036] In some implementations, the memory cell array may include a NAND flash memory array divided into many blocks, with each block containing a number of pages. Each block includes multiple memory cell strings, and each memory cell string includes multiple memory cells.

[0037] In some implementations where the memory cell array is a NAND flash memory array, read and write (programming) operations are performed on a page-by-page basis, and erase operations are performed on a block-by-block basis. Before programming any page within a block, all memory cells within the same block must be erased simultaneously. In one implementation, the NAND flash memory can use an even / odd bit-line structure. In another implementation, the NAND flash memory can use a full bit-line structure. In an even / odd bit-line structure, even and odd bit lines are interleaved and alternately accessed along each word line, allowing each pair of even and odd bit lines to share peripheral circuitry such as a page buffer. In a full bit-line structure, all bit lines are accessed simultaneously.

[0038] Figure 3 An example of a threshold voltage distribution curve in a multi-layer cell device is shown, where the number of cells in each program / erase state is plotted as a function of the threshold voltage. As shown, the threshold voltage distribution curve includes the erase state with the lowest threshold voltage (denoted as "ER" and corresponding to "11") and three programming states (denoted as "P1", "P2", and "P3", corresponding to "01", "00", and "10", respectively) with read voltages between the states (indicated by dashed lines). In some embodiments, each of the threshold voltage distributions in the program / erase states has a finite width because of the different material properties across the memory array.

[0039] although Figure 3 The multi-level cell device is illustrated by way of example, but each of the memory cells can be configured to store any number of bits per cell. In some embodiments, each of the memory cells can be configured as a single-level cell (SLC) storing one bit of information per cell, or a three-level cell (TLC) storing three bits of information per cell, or a four-level cell (QLC) storing four bits of information per cell.

[0040] When writing more than one data bit into a memory cell, the threshold voltage levels of the memory cell need to be finely arranged because the distance between adjacent distributions decreases. This is achieved by using Incremental Stepped Pulse Programming (ISPP), which involves repeatedly programming memory cells on the same word line using a programming and verification method, where stepped programming voltages are applied to the word line. Each programming state is associated with a verification voltage used in the verification operation, setting the target position for each threshold voltage distribution window.

[0041] Read errors can be caused by distorted or overlapping threshold voltage distributions. Ideal memory cell threshold voltage distributions can be significantly distorted or overlapping due to factors such as program / erase (P / E) cycles, inter-cell interference, and data retention errors, which will be discussed below. In most cases, such read errors can be managed using error correction codes (ECC).

[0042] Figure 4 Examples of ideal threshold voltage distribution curve 410 and distorted threshold voltage distribution curve 420 are shown. The vertical axis represents the number of memory cells, and each memory cell has a specific threshold voltage represented on the horizontal axis.

[0043] For an n-bit multi-cell NAND flash memory, the threshold voltage of each cell can be programmed to be 2. n There are several possible values. In an ideal multi-cell NAND flash memory, each value corresponds to a non-overlapping threshold voltage window.

[0044] Flash memory P / E cycles damage the tunnel oxide of the floating gate in the charge-harvesting layer of the cell transistors, causing threshold voltage shifts and thus gradually reducing the noise margin of the memory device. As the number of P / E cycles increases, the margin between adjacent distributions of different programming states decreases, and eventually the distributions begin to overlap. Threshold voltage programming can lead to data bits stored in memory cells within the overlapping range of adjacent distributions being misinterpreted as values ​​other than the original target value.

[0045] Figure 5 An example of inter-cell interference (ICI) in NAND flash memory is shown. ICI can also cause threshold voltage distortion in flash memory cells. Threshold voltage offsets of memory cell transistors can affect the threshold voltages of their neighboring memory cell transistors through parasitic capacitive coupling between interfering and sacrificial cells. The amount of ICI can be affected by the NAND flash memory bitline structure. In an even / odd bitline structure, memory cells on a word line are alternately connected to even and odd bitlines, and even cells are programmed before odd cells on the same word line. Therefore, even and odd cells experience different amounts of ICI. Cells in a full bitline structure experience less ICI than even cells in an even / odd bitline structure, and the full bitline structure can effectively support high-speed current sensing to improve memory read and verification speeds.

[0046] Figure 5 The dashed lines in the diagram represent the nominal distribution of the P / E states of the considered cell (before programming interference), and "neighboring state values" indicate the values ​​to which neighboring states have been programmed. For example... Figure 5 As shown, if the neighboring state is programmed as P1, the threshold voltage distribution of the considered cell shifts by a specific amount. However, if the neighboring state is programmed as P2, which has a higher threshold voltage than P1, this will result in a larger shift compared to the neighboring state being P1. Similarly, the shift in the threshold voltage distribution is greatest when the neighboring state is programmed as P3.

[0047] Figure 6 An example of retention errors in NAND flash memory is illustrated by comparing normal threshold voltage distributions and offset threshold voltage distributions. Data stored in NAND flash memory becomes corrupted over time, and this is known as a data retention error. Retention errors are caused by charge loss stored in the floating gate or charge-harvesting layer of the cell transistor. Memory cells with more program-erase cycles are more likely to experience retention errors due to losses in the floating gate or charge-harvesting layer. Figure 6 In the example, the leftward offset is revealed by comparing the voltage distribution of the top row (before the damage) and the distribution of the bottom row (damaged by the retained error).

[0048] In NAND-based storage systems (e.g., Figures 1 to 6 In the example shown, a deep neural network (DNN) can be used to estimate the threshold voltage. The threshold voltage is the intersection of the threshold voltage distributions of any two adjacent programmed verification (PV) levels.

[0049] Figure 7 An example of the theoretical voltage probability distribution of a three-layer cell (TLC) NAND flash memory block is shown. As illustrated, the voltage distributions of eight TLC blocks (in...) Figure 7 The PV distributions (denoted as PV0, PV1, ..., PV8) are not the same. Therefore, in order to determine the optimal read voltage for this TLC NAND flash memory block, the probability distribution (e.g., probability density function (pdf) or cumulative distribution function (CDF)) used to model each of the PV distributions should not be the same as the DNN used to determine the parameters of the probability distribution.

[0050] However, as Figure 8 As shown, existing implementations use the same DNN for adjacent PV distributions. As illustrated, PV L and PV R These are two adjacent PV levels. It is applied to PV L A series of sampled voltage values, Corresponding to in PV of the threshold voltage distribution sampled at the voltage level L The cumulative density function (CDF) value, It is applied to PV R A series of sampled voltage values, Corresponding to in PV sampled at voltage in R The CDF value of the threshold voltage distribution, Θ L It is PV L The parameter set, Θ R It is PV R The parameter set, and "optimal V" t "It is PV" L and PV R The voltage at the crossover point between the threshold voltage distributions.

[0051] In other words, each of the DNNs is configured similarly and uses the same weights to determine the parameters of the potential probability distribution of the PV distribution. The estimated parameters from each of the first set of DNNs (810a and 810b) are input into a second DNN (820), which is configured to determine the optimal read voltage. Using the same DNN weights for each of the first set of DNNs generally results in suboptimal estimated read voltages and is inefficient in accurately retrieving information from the memory device coupled to the first set of DNNs (810a and 810b) and the second DNN (820).

[0052] continue Figure 8 The description states that a typical implementation plan requires three components:

[0053] (1) Parameter distribution, which simulates the distribution of cell threshold voltages for all PV levels. It is assumed that this parameter distribution models the relationship between the sampled voltage, CDF sample, and channel parameter Θ for each PV level;

[0054] (2) A set of first DNNs (810a and 810b, denoted as DNN1) are used to estimate the distribution parameters of the CDF samples given by NAND measurements. In the example, DNN1 is trained on a dataset containing multiple sampled voltages x and CDF values ​​CDF(x) of the parameter distribution at the sampled voltages x. Figure 8 As shown, the sampled voltage and CDF value are input to DNN1, and DNN1 is configured to output the distributed parameter Θ; and

[0055] (3) The second DNN (820, denoted as DNN2) predicts the optimal voltage threshold (V) based on a set of outputs from the first DNN. t In the example, DNN2 includes the parameter Θ. L and Θ R The dataset is used as input for training and is configured to output the optimal threshold voltage (V). t ).

[0056] Different probability distributions and different DNN weights can be used for each of the PV distributions to determine the optimal read voltage, instead of using the same probability distribution and DNN weights for all PV distributions. However, this approach may lead to increased complexity. For example, a four-cell (QLC) NAND flash block using four bit layers per cell would require 16 DNNs to obtain the parameters for all QLC PV distributions.

[0057] For example, such as Figure 9As shown, embodiments of the disclosed technology use a first set of DNNs dedicated to PVs, i.e., the weights of each in the DNNs are different when the latent probability distributions are different. This is to estimate any two adjacent PVs, PVs... i and PV j The optimal read threshold, estimated DNN1 is for PV i The design, which can be compared with the estimate The DNN1 is different. In this paper, we take a TLC NAND flash memory block as an example, where i∈{0,1,2,3,4,5,6} and j=i+1.

[0058] exist Figure 7 In the examples shown, the parameters of the PV distribution are significantly different. For instance, if we assume a skewed normal distribution (with parameters (ξ, ψ, α)) is used, the PV1 distribution has parameter Θ. PV1 = [0.2, 0.1, 0], while the PV7 distribution has the parameter Θ. PV7 =[3.5,0.3,-3]. Therefore, using Figure 8 The conventional implementation shown, which assumes the same probability distribution and uses the same weights for each of the first set of DNNs, may result in suboptimal read voltages.

[0059] The described embodiments use different probability distributions and different DNN weights for different potential cell voltage distributions. For example, PV0, PV5, PV6, and PV7 can use different probability distributions and different weights, and PV1, PV2, PV3, and PV4 can use another different probability distribution and another different DNN weights because their PV distributions are very similar. Here, five probability distributions and DNNs can be used to determine the optimal read voltage for a TLC NAND flash memory block. Alternatively, a common probability distribution and DNN can be used for PV1, PV2, PV3, PV4, and PV5, which reduces the required number of DNNs to four.

[0060] In this way, embodiments of the disclosed technology advantageously offer a trade-off between complexity and performance. As previously mentioned, using a different DNN for each PV distribution, especially if a single DNN is used for very similar cell voltage distributions, can significantly increase the complexity of the implementation and may not provide a significant performance advantage compared to using a smaller number of different DNNs.

[0061] Figure 9 This shows the PV levels of two adjacent PV levels. i and PV j The read threshold estimation for (i∈{0,1,2,3,4,5,6},j=i+1) is modeled with a parameterized distribution of cell level. and The skewed normal distribution. The pdf of the skewed normal distribution is given by the following formula:

[0062]

[0063] in and These are the PDF and CDF of the normal (Gaussian) distribution, respectively.

[0064] In this paper, the optimal reading threshold is x. * , which makes

[0065] Figure 10 An example is shown that uses the probability density function (PDF) of a skewed normal distribution to determine the optimal read voltage. As shown, the optimal read voltage is the intersection of two skewed normal distributions, the parameters of which are determined using a DNN with different weights.

[0066] In some embodiments, a differentiable loss function (e.g., mean squared error) is defined to measure the error between the predicted and actual distribution parameters Θ, and a training algorithm such as stochastic gradient descent is used to optimize the weights of each in the first DNN (910a and 910b). Similarly, a differentiable loss function is defined to measure the error between the optimal threshold voltage and the output of the second DNN (920), and a training algorithm such as stochastic gradient descent is used to obtain the weights of DNN2. In the example, a set of DNN1 and DNN2 are designed and trained independently on different datasets. In another example, they can be trained using the same sample set.

[0067] continue Figure 9 The description of the embodiment for determining the optimal read voltage for adjacent PV distributions requires four components:

[0068] (1) Parameter distribution, which simulates the distribution of the cell threshold voltage for each of the adjacent PV levels. Each parameter distribution models the relationship between the sampled voltage, CDF sample, and parameter Θ for each PV level;

[0069] (2)PV i dedicated According to the input Output in It is applied to PV i The sampled voltage value, and Is PV of the threshold voltage distribution sampled at the voltage level i CDF value;

[0070] (3)PVj dedicated According to the input Output in It is applied to PV j The sampled voltage value, and Is PV of the threshold voltage distribution sampled at the voltage level j The CDF value; and

[0071] (4) PV-insensitive DNN2 was used based on Calculate the optimal V t .

[0072] In some embodiments, for PV i dedicated Three readings (denoted as read0, read1, and read2) are used to collect input samples.

[0073]

[0074] In some embodiments, PV i dedicated The weights can be multiple matrices. and bias vector The form is given, and the latent distribution parameters can be calculated as:

[0075]

[0076] In this paper, R(·) denotes the activation function. In the example, the rectified linear unit (ReLU) activation function can be used. In another example, the sigmoid function can be used.

[0077] Similarly, for PV j dedicated Three readings (denoted as read0, read1, and read2) are used to collect input samples.

[0078]

[0079] and The weights can be multiple matrices. and bias vector The form of allows the latent distribution parameters to be calculated as:

[0080]

[0081] Where R(·) represents the activation function.

[0082] exist and In the example above, you can choose the size of matrix A and vector b, as well as the value of M (representing the number of layers in the DNN), to make a trade-off between complexity and performance.

[0083] In some embodiments, and referring to Figure 9 The input to DNN2(920) is from Output and from Output The cascade is given by the following formula:

[0084]

[0085] The weights of DNN2(920) can be multiple matrices. and bias vector The optimal read threshold is calculated as follows:

[0086]

[0087] In this paper, R(·) denotes the activation function. In the example, the rectified linear unit (ReLU) activation function can be used. In another example, the sigmoid function can be used.

[0088] In some embodiments, the parameter distribution may be a skewed normal distribution. However, if the potential cell voltage distribution does not conform to a skewed normal distribution, a different distribution can be used to determine a more accurate read voltage. For example, the parameter distribution may be a Gaussian distribution and determined by a first DNN (e.g., Figure 9 The parameters for each output in 910a and 910b can be the mean and variance.

[0089] In some embodiments, multiple PV distributions can share a single first DNN. As previously stated... Figure 7 As discussed in the context above, a single DNN1 is used to determine common latent distribution parameters for modeling PV1, PV2, PV3, and PV4. In this case, samples from each of these distributions can initially be used to train the corresponding DNN, and then the parameters are estimated to determine the optimal readout voltage.

[0090] In some embodiments, samples from the PV distributions can be compared in pairs to determine whether the two distributions can be represented by a common DNN1. This determination can be made by computing a metric for the two distributions and comparing it to a threshold. In one example, this metric is based on the Fisher Information metric computed for the two distributions. In another example, this metric is based on the Kullback-Leibler divergence or Jensen-Shannon divergence computed for the two distributions. In yet another example, other metrics measuring the similarity between two probability distributions can be used to determine whether two(or more) distributions should share a DNN1.

[0091] The threshold for comparison with the metric can be chosen based on a trade-off between complexity and performance. For example, if a low-complexity implementation is preferred, a threshold can be chosen to allow slightly different PV distributions to share DNN1, which will adversely affect performance but maintain low complexity. Alternatively, a threshold can be chosen to allow only substantially identical PV distributions to share DNN1, which will result in better performance but higher complexity.

[0092] Figure 11A A flowchart of an example method 1100 for improving the performance of a memory device is shown. Method 1100 includes, in operation 1105, obtaining a plurality of samples corresponding to a probability distribution of each of a plurality of cell voltage distributions of the memory device, wherein each of the plurality of cell voltage distributions corresponds to a read voltage.

[0093] Method 1100 includes: in operation 1110, determining the number of a plurality of first deep neural networks (DNNs) based on samples obtained for a plurality of unit voltage distributions.

[0094] Method 1100 includes: in operation 1115, for each of the plurality of first DNNs, estimating one or more parameters of the corresponding probability distribution based on the plurality of samples.

[0095] Method 1100 includes: in operation 1120, training each of a plurality of first DNNs based on samples and corresponding one or more parameters.

[0096] Method 1100 includes: in operation 1125, training a second DNN based on samples and one or more parameters from each of a plurality of first DNNs, such that updated read voltage values ​​can be generated for retrieving information from a memory device.

[0097] In some embodiments, operations 1105 to 1125 may be controlled by a memory controller (e.g., Figure 12 The memory controller 1220 in the memory can be used to execute this, and it can also be used with a DNN (e.g., Figure 12 A deep neural network (1230) is used to estimate the updated multiple read voltages.

[0098] In some embodiments, determining the number of multiple first DNNs includes: for each pair of probability distributions, performing comparisons between the corresponding multiple samples to generate a metric indicating the distance between the pair of probability distributions, and determining the number of multiple first DNNs based on the metric.

[0099] In some embodiments, the metric is based on Fisher information metrics.

[0100] In some embodiments, the metric is based on Kullback-Leibler divergence or Jensen-Shannon divergence.

[0101] In some embodiments, estimating one or more parameters of the probability distribution includes measuring the error between the actual values ​​of one or more parameters and the predicted values ​​of one or more parameters.

[0102] In some embodiments, the error is mean square error.

[0103] In some embodiments, method 1100 further includes the operation of determining one or more weights of a corresponding first DNN using a stochastic gradient descent algorithm based on the corresponding error.

[0104] The disclosed embodiments include an apparatus for improving the performance of a memory device. The apparatus includes a memory controller, a plurality of first deep neural networks (DNNs), and a second DNN, each first DNN being communicatively connected to the memory controller. Each second DNN includes a plurality of inputs and an output connected to the memory controller, each input being connected to the output of a corresponding first DNN. The memory controller is configured to obtain a plurality of samples of a probability distribution of each of a plurality of cell voltage distributions corresponding to the memory device, each of the plurality of cell voltage distributions corresponding to a read voltage; determine the number of the plurality of first deep neural networks (DNNs) based on the samples obtained for the plurality of cell voltage distributions; estimate one or more parameters of a corresponding probability distribution for each of the plurality of first DNNs based on the plurality of samples; train each of the plurality of first DNNs based on the samples and the corresponding one or more parameters; and train a second DNN based on the samples and one or more parameters from each of the plurality of first DNNs, such that an updated read voltage value can be generated for retrieving information from the memory device.

[0105] Figure 11BA flowchart of a method 1150 for improving the performance of a memory device is shown. Method 1150 includes, at operation 1155, obtaining samples corresponding to a plurality of probability distributions, each of which is associated with each of a plurality of cell voltage distributions of the memory device, and each of the plurality of cell voltage distributions corresponding to a read voltage.

[0106] Method 1150 includes: in operation 1160, for each of a plurality of first deep neural networks (DNNs) associated with a corresponding probability distribution, estimating one or more parameters of the corresponding probability distribution based on samples, each of the first DNNs having been trained using samples obtained from the corresponding probability distribution.

[0107] Method 1150 includes: in operation 1165, determining an updated read voltage based on the output of a second DNN, the input of which includes one or more parameters from each of a plurality of first DNNs, and the second DNN has been trained using samples and one or more parameters from each of the plurality of first DNNs.

[0108] Method 1150 includes: in operation 1170, applying an updated read voltage to the memory device to retrieve information from the memory device.

[0109] In some embodiments, operations 1155, 1160, and 1165 may be controlled by a memory controller (e.g., Figure 12 The operation 1170 can be performed by the memory device or the memory controller (e.g., respectively) 1220. Figure 12 The flash memory 1210 and memory controller 1220 in the memory are used to execute the operation, and a DNN (e.g., Figure 12 The deep neural network 1230 in the model estimates the updated multiple read voltages.

[0110] In some embodiments, the sample corresponds to a cumulative distribution function associated with the voltage distributions of multiple cells.

[0111] In some embodiments, the probability distribution is a skewed normal distribution.

[0112] In some embodiments, the probability distribution is a Gaussian distribution.

[0113] In some embodiments, the memory device is a three-level cell (TLC) NAND flash memory device.

[0114] The disclosed embodiments include an apparatus for improving the performance of a memory device. The apparatus includes: a memory controller, a plurality of first deep neural networks (DNNs), and a second DNN, each first DNN being communicatively connected to the memory controller. Each second DNN includes a plurality of inputs and an output connected to the memory controller, each input being connected to the output of a corresponding first DNN. The memory controller is configured to obtain samples corresponding to a plurality of probability distributions, each of which is associated with each of a plurality of cell voltage distributions of the memory device, and each of the plurality of cell voltage distributions corresponds to a read voltage. For each of the plurality of first deep neural networks (DNNs) associated with a corresponding probability distribution, one or more parameters of the corresponding probability distribution are estimated based on the samples, each of the first DNNs having been trained using samples obtained from the corresponding probability distribution. An updated read voltage is determined based on the output of the second DNN, the input of which includes one or more parameters from each of the plurality of first DNNs, and the second DNN having been trained using the samples and one or more parameters from each of the plurality of first DNNs. The updated read voltage is then applied to the memory device to retrieve information from the memory device.

[0115] Figure 12 This is an example diagram illustrating a storage device that can be configured to implement the described embodiments. (Refer to...) Figure 12 The data storage device 1200 may include flash memory 1210, a memory controller 1220, and a deep neural network (DNN) 1230. The memory controller 1220 may control the flash memory 1210 and the DNN 1230 in response to control signals input externally to the data storage device 1200. In the data storage device 1200, the flash memory 1210 may be configured to be the same as or substantially the same as a non-volatile memory device. That is, the flash memory 1210 may use different read voltages to read data from selected memory cells to output data to the memory controller 1220. The DNN 1230 may be used to estimate different read voltages based on the embodiments described in this document. In the example, the DNN 1230 may be... Figure 9 The DNN shown in the image.

[0116] In some embodiments, the data storage device 1200 may be a memory card device, an SSD device, a multimedia card device, an SD card, a memory stick device, an HDD device, a hybrid drive device, or a USB flash drive device. For example, the data storage device 1200 may be a card that conforms to the standards of user devices such as digital cameras and personal computers.

[0117] The embodiments of the subject matter and functional operation described in this patent document can be implemented in various systems, digital electronic circuits, or as computer software, firmware, or hardware, including the structures disclosed in this specification and their structural equivalents, or one or more combinations thereof. Embodiments of the subject matter described in this specification can be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a tangible and non-transitory computer-readable medium for use by or to control the operation of a data processing device. The computer-readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of substances affecting machine-readable propagation signals, or one or more combinations thereof. The terms "data processing unit" or "data processing device" encompass all devices, apparatuses, and machines for processing data, including, for example, programmable processors, computers, or multiple processors or multiple computers. In addition to hardware, the device may also include code that creates an operating environment for the computer program in question, for example, code constituting processor firmware, a protocol stack, a database management system, an operating system, or one or more combinations thereof.

[0118] A computer program (also called a program, software, software application, script, or code) can be written in any programming language, including compiled or interpreted languages, and can be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for a computing environment. A computer program does not necessarily correspond to a file in a file system. A program can be stored as part of a file containing other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinating files (e.g., a file storing one or more modules, subroutines, or portions of code). Computer programs can be deployed to run on a single computer or on multiple computers located at a site or distributed across multiple sites and interconnected by a communication network.

[0119] The processes and logic described in this specification can be executed by one or more programmable processors running one or more computer programs, manipulating input data and generating output. The processes and logic can also be executed by dedicated logic circuitry, and the device can be implemented as dedicated logic circuitry, such as an FPGA (Field-Programmable Gate Array) or an ASIC (Application-Specific Integrated Circuit).

[0120] Processors suitable for executing computer programs include, for example, general-purpose and special-purpose microprocessors, and any one or more processors of any type of digital computer. Typically, a processor receives instructions and data from read-only memory or random access memory, or both. The basic components of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include a mass storage device (e.g., a magnetic disk, magneto-optical disk, or optical disk) or be operatively coupled to a mass storage device to receive data from or transfer data to one or more mass storage devices for storing data, or both. However, a computer does not need to have such a device. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including semiconductor memory devices such as EPROM, EEPROM, and flash memory devices. The processor and memory may be supplemented by or integrated within dedicated logic circuitry.

[0121] While this patent document contains numerous details, these should not be construed as limiting the scope of any invention or what may be claimed, but rather as descriptions of features that may be specific to particular embodiments of a particular invention. Specific features described in the context of individual embodiments in this patent document may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments. Furthermore, although the foregoing features may be described, or even initially stated, as functioning in a particular combination, in some cases, one or more features in the claimed combination may be removed from the combination, and the claimed combination may be for sub-combinations or variations thereof.

[0122] Similarly, although operations are depicted in a specific order in the accompanying drawings, this should not be construed as requiring these operations to be performed in the specific order or sequence shown, or requiring all illustrated operations to be performed to achieve the desired result. Furthermore, the separation of various system components in the embodiments described in this patent document should not be construed as requiring such separation in all embodiments.

[0123] Only a few implementation schemes and examples are described, and other implementation schemes, enhancements and variations can be made based on what is described and shown in this patent document.

Claims

1. A method for improving the performance of a memory device, comprising: Obtain a plurality of samples corresponding to a probability distribution of each of a plurality of cell voltage distributions of the memory device, wherein each of the plurality of cell voltage distributions corresponds to a read voltage; Based on the multiple samples obtained for the voltage distribution of the multiple units, the number of multiple first deep neural networks, i.e., multiple first DNNs, is determined. For each of the plurality of first DNNs, one or more parameters of the corresponding probability distribution are estimated based on the plurality of samples; Each of the plurality of first DNNs is trained based on the plurality of samples and one or more corresponding parameters; and A second DNN is trained based on the plurality of samples and one or more parameters from each of the plurality of first DNNs, so as to generate updated read voltage values ​​to retrieve information from the memory device.

2. The method of claim 1, wherein determining the number of the plurality of first DNNs comprises: For each pair of probability distributions, comparisons are performed between the corresponding multiple samples to generate a metric indicating the distance between each pair of probability distributions; and Based on the metric, the number of the plurality of first DNNs is determined.

3. The method of claim 2, wherein the metric is based on Fisher information metric.

4. The method of claim 2, wherein the metric is based on Kullback-Leibler divergence or Jensen-Shannon divergence.

5. The method of claim 1, wherein estimating the one or more parameters of the probability distribution comprises: Measure the error between the actual value and the predicted value of the one or more parameters.

6. The method of claim 5, wherein the error is a mean square error.

7. The method of claim 5, further comprising: Based on the corresponding error, a stochastic gradient descent algorithm is used to determine one or more weights of the corresponding first DNN.

8. A method for improving the performance of a memory device, comprising: Samples are obtained corresponding to a plurality of probability distributions, each of which is associated with each of a plurality of cell voltage distributions of the memory device, and each of the plurality of cell voltage distributions corresponds to a read voltage; For each of a plurality of first deep neural networks, i.e. a plurality of first DNNs, associated with a corresponding probability distribution, one or more parameters of the corresponding probability distribution are estimated based on the samples, each of the plurality of first DNNs having been trained using samples obtained from the corresponding probability distribution; The updated read voltage is determined based on the output of the second DNN, the input of which includes one or more parameters from each of the plurality of first DNNs, and the second DNN has been trained using the samples and the one or more parameters from each of the plurality of first DNNs; and The updated read voltage is applied to the memory device to retrieve information from the memory device.

9. The method of claim 8, wherein the sample corresponds to a cumulative distribution function associated with the plurality of unit voltage distributions.

10. The method of claim 8, wherein the probability distribution is a skewed normal distribution.

11. The method of claim 10, wherein the probability density function of the skewed normal distribution is... It is given by the following formula: in Corresponding to one or more of the parameters, and and .

12. The method of claim 8, wherein the probability distribution is a Gaussian distribution.

13. The method of claim 8, wherein the memory device is a three-level cell NAND flash memory device, i.e., a TLC NAND flash memory device.

14. An apparatus for improving the performance of a memory device, comprising: Memory controller; Multiple first deep neural networks, i.e., multiple first DNNs, each of which is communicatively connected to the memory controller; as well as The second DNN includes multiple inputs and an output connected to the memory controller, with each input connected to the corresponding output of the first DNN. The memory controller mentioned above: Obtain a plurality of samples corresponding to a probability distribution of each of a plurality of cell voltage distributions of the memory device, wherein each of the plurality of cell voltage distributions corresponds to a read voltage; Based on the multiple samples obtained for the multiple unit voltage distributions, the number of the multiple first DNNs is determined; For each of the plurality of first DNNs, one or more parameters of the corresponding probability distribution are estimated based on the plurality of samples; Each of the plurality of first DNNs is trained based on the plurality of samples and one or more corresponding parameters; and A second DNN is trained based on the plurality of samples and one or more parameters from each of the plurality of first DNNs, so as to generate updated read voltage values ​​to retrieve information from the memory device.

15. The device of claim 14, wherein, as part of determining the number of the plurality of first DNNs, the memory controller further: For each pair of probability distributions, comparisons are performed between the corresponding multiple samples to generate a metric indicating the distance between each pair of probability distributions; and Based on the metric, the number of the plurality of first DNNs is determined.

16. The device of claim 15, wherein the metric is based on Fisher information metric, Kullback-Leibler divergence, or Jensen-Shannon divergence.

17. An apparatus for improving the performance of a memory device, comprising: Memory controller; Multiple first deep neural networks, i.e., multiple first DNNs, each of which is communicatively connected to the memory controller; as well as The second DNN includes multiple inputs and an output connected to the memory controller, with each input connected to the corresponding output of the first DNN. The memory controller mentioned above: Samples are obtained corresponding to a plurality of probability distributions, each of which is associated with each of a plurality of cell voltage distributions of the memory device, and each of the plurality of cell voltage distributions corresponds to a read voltage; For each of the plurality of first DNNs associated with a corresponding probability distribution, one or more parameters of the corresponding probability distribution are estimated based on the samples, each of the plurality of first DNNs having been trained using samples obtained from the corresponding probability distribution; The updated read voltage is determined based on the output of the second DNN, the input of which includes one or more parameters from each of the plurality of first DNNs, and the second DNN has been trained using the samples and the one or more parameters from each of the plurality of first DNNs; and The updated read voltage is applied to the memory device to retrieve information from the memory device.

18. The device of claim 17, wherein the sample corresponds to a cumulative distribution function associated with the plurality of unit voltage distributions.

19. The device according to claim 17, wherein the probability distribution is a skewed normal distribution.

20. The device of claim 17, wherein the memory device is a three-level cell NAND flash memory device, i.e., a TLC NAND flash memory device.

Images