Decoding method, neural network training method and memory device

Abstract

Disclosed are methods of decoding, methods of training neural networks, and memory devices. Methods, apparatuses, non-transitory computer-readable media, and systems for using error-correcting codes in a memory device using a neural network are described. Embodiments of the methods, apparatuses, non-transitory computer-readable media, and systems may receive a signal from a physical channel, wherein the signal is based on modulated symbols representing information bits encoded using an error-correcting coding scheme; extract features from the signal using a feature extractor trained using probability data collected from the physical channel; and decode the information bits using a neural network decoder using the extracted features as input.

Description

Technical Field

[0001] The following relates generally to decoding error correction codes using neural networks and, more particularly, to decoding error correction codes using neural networks in memory devices. Background Art

[0002] Memory devices are commonly used electronic components for storing data. NAND flash memory devices allow for the storage of multiple bits of data per memory cell, offering improvements in manufacturing costs and performance. Memory cells that store multiple bits of data are referred to as multi-layer memory cells. Multi-layer memory cells divide the threshold voltage range of a memory cell into several voltage states, and use the memory cell voltage levels to extract the data value written to the memory cell.

[0003] In some cases, data may be encoded before being programmed into a memory device to provide the ability to correct read errors. Polar coding schemes are examples of efficient error correction code (ECC) schemes. However, many decoders assume an additive white Gaussian noise (AWGN) model, but the noise introduced by flash memory cells may not conform to the AWGN model. Furthermore, decoding polar decoding schemes can be computationally intensive. Therefore, there is a need in the art for improved systems and methods for decoding information stored in memory devices. Summary of the Invention

[0004] Methods, apparatus, non-transitory computer-readable media, and systems for decoding error-correcting codes using a neural network are described. Embodiments of the method, apparatus, non-transitory computer-readable media, and system may receive a signal from a physical channel, wherein the signal is based on modulated symbols representing information bits encoded using an error-correcting coding scheme; extract features from the signal using a feature extractor trained using probability data collected from the physical channel; and decode the information bits using a neural network decoder using the extracted features as input.

[0005] Methods, apparatus, non-transitory computer-readable media, and systems for decoding error-correcting codes using a neural network are described. Embodiments of the method, apparatus, non-transitory computer-readable media, and system may prepare a labeled training set comprising ground truth information bits and voltage levels read from a memory device, wherein the voltage levels are based on encoding the ground truth information bits using a set of modulation symbols and programming the encoded ground truth information bits to the memory device; identify a plurality of conditional probability mass functions based on the labeled training set, wherein each of the plurality of conditional probability mass functions describes a voltage level probability of one of the set of modulation symbols; read the voltage levels from the memory device; extract features from the voltage levels based at least in part on the conditional probability mass functions; apply a neural network decoder to the features to generate predicted information bits; compare the predicted information bits to the ground truth information bits; and update the neural network decoder based on the comparison.

[0006] Memory devices and methods for decoding error correction codes using neural networks are described. Embodiments of the memory devices and methods may include: a plurality of memory cells configured to store encoded information bits; a feature extractor configured to extract features from read voltage levels based on probability data collected for the memory device; and a neural network decoder configured to decode the encoded information using the extracted features as input. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Features of the present inventive concept will become more apparent by describing in detail exemplary embodiments of the present inventive concept with reference to the attached drawings.

[0008] Figure 1 is a block diagram illustrating an implementation of a data processing system including a memory system according to an exemplary embodiment of the inventive concept.

[0009] Figure 2 is a diagram showing an exemplary embodiment according to the inventive concept Figure 1 Block diagram of the memory system.

[0010] Figure 3 According to an exemplary embodiment of the inventive concept Figure 1 Detailed block diagram of a non-volatile memory device.

[0011] Figure 4 According to an exemplary embodiment of the inventive concept Figure 2 Block diagram of a memory cell array.

[0012] Figure 5 According to an exemplary embodiment of the inventive concept Figure 4 Circuit diagram of a memory block of a memory cell array.

[0013] Figure 6 An example of a memory device including a neural network decoder according to aspects of the present disclosure is shown.

[0014] Figure 7 An example of a process of decoding an error correction code using a neural network according to aspects of the present disclosure is shown.

[0015] Figure 8 An example of a process of training a neural network for decoding error-correcting codes according to aspects of the present disclosure is shown. DETAILED DESCRIPTION

[0016] The present disclosure describes systems and methods for error correction, and more particularly, describes systems and methods for decoding error-correcting codes using neural networks.

[0017] This disclosure describes systems and methods for decoding error-correcting codes using neural networks, and more particularly, in memory devices.

[0018] NAND programming is a complex process based on applying voltage to the memory cell. However, the cell voltage may be affected by variables such as current voltage level, pulse power, and inter-cell interference. The cell voltage may also be affected by inhibited cell disruption, word line (WL) coupling, and cell retention. In addition, the results of writing to a NAND device are random. For example, the data may also be noisy, leading to observed problems.

[0019] A decoder is a logic circuit used to convert binary information from a coded input into a unique output. Multiple inputs and multiple outputs can be used. Channel coding performs encoding and decoding operations on a data stream to correct for communication errors (such as interference or noise). Channel coding is performed on a first data stream using a first set of operations at a transmitting device, and on a second data stream using a second set of operations at a receiving device. Channel coding can be either block or convolutional codes.

[0020] Polar coding is a subset of channel coding. Polar coding is a block coding method in which data blocks and bits have a set size. Block manipulation is performed at the transmitter and receiver. Operations known as channel combining and channel splitting are performed. First, channel combining assigns and / or maps combinations of symbols to channels. Then, channel splitting performs a transform operation, converting the symbol combinations into time-domain vectors used in the decoder. The decoding operation, together with the encoding operation, estimates the time-domain bit stream. This converts the blocks of bits and channels into a polarized bit stream at the receiver.

[0021] Polar codes have been shown to achieve capacity in additive white Gaussian noise (AWGN) channels and comparable performance to low-density parity-check (LDPC) codes using a finite-length code with a successive cancellation list (SCL) decoder and a cyclic redundancy check (CRC). Because each bit is decoded sequentially, the SCL decoder has high latency. A neural network (NN) decoder can be used with polar codes. For relatively short codes, the NN decoder performs similarly to the SCL decoder. Because all bits are decoded in parallel, the NN decoder performs with very low latency.

[0022] The neural network decoder of the present disclosure can be used with an AWGN channel having binary phase shift keying (BPSK) modulation. However, many flash memory channels do not utilize AWGN channels, and the disclosed embodiments do not necessarily assume AWGN channels. Each cell can store multiple bits, so the modulation can be pulse amplitude modulation (PAM). For example, 4 PAMs can be used for 2 bits per cell, and 8 PAMs can be used for 3 bits per cell.

[0023] Hereinafter, exemplary embodiments of the inventive concept will be described more fully with reference to the accompanying drawings.Throughout the drawings, like reference numerals may refer to like elements.

[0024] It will also be understood that the terms "first," "second," "third," etc. are used herein to distinguish one element from another, and that the elements are not limited by these terms. Thus, a "first" element in an exemplary embodiment may be described as a "second" element in another exemplary embodiment.

[0025] It should be understood that descriptions of features or aspects within each exemplary embodiment should typically be considered as available for other similar features or aspects in other exemplary embodiments, unless the context clearly dictates otherwise.

[0026] As used herein, the singular forms are intended to include the plural forms as well, unless the context clearly indicates otherwise.

[0027] Here, when a value is described as being approximately equal to another value or being substantially the same as another value or being substantially equal to another value, it will be understood that these values ​​are equal to each other within the measurement error, or if measurably unequal, then as one of ordinary skill in the art will understand, these values ​​are close enough in value to be functionally equal to each other. For example, the term "about" as used herein includes the stated value and means within an acceptable range of deviations of a particular value determined by one of ordinary skill in the art taking into account the measurement in question and the errors associated with the measurement of the particular quantity (i.e., the limitations of the measurement system). For example, "about" can mean within one or more standard deviations as one of ordinary skill in the art understands. In addition, it will be understood that although according to exemplary embodiments, a parameter may be described herein as having "approximately" a certain value, as one of ordinary skill in the art will understand, the parameter may be the exact certain value or an approximate certain value within the measurement error.

[0028] Exemplary Memory System

[0029] Figure 1 is a block diagram illustrating an implementation of a data processing system including a memory system according to an exemplary embodiment of the inventive concept.

[0030] Reference Figure 1 , the data processing system 10 may include a host 100 and a memory system 200 . Figure 1 The memory system 200 shown in FIG. 2 may be used in various systems including data processing functions. The various systems may be various devices, including, for example, mobile devices (such as smart phones or tablet computers). However, the various devices are not limited thereto.

[0031] The memory system 200 may include various types of memory devices. Here, the exemplary embodiments of the inventive concept will be described as including a memory device as a non-volatile memory. However, the exemplary embodiments are not limited thereto. For example, the memory system 200 may include a memory device as a volatile memory.

[0032] According to an exemplary embodiment, the memory system 200 may include a non-volatile memory device (e.g., a read-only memory (ROM), a magnetic disk, an optical disk, a flash memory, etc.). The flash memory may be a memory that stores data based on a change in the threshold voltage of a metal oxide semiconductor field effect transistor (MOSFET), and may include, for example, NAND and NOR flash memories. The memory system 200 may be implemented using a memory card including a non-volatile memory device (e.g., an embedded multimedia card (eMMC), a secure digital (SD) card, a micro SD card, or a universal flash memory (UFS)), or may be implemented using, for example, an SSD including a non-volatile memory device. Here, the configuration and operation of the memory system 200 will be described assuming that the memory system 200 is a non-volatile memory system. However, the memory system 200 is not limited thereto. The host 100 may include, for example, a system-on-chip (SoC) application processor (AP) mounted on, for example, a mobile device, or a central processing unit (CPU) included in a computer system.

[0033] As described above, the host 100 may include the AP 110. The AP 110 may include various intellectual property (IP) blocks. For example, the AP 110 may include a memory device driver 111 that controls the memory system 200. The host 100 may communicate with the memory system 200 to transmit commands related to memory operations and receive confirmation commands in response to the transmitted commands.

[0034] The memory system 200 may include, for example, a memory controller 210 and a memory device 220. The memory controller 210 may receive commands related to memory operations from the host 100, generate internal commands and internal clock signals using the received commands, and provide the internal commands and internal clock signals to the memory device 220. The memory device 220 may store write data in a memory cell array in response to the internal commands, or may provide read data to the memory controller 210 in response to the internal commands.

[0035] The memory device 220 includes a memory cell array that retains the data stored therein even when the memory device 220 is not powered. The memory cell array may include, for example, NAND or NOR flash memory, magnetoresistive random access memory (MRAM), resistive random access memory (RRAM), ferroelectric random access memory (FRAM), or phase change memory (PCM) as memory cells. For example, when the memory cell array includes NAND flash memory, the memory cell array may include multiple blocks and multiple pages. Data can be programmed and read in units of pages, and data can be erased in units of blocks. Figure 4 An example of a memory block included in a memory cell array is shown in FIG.

[0036] Figure 2is a diagram showing an exemplary embodiment according to the inventive concept Figure 1 A block diagram of a memory system 200 is shown.

[0037] Reference Figure 2 Memory system 200 includes memory device 220 and memory controller 210. Memory controller 210 may also be referred to herein as a controller circuit. Memory device 220 may perform a write operation, a read operation, or an erase operation under the control of memory controller 210.

[0038] The memory controller 210 may control the memory device 220 according to a request received from the host 100 or an internally specified schedule. The memory controller 210 may include a controller core 211, an internal memory 214, a host interface block 215, and a memory interface block 216.

[0039] The controller core 211 may include a memory control core 212 and a machine learning core 213, and each of these cores may be implemented by one or more processors. The memory control core 212 may control and access the memory device 220 according to a request received from the host 100 or an internally specified schedule. The memory control core 212 may manage and execute various metadata and codes for managing or operating the memory system 200.

[0040] The machine learning core 213 may be used to perform training and inference of a neural network designed to perform noise cancellation on the memory device 220 , as described in further detail below.

[0041] The internal memory 214 may be used as, for example, a system memory used by the controller core 211, a cache memory storing data of the memory device 220, or a buffer memory temporarily storing data between the host 100 and the memory device 220. The internal memory 214 may store a mapping table MT indicating a relationship between a logical address assigned to the memory system 200 and a physical address of the memory device 220. The internal memory 214 may include, for example, a DRAM or an SRAM.

[0042] In one exemplary embodiment, a neural network (such as, for example, Figure 6The neural network described herein may be included in a computer program stored in the internal memory 214 of the memory controller 210 or in the memory device 220. The computer program including the neural network may be executed by the machine learning core 213 to perform noise reduction on the data stored in the memory device 220. Therefore, according to an exemplary embodiment, the memory system 200 may perform noise reduction on the data stored in the memory device 220 during a normal read operation of the memory device 220. That is, after the manufacture of the memory system 200 is completed, during the normal operation of the memory system 200, specifically, during the normal read operation of the memory system 200 in which data is read from the memory device 220, the neural network locally stored and executed in the memory system 200 may be used to reduce the noise of the data stored in the memory device 220 being read, and the noise-reduced data may be read out from the memory device 220.

[0043] The host interface block 215 may include components (eg, physical blocks) for communicating with the host 100 . The memory interface block 216 may include components (eg, physical blocks) for communicating with the memory device 220 .

[0044] Hereinafter, the operation of the memory system 200 over time will be described. When power is supplied to the memory system 200 , the memory system 200 may perform initialization with the host 100 .

[0045] The host interface block 215 may provide the memory control core 212 with a first request REQ1 received from the host 100. The first request REQ1 may include a command (e.g., a read command or a write command) and a logical address. The memory control core 212 may convert the first request REQ1 into a second request REQ2 suitable for the memory device 220.

[0046] For example, the memory control core 212 may convert the format of the command. The memory control core 212 may refer to the mapping table MT stored in the internal memory 214 to obtain the address information AI. The memory control core 212 may convert the logical address into the physical address of the memory device 220 by using the address information AI. The memory control core 212 may provide the second request REQ2 suitable for the memory device 220 to the memory interface block 216.

[0047] The memory interface block 216 may register the second request REQ2 from the memory control core 212 in the queue. The memory interface block 216 may send the request first registered in the queue to the memory device 220 as a third request REQ3.

[0048] When the first request REQ1 is a write request, the host interface block 215 may write data received from the host 100 to the internal memory 214. When the third request REQ3 is a write request, the memory interface block 216 may transmit data stored in the internal memory 214 to the memory device 220.

[0049] When the data is completely written, the memory device 220 may send a third response RESP3 to the memory interface block 216. In response to the third response RESP3, the memory interface block 216 may provide the memory control core 212 with a second response RESP2 indicating that the data is completely written.

[0050] After the data is stored in the internal memory 214 or after the second response RESP2 is received, the memory control core 212 may send a first response RESP1 indicating request completion to the host 100 through the host interface block 215 .

[0051] When the first request REQ1 is a read request, the read request may be transmitted to the memory device 220 through the second request REQ2 and the third request REQ3. The memory interface block 216 may store the data received from the memory device 220 in the internal memory 214. When the data is completely transmitted, the memory device 220 may transmit a third response RESP3 to the memory interface block 216.

[0052] When receiving the third response RESP3, the memory interface block 216 may provide the memory control core 212 with a second response RESP2 indicating that the data is completely transmitted. When receiving the second response RESP2, the memory control core 212 may transmit the first response RESP1 to the host 100 through the host interface block 215.

[0053] The host interface block 215 may transmit data stored in the internal memory 214 to the host 100. In one exemplary embodiment, if data corresponding to the first request REQ1 is stored in the internal memory 214, transmission of the second request REQ2 and the third request REQ3 may be omitted.

[0054] Figure 3 According to an exemplary embodiment of the inventive concept Figure 1 Detailed block diagram of the non-volatile memory device 220. Figure 2 , the memory device 220 may include, for example, a memory cell array 221 , a control logic 222 , a voltage generating unit 223 , a row decoder 224 , and a page buffer 225 .

[0055] The memory cell array 221 may be connected to one or more string selection lines SSL, multiple word lines WL, one or more ground selection lines GSL, and multiple bit lines BL. The memory cell array 221 may include a plurality of memory cells disposed at intersections between the multiple word lines WL and the multiple bit lines BL.

[0056] The control logic 222 may receive a command CMD (e.g., an internal command) and an address ADD from the memory controller 210, and may receive a control signal CTRL for controlling various functional blocks within the memory device 220 from the memory controller 210. Based on the command CMD, the address ADD, and the control signal CTRL, the control logic 222 may output various control signals for writing data to the memory cell array 221 or reading data from the memory cell array 221. In this manner, the control logic 222 may control the overall operation of the memory device 220.

[0057] Various control signals output by the control logic 222 may be provided to the voltage generation unit 223, the row decoder 224, and the page buffer 225. For example, the control logic 222 may provide the voltage control signal CTRL_vol to the voltage generation unit 223, provide the row address X-ADD to the row decoder 224, and provide the column address Y-ADD to the page buffer 225.

[0058] The voltage generation unit 223 can generate various voltages for performing programming, reading, and erasing operations on the memory cell array 221 based on the voltage control signal CTRL_vol. For example, the voltage generation unit 223 can generate a first drive voltage VWL for driving a plurality of word lines WL, a second drive voltage VSSL for driving a plurality of string select lines SSL, and a third drive voltage VGSL for driving a plurality of ground select lines GSL. In this case, the first drive voltage VWL can be a program voltage (e.g., a write voltage), a read voltage, an erase voltage, a pass voltage, or a program verification voltage. In addition, the second drive voltage VSSL can be a string select voltage (e.g., a turn-on voltage or a turn-off voltage). In addition, the third drive voltage VGSL can be a ground select voltage (e.g., a turn-on voltage or a turn-off voltage).

[0059] The row decoder 224 may be connected to the memory cell array 221 through a plurality of word lines WL and may activate a portion of the plurality of word lines WL in response to a row address X-ADD received from the control logic 222. For example, in a read operation, the row decoder 224 may apply a read voltage to a selected word line and a pass voltage to unselected word lines.

[0060] In a program operation, row decoder 224 may apply a program voltage to a selected word line and a pass voltage to unselected word lines. In one exemplary embodiment, in at least one of a plurality of program loops, row decoder 224 may apply a program voltage to a selected word line and additionally selected word lines.

[0061] The page buffer 225 may be connected to the memory cell array 221 through a plurality of bit lines BL. For example, in a read operation, the page buffer 225 may operate as a sense amplifier that outputs data stored in the memory cell array 221. Alternatively, in a program operation, the page buffer 225 may operate as a write driver that writes desired data to the memory cell array 221.

[0062] Figure 4 and Figure 5 An example of implementing memory system 200 using a three-dimensional flash memory is shown. The three-dimensional flash memory may include three-dimensional (e.g., vertical) NAND (e.g., VNAND) memory cells. The implementation of memory cell array 221 including three-dimensional memory cells is described below. Each of the memory cells described below may be a NAND memory cell.

[0063] Figure 4 According to an exemplary embodiment of the inventive concept Figure 2 1 is a block diagram of a memory cell array 221.

[0064] Reference Figure 4 According to an exemplary embodiment, the memory cell array 221 includes a plurality of memory blocks BLK1 to BLKz. Each of the memory blocks BLK1 to BLKz has a three-dimensional structure (e.g., a vertical structure). For example, each of the memory blocks BLK1 to BLKz may include a structure extending from a first direction to a third direction. For example, each of the memory blocks BLK1 to BLKz may include a plurality of NAND strings extending from a second direction. The plurality of NAND strings may be arranged, for example, from the first direction to the third direction.

[0065] Each of the NAND strings is connected to a bit line BL, a string selection line SSL, a ground selection line GSL, a word line WL, and a common source line CSL. That is, each of the memory blocks BLK1 to BLKz may be connected to a plurality of bit lines BL, a plurality of string selection lines SSL, a plurality of ground selection lines GSL, a plurality of word lines WL, and a common source line CSL. Figure 5 The memory blocks BLK1 to BLKz are described in further detail.

[0066] Figure 5 is a circuit diagram of a memory block BLKi according to an exemplary embodiment of the inventive concept. Figure 5 Show Figure 4An example of one of the memory blocks BLK1 to BLKz in the memory cell array 221. Figure 5 An example with 6 word lines and 6 memory cells is shown, but this is merely an example and any number of word lines and memory cells may be used.

[0067] The memory block BLKi may include a plurality of cell strings CS11 to CS41 and CS12 to CS42. The plurality of cell strings CS11 to CS41 and CS12 to CS42 may be arranged in column and row directions to form columns and rows. Each of the cell strings CS11 to CS41 and CS12 to CS42 may include a ground selection transistor GST, memory cells MC1 to MC6, and a string selection transistor SST. The ground selection transistor GST, memory cells MC1 to MC6, and string selection transistor SST included in each of the cell strings CS11 to CS41 and CS12 to CS42 may be stacked in a height direction substantially perpendicular to the substrate.

[0068] Columns of multiple cell strings CS11 to CS41 and CS12 to CS42 may be connected to different string selection lines SSL1 to SSL4, respectively. For example, the string selection transistors SST of cell strings CS11 and CS12 may be commonly connected to string selection line SSL1. The string selection transistors SST of cell strings CS21 and CS22 may be commonly connected to string selection line SSL2. The string selection transistors SST of cell strings CS31 and CS32 may be commonly connected to string selection line SSL3. The string selection transistors SST of cell strings CS41 and CS42 may be commonly connected to string selection line SSL4.

[0069] The rows of multiple cell strings CS11 to CS41 and CS12 to CS42 can be connected to different bit lines BL1 and BL2, respectively. For example, the string selection transistors SST of the cell strings CS11 to CS41 can be commonly connected to the bit line BL1. The string selection transistors SST of the cell strings CS12 to CS42 can be commonly connected to the bit line BL2.

[0070] Columns of multiple cell strings CS11 to CS41 and CS12 to CS42 may be connected to different ground selection lines GSL1 to GSL4, respectively. For example, the ground selection transistors GST of cell strings CS11 and CS12 may be commonly connected to ground selection line GSL1. The ground selection transistors GST of cell strings CS21 and CS22 may be commonly connected to ground selection line GSL2. The ground selection transistors GST of cell strings CS31 and CS32 may be commonly connected to ground selection line GSL3. The ground selection transistors GST of cell strings CS41 and CS42 may be commonly connected to ground selection line GSL4.

[0071] Memory cells arranged at the same height from the substrate (or ground selection transistor GST) may be connected in common to a single word line, and memory cells arranged at different heights from the substrate may be connected to different word lines WL1 to WL6, respectively. For example, memory cell MC1 may be connected in common to word line WL1. Memory cell MC2 may be connected in common to word line WL2. Memory cell MC3 may be connected in common to word line WL3. Memory cell MC4 may be connected in common to word line WL4. Memory cell MC5 may be connected in common to word line WL5. Memory cell MC6 may be connected in common to word line WL6. The ground selection transistors GST of cell strings CS11 to CS41 and CS12 to CS42 may be connected in common to a common source line CSL.

[0072] Neural Network Decoder

[0073] Figure 6 An example of a memory device 600 including a neural network decoder 625 according to aspects of the present disclosure is shown. Embodiments of the present disclosure use a flash-polarized neural network decoder.

[0074] The memory device 600 can store information for various programs and applications on the computing device. For example, the information may include data used to run the operating system. The memory may include both volatile memory and non-volatile memory. Volatile memory may include random access memory (RAM), and non-volatile memory may include read-only memory (ROM), flash memory, electrically erasable programmable read-only memory (EEPROM), digital tape, hard disk drive (HDD), and solid-state drive (SSD). The memory may include any combination of readable and / or writable volatile memory and / or non-volatile memory, as well as other possible storage devices.

[0075] According to one embodiment, the neural network decoder (e.g., a flash NAND polarized neural network decoder, hereinafter also referred to as a decoder) 625 includes a multilayer perceptron (MLP). An MLP is a feedforward neural network typically composed of multiple layers of perceptrons. Each of the composed perceptron layers may include an input layer, one or more hidden layers, and an output layer. Each node may include a nonlinear activation function. The MLP may be trained using backpropagation (i.e., calculating the gradient of a loss function with respect to its parameters).

[0076] According to various embodiments, the memory device 600 may receive a signal from a physical channel, wherein the signal is based on modulated symbols representing information bits encoded using an error correction coding scheme. In some examples, the physical channel includes the NAND memory device 600. The memory device 600 may also detect a voltage level from the NAND memory device 600, wherein the detected voltage level includes a noisy realization of the modulated symbol.

[0077] In some examples, the modulated symbols comprise channel inputs to the NAND memory device 600, and the signals comprise channel outputs from the NAND memory device 600. The memory device 600 can also program test symbols into the NAND memory and detect voltage levels of the NAND memory, where the probability data is based on a conditional probability of detecting the voltage level given each test symbol.

[0078] Thus, memory device 600 can read a voltage level from memory device 600. In some examples, the voltage level read from memory device 600 corresponds to a noisy version of a symbol from the set of modulated symbols. Memory device 600 can include an encoder 605, a modulation component 610, a scaling component 615, a memory unit 620, and a decoder 625.

[0079] The encoder 605 may encode the information bits using an error correction coding scheme. During training, the encoder 605 may encode the ground truth information bits using an error correction coding scheme. In some examples, the error correction coding scheme includes a polar coding scheme. The encoder 605 may include an encoding component configured to encode the information bits to generate encoded information bits. For example, Figure 6 As shown in , (n, k) may indicate that the encoder 605 may encode n-bit data using a block size of k bits.

[0080] Error correction coding (ECC) and decoding operations can be performed on the data stream to correct communication errors (such as interference or noise). Polar codes are linear block error correction codes based on multiple recursive concatenations of short kernel codes that transform the physical channel into multiple virtual external channels. Virtual channels tend to have high or low reliability (i.e., they are polarized). Data bits are assigned to the most reliable channels, and unreliable channels are "frozen" or set to 0.

[0081] The modulation component 610 can apply a modulation scheme to the encoded information bits to generate modulated symbols. For example, the programming voltage selection of the cell can be performed by a pulse amplitude modulation (PAM) modulation algorithm, wherein the encoded bits are divided into multiple groups of multiple bits per cell. For example, in a triple-layer cell (TLC), the number of bits per cell is 3. Each group of cells is called a symbol. For example, a symbol with bits 010 is equal to 2. For N bits per cell, the threshold voltage Vth dynamic range is divided into 2 Ntarget voltages. For 3 bits per cell, the Vth dynamic range is divided into 8 target voltages. Each target voltage is mapped to a symbol using a Gray code, where only a single bit changes between adjacent target voltages. For example, if the dynamic range is between -3V and 4V, a modulation of -3V is 111, -2V is 110, -1V is 100, 0V is 101, 1V is 001, 2V is 000, 3V is 010, and 4V is 011 can be used.

[0082] The scaling component 615 can be configured to scale the output of the modulation component 610 based on the dynamic voltage range of the memory cell 620. The memory cell 620 can be configured as described above with reference to Figures 1 to 5 The coded information bits are stored.

[0083] Decoder 625 can use a neural network to decode the information bits using the extracted features as input. In some examples, the error correction coding scheme includes a polar coding scheme. In some examples, the modulated symbols are based on a phase amplitude modulation (PAM) scheme or a binary phase shift keying (BPSK) modulation scheme.

[0084] Decoder 625 may include probability information 630, feature extractor 635, and ANN 640. Feature extractor 635 may extract features from the signal, and feature extractor 635 is trained using probability data collected from the physical channel. In some examples, the probability data collected from the physical channel includes one or more probability mass functions. In some examples, the probability data does not assume an additive white Gaussian noise (AWGN) model.

[0085] The feature extractor 635 can identify a set of conditional probability mass functions based on the labeled training set, where each of the conditional probability mass functions describes the voltage level probability of one modulated symbol in the set of modulated symbols. The feature extractor 635 can also extract features from the voltage levels based on the conditional probability mass functions. Thus, the feature extractor 635 can be configured to extract features from the read voltage levels based on the probability data collected for the memory device 600.

[0086] Embodiments of the neural network decoder 625 include an artificial neural network (ANN). An ANN is a hardware or software component that includes multiple connected nodes (also known as artificial neurons), which can loosely correspond to neurons in the human brain. Each connection or edge can send a signal from one node to another node (just like a physical synapse in the brain). When a node receives a signal, it can process the signal and then send the processed signal to other connected nodes. In some cases, the signals between nodes include real numbers, and the output of each node can be calculated as a function of the weighted sum of its inputs. Each node and edge can be associated with one or more node weights that determine how the signal is processed and sent.

[0087] During the training process, these weights can be adjusted to improve the accuracy of the results (i.e., by minimizing a loss function that corresponds in some way to the difference between the current result and the target result). The weights of the edges can increase or decrease the strength of the signal transmitted between the nodes. In some cases, the nodes can have a threshold below which no signal is sent at all. Nodes can also be grouped into layers. Different layers can perform different transformations on their inputs. The initial layer can be called the input layer, and the last layer can be called the output layer. In some cases, the signal can pass through a particular layer multiple times.

[0088] Decoding process

[0089] Figure 7 An example of a process for decoding an error-correcting code using a neural network according to aspects of the present disclosure is shown. In some examples, these operations may be performed by a system including a processor that executes a set of code to control functional elements of a device. Additionally or alternatively, dedicated hardware may be used to perform the process. Generally, these operations may be performed according to the methods and processes described based on aspects of the present disclosure. For example, the operations may consist of various sub-steps or may be performed in combination with other operations described herein.

[0090] At operation 700, the system receives a signal from a physical channel, wherein the signal is based on modulated symbols representing information bits encoded using an error correction coding scheme. In some cases, the operations of this step may be represented as described with reference to Figure 6 The memory device described or by reference to Figure 6 The described memory device performs.

[0091] For each unit of n bits (n _bpc ) memory device and code length n, the codeword is stored in Each cell j has a read V j For each unit, 2 n_bpc Histogram P(V j |L i) is created offline and evaluated online, where L i is the i-th level, i∈{0, 1, ...2^n_bpc-1}, j∈{1, 2, ...n -cells Here and throughout this disclosure, the term "level" may be used in a general sense as an example of a symbol. For example, a level may represent a combination of voltage levels on one or more cells.

[0092] Feature extractor calculates n_cells*2 n_bpc The probability {P(V j |L i )}, j=1:n_cells; i=0:2^n_bpc-1. In addition, the feature extractor calculates the word line index, P / E cycle index, retention time and pillar index. The probability can be evaluated using a histogram, which is calculated offline.

[0093] In operation 705, the system extracts features from the signal using a feature extractor trained using probability data collected from the physical channel. In some cases, the operation of this step may be represented as referring to Figure 6 The feature extractor described or by reference Figure 6 Describes the feature extractor implementation.

[0094] Feature extraction can be an online process (i.e., performed when data is to be read from memory), where, given a new measurement value V j , V j According to the voltage value, it is quantized into voltage bin B m , where m∈{1, 2, ...M}. Each bin represents all voltage measurements between some lower and upper limits: B m-1 ≤V j <B m The number of bins M is configurable and defines the voltage resolution. j Use 2 n_bpc The probability mass function is used to evaluate, and the result is obtained by 2 n_bpc A vector consisting of Each element is a scalar number. The feature vector is the input to the prediction model (neural network).

[0095] The histogram values ​​can be normalized before being used so that each bin does not represent a simple count (i.e., how many samples in the training set fall into that bin given a particular symbol). Instead, the bin represents the ratio between that count and the total count (i.e., the number of samples given a particular symbol). This normalization will create a probability vector of M elements, which sums to 1.

[0096] The collection of training sets and the creation of probability mass functions are performed offline once, and feature extraction is performed repeatedly online. j and the probability mass function vector can be used to create a new 2 n_bpc +1-element character vector.

[0097] In operation 710, the system uses a neural network decoder to decode the information bits using the extracted features as input. In some cases, the operation of this step can be represented as referring to Figure 6 The decoder described or by reference to Figure 6 Describes the decoder implementation.

[0098] Decoder training

[0099] Figure 8 An example of a process for training a neural network for decoding error-correcting codes according to aspects of the present disclosure is shown. In some examples, these operations may be performed by a system including a processor that executes a set of code to control functional elements of a device. Additionally or alternatively, dedicated hardware may be used to perform the process. Generally, these operations may be performed according to the methods and processes described based on aspects of the present disclosure. For example, the operations may consist of various sub-steps or may be performed in combination with other operations described herein.

[0100] The training of the decoder can be described according to the following algorithm:

[0101] FPD θ Representing a neural network

[0102] make Indicates FPD θ (f)

[0103] Let info denote the minibatch of information bits used by each network

[0104] make Represents the estimated information bits

[0105] Let f denote the feature vector computed using the feature extractor

[0106] Let λ θ Represents the learning rate

[0107] Initialize the parameter vector θ

[0108] For each iteration, execute

[0109]

[0110] Finish

[0111] Thus, at operation 800, the system prepares a labeled training set including ground truth information bits and voltage levels read from a memory device, wherein the voltage levels encode the ground truth information bits based on a set of modulation symbols used, and programs the encoded ground truth information bits into the memory device. In some cases, the operations of this step may represent or be performed by a training component.

[0112] At operation 805, the system identifies a set of conditional probability mass functions based on the labeled training set, wherein each of the conditional probability mass functions describes the voltage level probability of one of the set of modulation symbols. In some cases, the operation of this step may be represented as referring to Figure 6 The feature extractor described or by reference Figure 6 Describes the feature extractor implementation.

[0113] At operation 810, the system reads a voltage level from a memory device. In some cases, the operation of this step may be represented as referring to Figure 6 The memory device described or by reference to Figure 6 The memory device described is executed.

[0114] At operation 815, the system extracts features from the voltage level based on the conditional probability mass function. In some cases, the operation of this step can be represented as referring to Figure 6 The feature extractor described or by reference Figure 6 Describes the feature extractor implementation.

[0115] At operation 820, the system applies a neural network decoder to the features to generate predicted information bits. In some cases, the operation of this step may be represented as referring to Figure 6 The decoder described or by reference to Figure 6 Describes the decoder implementation.

[0116] At operation 825, the system compares the predicted information bits with the ground truth information bits. In some cases, the operations of this step may represent or be performed by a training component. In one example, the system calculates the gradient of the cross entropy of the predicted information bits and the ground truth information bits, wherein the comparison is based on the gradient of the cross entropy.

[0117] At operation 830, the system updates the neural network decoder based on the comparison. In some cases, the operations of this step may represent or be performed by a training component.

[0118] For the set collection training which can be processed offline, the information bits {b j}. The information bits are then encoded into codewords {c j The codewords are then grouped and modulated into symbols {S jThe symbol is then programmed into the NAND cell and the voltage {V j For each unit, record a pair of (V j ,S j ), where V j is the voltage read, S j is the sign programmed to that voltage.

[0119] To create the probability mass function (which can also be an offline process), the training set (V j ,S j )’s voltage dynamic range is divided into bins. Each bin represents all voltage measurements between some lower and upper limits: B m-1 ≤V j <B m , where j is the sample index and m is the bin index. 1≤m≤M. The number of bins M is configurable and sets the voltage resolution. For the probability mass function for S0, with S i = 0 are collected to generate the conditional probability mass function (histogram) P(V|S0). This means that for each bin, in the case where a given cell is programmed to S0, m The length of the probability mass function vector is represented by the number of voltage bins M.

[0120] For the probability mass function S1, collect the i = 1 to generate the conditional probability mass function (histogram) P(V|S1). This means that for each bin, in a specific bin B, given that the cell is programmed to S1, j The number of voltage measurements in .

[0121] Continue this process until where n_bpc is the number of bits per unit. As a result, there are 2 n _bpc probability mass functions, each of which is a vector of size M.

[0122] Therefore, rather than using the raw voltages provided to the decoder, by providing likelihood probabilities for voltage measurements given each possible logic level, faster and more accurate convergence can be achieved.

[0123] For example, a (7, 8) code with 3 bits / cell is used. By using a vanilla voltage decoder (VV decoder), each codeword is stored in 3 cells, so the vanilla decoder is provided with 3 voltages. Alternatively, a likelihood voltage decoder (LV decoder) is provided with 24 values, thereby using the voltage distribution in each level. The histogram will be created offline, and the voltage measurements will be accurately evaluated during training / testing time.

[0124] The histogram takes into account additional data (such as word line index, P / E cycle index, retention time, and column index). Some of this additional data may not be available in practice. Therefore, the histogram can be created as part of an offline preprocessing phase before training and testing time. This information may have already been created by the log-likelihood ratio (LLR) decoder to calculate the LLR for each bit, so both decoders can share the same statistical information. Another variant of this feature vector, called the log-likelihood voltage decoder (LLV decoder), can take into account the log-likelihood probabilities.

[0125] Therefore, this disclosure proposes a neural network decoder that can be used in flash memory scenarios. The feature extractor module reduces convergence time compared to the original input of a noisy signal. The example architecture of the neural network decoder uses PAM modulation, or can be designed for BPSK modulation. Histogram probabilities increase the decoder's coverage for NAND channels or AWGN channels.

[0126] The present disclosure includes the following embodiments.

[0127] A method for decoding error-correcting codes using a neural network is described. Embodiments of the method may receive a signal from a physical channel, wherein the signal is based on modulated symbols representing information bits encoded using an error-correcting coding scheme; extract features from the signal using a feature extractor trained using probability data collected from the physical channel; and decode the information bits using a neural network decoder using the extracted features as input.

[0128] A device for decoding error-correcting codes using a neural network is described. The device may include a processor, a memory in electronic communication with the processor, and instructions stored in the memory. The instructions are operable to cause the processor to: receive a signal from a physical channel, wherein the signal is based on modulated symbols representing information bits encoded using an error-correcting coding scheme; extract features from the signal using a feature extractor trained using probability data collected from the physical channel; and decode the information bits using a neural network decoder using the extracted features as input.

[0129] A non-transitory computer-readable medium storing code for decoding an error-correcting code using a neural network is described. In some examples, the code includes instructions executable by a processor to: receive a signal from a physical channel, wherein the signal is based on modulated symbols representing information bits encoded using an error-correcting coding scheme; extract features from the signal using a feature extractor trained using probability data collected from the physical channel; and decode the information bits using a neural network decoder using the extracted features as input.

[0130] In some examples, the physical channel includes a NAND memory device.

[0131] Some examples of the above methods, apparatus, non-transitory computer-readable media, and systems may further include detecting a voltage level from the NAND memory device, wherein the detected voltage level comprises a noisy realization of the modulated symbol.

[0132] In some examples, the modulated symbols comprise channel inputs to a NAND memory device, and the signals comprise channel outputs from the NAND memory device.

[0133] In some examples, the probability data collected from the physical channel includes one or more probability mass functions.

[0134] In some examples, the probability data does not assume an additive white Gaussian noise (AWGN) model.

[0135] Some examples of the above methods, apparatuses, non-transitory computer-readable media, and systems may further include programming a test symbol into a NAND memory device. Some examples may further include detecting a voltage level of the NAND memory device, wherein the probability data is based on a conditional probability of detecting the voltage level given each test symbol.

[0136] Some examples of the above methods, devices, non-transitory computer-readable media, and systems may further include encoding the information bits using an error correction coding scheme. Some examples may further include applying a modulation scheme to the encoded information bits to generate modulated symbols.

[0137] In some examples, the error correction coding scheme includes a polar coding scheme.

[0138] In some examples, the modulated symbols are based on a pulse amplitude modulation (PAM) scheme.

[0139] In some examples, the modulated symbols are based on a binary phase shift keying (BPSK) modulation scheme.

[0140] A method for decoding an error-correcting code using a neural network is described. Embodiments of the method may prepare a labeled training set comprising ground truth information bits and voltage levels read from a memory device, wherein the voltage levels are based on encoding the ground truth information bits using a set of modulation symbols and programming the encoded ground truth information bits to the memory device; identify a plurality of conditional probability mass functions based on the labeled training set, wherein each of the plurality of conditional probability mass functions describes a probability of a voltage level for one of the set of modulation symbols; read the voltage levels from the memory device; extract features from the voltage levels based at least in part on the plurality of conditional probability mass functions; apply a neural network decoder to the features to generate predicted information bits; compare the predicted information bits to the ground truth information bits; and update the neural network decoder based on the comparison.

[0141] An apparatus for decoding an error-correcting code using a neural network is described. The apparatus may include a processor, a memory in electronic communication with the processor, and instructions stored in the memory. The instructions are operable to cause the processor to: prepare a labeled training set comprising ground truth information bits and voltage levels read from a memory device, wherein the voltage levels are based on encoding the ground truth information bits using a set of modulation symbols and programming the encoded ground truth information bits into the memory device; identify a plurality of conditional probability mass functions based on the labeled training set, wherein each of the plurality of conditional probability mass functions describes a probability of a voltage level for one of the set of modulation symbols; read the voltage levels from the memory device; extract features from the voltage levels based at least in part on the plurality of conditional probability mass functions; apply a neural network decoder to the features to generate predicted information bits; compare the predicted information bits to the ground truth information bits; and update the neural network decoder based on the comparison.

[0142] A non-transitory computer-readable medium storing code for decoding an error-correcting code using a neural network is described. In some examples, the code includes instructions executable by a processor to: prepare a labeled training set comprising ground truth information bits and voltage levels read from a memory device, wherein the voltage levels are based on encoding the ground truth information bits using a set of modulation symbols and programming the encoded ground truth information bits to the memory device; identify a plurality of conditional probability mass functions based on the labeled training set, wherein each of the plurality of conditional probability mass functions describes a probability of a voltage level for one of the set of modulation symbols; read the voltage levels from the memory device; extract features from the voltage levels based at least in part on the plurality of conditional probability mass functions; apply a neural network decoder to the features to generate predicted information bits; compare the predicted information bits to the ground truth information bits; and update the neural network decoder based on the comparison.

[0143] Some examples of the above methods, devices, non-transitory computer-readable media, and systems may further include calculating a gradient of a cross entropy between the predicted information bits and the ground truth information bits, wherein the comparison is based on the gradient of the cross entropy.

[0144] In some examples, the voltage level read from the memory device corresponds to a noisy version of a symbol from the set of modulated symbols.

[0145] Some examples of the methods, devices, non-transitory computer-readable media, and systems described above may further include encoding the ground truth information bits using an error correction coding scheme. Some examples may further include applying a modulation scheme to the encoded ground truth information bits to generate modulated symbols, wherein the voltage level is based on the modulated symbols.

[0146] In some examples, the error correction coding scheme includes a polar coding scheme.

[0147] Some examples of the above memory devices and methods may further include: an encoding component configured to encode the information bits to generate encoded information bits. Some examples may further include: a modulation component configured to modulate the encoded information bits.

[0148] Some examples of the above memory devices and methods may further include: a scaling component configured to scale an output of the modulation component based on a dynamic voltage range of the memory cell.

[0149] In some examples, the neural network decoder includes one or more fully connected multilayer perceptron (MLP) layers.

[0150] The descriptions and drawings described herein represent example configurations and do not represent all embodiments within the scope of the claims. For example, operations and steps may be rearranged, combined, or otherwise modified. In addition, structures and devices may be represented in block diagram form to illustrate the relationships between components and avoid obscuring the concepts being described. Similar components or features may have the same name but may have different reference numerals corresponding to different figures.

[0151] For those skilled in the art, some modifications to the present disclosure may be easy and clear, and the principles defined herein may be applied to other variations without departing from the scope of the present disclosure. Therefore, the present disclosure is not limited to the examples and designs described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0152] The methods described herein may be implemented or performed by a device comprising a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof. A general purpose processor may be a microprocessor, a conventional processor, a controller, a microcontroller, or a state machine. A processor may also be implemented as a combination of computing devices (e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in combination with a DSP core, or any other such configuration). Thus, the functions described herein may be implemented in hardware or software and may be performed by a processor, firmware, or any combination thereof. If implemented in software executed by a processor, the functions may be stored in the form of instructions or code on a computer-readable medium.

[0153] Computer-readable media include both non-transitory computer storage media and communication media, including any media that facilitates the transmission of code or data. Non-transitory storage media can be any available media that can be accessed by a computer. For example, non-transitory computer-readable media may include random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), compact disk (CD) or other optical disk storage device, magnetic disk storage device, or any other non-transitory medium for carrying or storing data or code.

[0154] Additionally, a connecting component may be properly referred to as a computer-readable medium. For example, if a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technology (such as infrared, radio, or microwave signals) is used to transmit code or data from a website, server, or other remote source, the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technology is included in the definition of medium. Combinations of media are also included within the scope of computer-readable media.

[0155] In this disclosure and claims, the word "or" indicates an inclusive list, so that, for example, a list of X, Y, or Z means X, or Y, or Z, or XY, or XZ, or YZ, or XYZ. Furthermore, the phrase "based on" is not used to indicate a closed set of conditions. For example, a step described as "based on condition A" may be based on both condition A and condition B. In other words, the phrase "based on" should be interpreted to mean "based, at least in part, on." Furthermore, the words "a" or "an" indicate "at least one."

Claims

1. A method for decoding an error correction code, comprising: receiving a signal from a physical channel, wherein the signal is based on modulated symbols representing information bits encoded using an error correction coding scheme, wherein the physical channel comprises a NAND memory device; extracting features from the signal using a feature extractor trained using probability data collected from a physical channel, wherein the probability data collected from the physical channel comprises one or more conditional probability mass functions, each of the one or more conditional probability mass functions describing a voltage level probability of one of a set of modulation symbols; and The information bits are decoded using a neural network decoder taking the extracted features as input, wherein extracting features from the signal using a feature extractor based at least in part on the one or more conditional probability mass functions, Among them, the neural network decoder is updated based on the comparison of the decoded information bits with the ground truth information bits.

2. The method according to claim 1, further comprising: A voltage level is detected from a NAND memory device, wherein the detected voltage level comprises a noisy realization of a modulated symbol.

3. The method according to claim 1, wherein The modulated symbols comprise channel inputs to the NAND memory device, and the signals comprise channel outputs from the NAND memory device.

4. The method according to claim 1, wherein Probabilistic data do not assume an additive Gaussian white noise model.

5. The method according to any one of claims 1 to 4, further comprising: programming test symbols into the NAND memory device; as well as A voltage level of the NAND memory device is detected, wherein the probability data is based on a conditional probability of detecting the voltage level given each test symbol.

6. The method according to any one of claims 1 to 4, further comprising: Encoding the information bits using an error-correcting coding scheme; as well as A modulation scheme is applied to the coded information bits to generate modulated symbols.

7. A method for training a neural network for decoding an error-correcting code, comprising: preparing a labeled training set including ground truth information bits and voltage levels read from a memory device, wherein the voltage levels are based on encoding the ground truth information bits using a set of modulation symbols and programming the encoded ground truth information bits to the memory device; identifying a plurality of conditional probability mass functions based on a labeled training set, wherein each of the plurality of conditional probability mass functions describes a voltage level probability of one of a set of modulation symbols; reading a voltage level from a memory device; extracting features from the voltage level based at least in part on the plurality of conditional probability mass functions; applying a neural network decoder to the features to generate predicted information bits; comparing the predicted information bits with the ground truth information bits; and The neural network decoder is updated based on the comparison.

8. The method according to claim 7, further comprising: The gradient of the cross entropy of the predicted information bits and the ground truth information bits is calculated, wherein the comparison is based on the gradient of the cross entropy.

9. The method according to claim 7, wherein: The voltage levels read from the memory device correspond to noisy versions of symbols from the set of modulated symbols.

10. The method according to any one of claims 7 to 9, further comprising: Encoding the ground truth information bits using an error correction coding scheme; as well as A modulation scheme is applied to the encoded ground truth information bits to generate modulated symbols, wherein the voltage level is based on the modulated symbols.

11. The method according to any one of claims 7 to 9, wherein: Error correction coding schemes include polar coding schemes.

12. A memory device, comprising: a plurality of memory cells configured to store encoded information bits; a feature extractor configured to: extract features from the read voltage levels based on probability data collected for the memory device; as well as A neural network decoder configured to decode the encoded information bits using the extracted features as input, wherein the neural network decoder includes one or more fully connected multi-layer perceptron layers, wherein the feature extractor is trained using probability data collected for the memory device, wherein the probability data comprises a plurality of conditional probability mass functions, each of the plurality of conditional probability mass functions describing a voltage level probability of one of a set of modulation symbols, wherein the feature extractor extracts features from the read voltage levels based at least in part on the plurality of conditional probability mass functions, Among them, the neural network decoder is updated based on the comparison of the decoded information bits with the ground truth information bits.

13. The memory device of claim 12, further comprising: an encoding component configured to: encode the information bits to generate encoded information bits; as well as The modulation component is configured to modulate the encoded information bits.

14. The memory device of claim 13 , further comprising: A scaling component is configured to scale the output of the modulation component based on a dynamic voltage range of the memory cell.

Images