Data storage device configured for use with a generative adversarial network (GAN)

Abstract

Data storage devices configured to utilize generative adversarial networks (GANs) are described herein, including high-resolution GANs (SRGANs). In some examples, a GAN-based decoding (reconstruction) procedure is implemented in a data storage controller to replace or supplement an error-correcting coding (ECC) decoding procedure to allow a reduction in the number of parity bits used during data storage. In other examples, soft-bit information is utilized during decoding using GANs. A dissimilarity matrix may be generated to represent differences between an initial image and a GAN-reconstructed image, with matrix values ​​mapped into low-density parity check (LDPC) codewords to facilitate LDPC decoding of data.In still other examples, confidence information obtained from a GAN is integrated into image pixels. In some examples, GAN reconstruction of data is limited to modifying talbits. Multiple GANs can be used in parallel, and their results can be aggregated. System and method examples are provided.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims the benefit of U.S. Non-Provisional Application No. 18 / 232,155, entitled "DATA STORAGE DEVICE CONFIGURED FOR USE WITH A GENERATIVE-ADVERSARIAL-NETWORK (GAN)," filed on August 9, 2023, in the U.S. Patent and Trademark Office, and hereby incorporates by reference for all purposes the entire contents of which are hereby incorporated by reference and claims priority to U.S. Provisional Application No. 63 / 457,951, filed on April 7, 2023. AREA

[0002] The disclosure relates, in some aspects, to data storage devices with non-volatile memory (NVM). In particular, but not exclusively, aspects relate to data storage devices configured to use generative adversarial networks (GANs). INTRODUCTION

[0003] Examples of non-volatile memory (NVM) data storage devices (DSDs) include solid-state drives (SSDs), sometimes called NAND flash drives, and more traditional spinning-disk hard disk drives. Generative adversarial networks (GANs) are a type of deep learning model within the family of generative models. Among other things, GANs are capable of generating artificial data, such as artificial images, that appear realistic. GANs can use two neural networks that compete with each other (and are thus "adversarial") to generate new and synthetic versions of datasets that can mimic real datasets, such as by creating realistic-looking images. One type of GAN is a super-resolution GAN (SRGAN), which can be used to reconstruct a low-quality image as a much sharper image.

[0004] Disclosed herein are data storage devices that implement and utilize GANs. SUMMARY

[0005] The following is a simplified summary of some aspects of the disclosure to provide a basic understanding of those aspects. This summary is not intended to be a comprehensive overview of all contemplated features of the disclosure, nor is it intended to identify key or critical elements of all aspects of the disclosure, nor to limit the scope of any or all aspects of the disclosure. Its sole purpose is to present various concepts of some aspects of the disclosure in a simplified form as a prelude to the more detailed description presented later.

[0006] One embodiment of the disclosure provides a data storage device including: a non-volatile memory (NVM); and a data storage controller coupled to the NVM. The data storage controller includes a processor configured to: read data from the NVM; generate confidence information representing confidence in the reliability of reading data from the NVM; and process the data with a generative adversarial network (GAN) procedure configured to use the confidence information to reconstruct the data.

[0007] Another embodiment of the disclosure provides a method for use by a data storage device, including: reading data from an NVM of the data storage device; generating confidence information representing confidence in the reliability of reading the data from the NVM; and processing the data using a GAN procedure configured to use the confidence information to reconstruct the data.

[0008] Yet another embodiment of the disclosure provides an apparatus for use in a data storage device. The apparatus includes: means for reading data from an NVM of the data storage device; means for generating confidence information representing confidence in the reliability of reading the data from the NVM; and means for processing the data using a GAN procedure configured to use the confidence information to reconstruct the data. BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 is a schematic block diagram of a data storage device in the form of an exemplary solid-state device (SSD) or other data storage device (DSD) having a non-volatile memory (NVM) array, wherein the NVM array is configured for generative adversarial network (GAN)-based processing, according to aspects of the disclosure. Fig. 2 illustrates example components of a DSD having a main memory partition and a GAN memory partition, as well as various components for GAN-based processing and ECC-based processing, according to aspects of the disclosure. Fig. 3 illustrates an example method for a write path for storing data in a GAN memory partition according to aspects of the disclosure. Fig. 4 illustrates an example method for a read path for decoding data read from an NVM using a GAN, according to aspects of the disclosure. Fig. 5 illustrates an exemplary alternative method for a read path for decoding data read from an NVM using a GAN, according to aspects of the disclosure. Fig. 6 illustrates additional features of an example method for a write path for storing data to an NVM in which some data is stored in a main memory partition and other data is stored in a GAN partition, according to aspects of the disclosure. Fig. 7 illustrates features of an example method for a read path for reading data from an NVM with GAN-based preprocessing to generate soft information bits in accordance with aspects of the disclosure. Fig. 8 illustrates features of an exemplary method for processing a dissimilarity matrix according to aspects of the disclosure. Fig. 9 illustrates features of an example method for using GAN-based soft-bit confidence information according to aspects of the disclosure. Fig. 10 is a diagram illustrating read voltage levels used to define and identify talbits for use in GAN processing, according to aspects of the disclosure. Fig. 11 illustrates a first exemplary method for talbit-based processing with GANs according to aspects of the disclosure. Fig. 12 illustrates a second exemplary method for talbit-based processing with GANs according to aspects of the disclosure. Fig. 13 is a diagram illustrating additional read voltage levels that may be used to define and identify talbits for use in GAN processing, according to aspects of the disclosure. Fig. 14 illustrates features of an example method for "re-inverting" high confidence data when modified by a GAN, in accordance with aspects of the disclosure. Fig. 15 illustrates features of an example method for GAN result aggregation according to aspects of the disclosure. Fig. 16 illustrates additional features of an example method for GAN result aggregation according to aspects of the disclosure. Fig. 17 is a schematic block diagram configuration for an exemplary DSD having an NVM and a controller configured to decode data using a decoding procedure including a GAN procedure, according to aspects of the disclosure. Fig. 18 illustrates an exemplary method for decoding data using a decoding procedure including a GAN procedure, according to aspects of the disclosure. Fig. 19 is a schematic block diagram configuration for an exemplary DSD having an NVM and a controller configured to decode data by applying an LDPC procedure to data along with GAN-based soft bit information generated by a GAN procedure, according to aspects of the disclosure. Fig. 20 illustrates an exemplary method for decoding data by applying an LDPC procedure to data along with GAN-based soft bit information generated by a GAN procedure, according to aspects of the disclosure. Fig. 21 is a schematic block diagram configuration for an exemplary DSD having an NVM and a controller configured to process data using a GAN procedure configured to use the confidence information to reconstruct the data, according to aspects of the disclosure. Fig. 22 illustrates an example method for processing data using a GAN procedure configured to use the confidence information to reconstruct the data, according to aspects of the disclosure. Fig. Figure 23 is a schematic block diagram configuration for an example device such as DSD with GAN-based features. DETAILED DESCRIPTION

[0009] In the following detailed description, reference is made to the accompanying drawings, which form a part hereof. In addition to the illustrative aspects, embodiments, and features described above, other aspects, embodiments, and features will become apparent by reference to the drawings and the following detailed description. The description of elements in each figure may refer to elements of the preceding figures. Like reference numerals may refer to like elements throughout the figures, including alternative embodiments of like elements.

[0010] The examples herein relate to non-volatile memory (NVM) and to data storage devices or devices for controlling the NVM, such as a controller of a data storage device (DSD), such as a solid-state device (SSD), and in some examples, to solid-state memory storage devices with NVM arrays, such as those using NAND flash memory (herein, "NANDs"). (A NAND is a type of non-volatile memory technology that does not require power to store data. It utilizes negative AND logic, i.e., NAND logic.) For brevity, an SSD with one or more NAND dies will be used below in describing various embodiments as a non-limiting example of a DSD. It should be understood that at least some of the aspects described herein may be applicable to other forms of data storage devices.For example, at least some of the aspects described herein may be applicable to a data storage or memory device, including phase-change memory (PCM) arrays, magnetoresistive random access memory (MRAM) arrays, storage-class memory, and resistive random access memory (ReRAM) arrays. Additionally, the various embodiments may be used in various machine learning devices that may include a combination of processing elements and memory / data storage elements, including NVM arrays constructed / configured according to the described embodiments. Although the primary examples described herein relate to SSDs with NVM arrays, such as NAND die arrays, many of the features described herein may also be applied to other types of NVM, such as hard disk drives (HDDs), tape drives, hybrid drives, etc.Some features may also be applicable to volatile memory. overview

[0011] As mentioned above, Generative Adversarial Networks (GANs) are capable of generating artificial data, such as artificial images, that appear realistic. GANs use two neural networks that compete with each other (and are thus "adversarial") to generate new and synthetic versions of datasets that can mimic real datasets, such as by creating realistic-looking images. One of the neural networks is called the generative network, which generates candidates. The other neural network is called the discriminatory network, which evaluates the candidates. In general, the generative network generates candidate datasets, such as candidate images. The discriminatory network attempts to distinguish the candidates from real datasets, such as real images. The training goal of the generative network is to increase the error rate of the discriminatory network (i.e.,trick the discriminatory network by generating candidates that the discriminatory network cannot identify as synthesized data).

[0012] One type of GAN is a super-resolution GAN (SRGAN), which can be used to transform low-quality images into much sharper ones. For example, an SRGAN can be used to process a compressed or corrupted image and reconstruct, reproduce, or recreate the image as a sharp, high-quality image. Note that there is no guarantee that the reconstructed image will match the initial low-quality image (i.e., the corrupted or compressed image). Rather, the SRGAN intelligently estimates a likely reconstruction of the original image if it is properly trained using a training procedure performed on a large dataset of similar images.

[0013] Herein, aspects of the present disclosure relate to data storage devices configured to utilize GANs (or GAN-based inference procedures) to improve data storage with data storage devices and, for example, to increase overall performance and / or provide cost reductions. Other aspects specifically relate to the use of SRGANs (or SRGAN-based inference procedures) to improve image storage with data storage devices and thus, again, to improve overall performance and / or provide cost reductions.

[0014] In one aspect, a GAN-based decoding procedure (reconstruction procedure) is implemented in a data storage controller to replace an ECC (Error Correction Coding) decoding procedure, thus enabling a reduction in the number of parity bits used when storing the data. In some GAN-based examples, no parity bits are used at all. By employing a GAN-based decoding procedure, errors in stored data—such as blurriness in heavily compressed images read from an NVM or noise in uncompressed images read from a heavily worn NVM—can be "corrected," for example, to produce sharper versions of the images. For images, the GAN-based method can be performed using an SRGAN.Because this example uses an SRGAN to reconstruct the image during decoding, the reconstructed image may not exactly match the original image. However, for many image processing applications (e.g., video games), an exact match is not required. Rather, it is sufficient for the reconstructed image to be a realistic-looking version of the original image. By reducing or eliminating parity bits, more data can be stored in the memory array.

[0015] It should be noted that the term "GAN-based decoding" or similar terms are used herein to refer to a GAN-based process that "decodes" data read from memory by reconstructing some or all of the data using GAN-based techniques, such as reconstructing corrupted or blurred images. It should be understood that GAN-based decoding is therefore different from traditional decoding (e.g., ECC-based decoding), which instead attempts to decode data while simultaneously eliminating errors in the data that may be introduced due to noise or other factors. Herein, the terms "GAN-based decoding" and "GAN-based reconstruction" are used interchangeably.

[0016] In some aspects, a separate GAN partition is provided in the NVM array for storing data (e.g., images) without parity bits for subsequent decoding using a GAN processing path. Other data (e.g., non-image data) may be stored in a separate ECC partition with parity bits for subsequent decoding using an ECC processing path (e.g., a decoding path that utilizes low-density parity check (LDPC) or the like). Accordingly, before writing data to the NVM array, the data storage controller determines which partition the data should be stored in (e.g., GAN partition for image data and ECC partition for non-image data). For example, the determination can be made based on metadata in data received from a host, such as a namespace identifier (ID), host IDs, and queue IDs, or based on a logical block address (LBA) range.When storing data in the GAN partition, different trim voltages can be used compared to the ECC partition, e.g., a first trim voltage compared to a second trim voltage. In other aspects, data to be stored in the GAN partition can be stored directly in a multi-level cell array (MLC), while data to be stored in the ECC partition can be stored in a single-level cell array (SLC).

[0017] In another aspect, data is stored with minimal parity bits, and an ECC decoding procedure is initially applied to attempt to decode the data. If the data is successfully decoded using ECC, it is sent to a host. However, if the ECC decoder cannot successfully decode the data, the data is provided to the GAN for GAN-based decoding. This allows images to be decoded using ECC without much noise. Images with significant noise (e.g., due to read errors from a worn NVM) can be reconstructed using the GAN. This, in turn, allows for a reduction in the number of parity bits used when storing data.

[0018] In another aspect, data is stored with minimal parity bits, and a cyclic redundancy check (CRC) is used only to detect any errors (without attempting to correct the errors). If no errors are detected, the data is sent to a host. However, if errors are detected, the data is then provided to the GAN for GAN-based decoding. This way, error-free images are promptly sent to the host. Error-containing images are reconstructed using the GAN. This allows for a reduction in the number of parity bits used in storing data and faster overall processing.

[0019] In some aspects, soft bit information is utilized using a GAN. For example, data is read from an NVM array, e.g., by acquiring it using a hard bit sensing threshold, and then a GAN is used to generate GAN-based soft bit information from the data. A low-density parity check (LDPC) procedure is used to decode the data using the GAN-based soft bit information. In an image-based example, pixels may be enhanced to include GAN-based soft bit information in addition to the usual pixel color information. The pixels with soft bit information are decoded using an LDPC procedure configured to utilize the GAN-based soft bit information. In some aspects, a bit error rate (BER) is estimated, and the GAN-based soft bit information is generated only if the BER exceeds the threshold.Otherwise, the LDPC decodes the data without GAN-based soft bit information. In other aspects, additional soft bit information may be generated by performing additional detections on opposite sides of the hard bit detection threshold. The LDPC procedure may be configured to utilize both the GAN-based soft bit information and the additional soft bit information.

[0020] In an image-based example, the GAN-based soft bit information is generated using a dissimilarity matrix. For example, the GAN procedure is applied to an initial image read from the NVM array to obtain a GAN-modified image. A dissimilarity matrix is ​​created from the differences between the initial image and the GAN-modified image. The matrix includes dissimilarity score values ​​that indicate the amount by which each pixel in the GAN-modified image differs from the initial image. The dissimilarity matrix is ​​mapped to a codeword representing the image to be decoded by the LDPC procedure. The values ​​in the matrix are assigned to the bits within the codewords, so that the LDPC procedure gives relatively less importance to bits with relatively high dissimilarity scores and relatively more importance to bits with relatively low dissimilarity scores.The LDPC then uses the modified codewords during the decoding process.

[0021] In these GAN-based soft-bit examples, the GAN can be initially trained offline (e.g., using a host processor) to generate the soft-bit information. GAN procedures can be trained differently depending on the particular type of image data being processed, e.g., video game images versus other types of images. In some examples, the GAN procedure is trained using a cost function including a penalty term representing a number of bit flips between initial images and GAN-reconstructed images, thereby training the GAN procedure to preserve a large portion of the initial images within the GAN-reconstructed images. That is, it is advantageous if the reconstructed images do not differ significantly from the original images (e.g., the total number of bit flips can be restricted or capped at a certain maximum).This is done so that the GAN only flips bits when there is a high confidence that the bit needs to be flipped. The GAN therefore has a high confidence that bits that are not flipped are correct.

[0022] In other aspects, different types of confidence information may be used. For example, pixel data is read from an NVM array, and confidence information is generated that represents confidence in the reliability of reading the pixel data from the NVM array. The pixel data is then decoded (reconstructed) using a GAN procedure configured to use the confidence information in decoding the pixel. In some examples, the confidence information is generated as soft bits based, for example, on log likelihood ratios (LLRs). In some examples, pixel data is initially decoded in a preliminary processing stage using two or more different GAN procedures in parallel (without using confidence information).The results are compared, and if the different GAN procedures yield the same preliminary decoded pixel data, the confidence is considered high. On the other hand, if there are numerous bit flips between the resulting decoded pixels, the confidence is considered low. This means that the reliability can be quantified based on the number of bit flips. Therefore, the confidence level can be quantified using both LLR and a GAN-based procedure. After quantification, the confidence level can be represented as soft bits for use in further decoding. In other examples, confidence values ​​can be obtained from inference-based or rule-based procedures, such as spell checkers, autocorrect, predictive text models, and artificial intelligence (AI) procedures.

[0023] In other aspects, confidence information is obtained by obtaining and utilizing talbits (i.e., bits that flip states when the read voltages are slightly adjusted). In one example, first and second acquisitions of bits of the pixel may be performed using first and second different read voltages to obtain first and second reads of the pixel data. A first set of talbits is identified during the first and second reads of the pixel data. The first and second reads of the pixel data are applied to first and second GAN procedures to obtain first and second GAN output data. A second set of talbits is then obtained from the first and second GAN output data, and the bits of the second set of talbits are counted and compared to an error threshold.If the talbit count of the second set exceeds the error threshold, the pixel data is assigned a first (low) confidence value; otherwise, a second (high) confidence value is assigned. If the confidence is low, the pixel data is reapplied to one or both of the GAN procedures to obtain additional GAN ​​output data and, thereby, additional sets of talbits. The talbit count is updated and again compared with the error threshold. In this way, the procedure can be repeated iteratively until the confidence level is satisfactory. As explained further below, the talbit-based procedures can be performed using high-speed AND and XOR operations to accelerate processing.

[0024] In other aspects, most or all of the above GAN-based procedures may utilize multiple GANs, aggregating or otherwise combining the results. In one example, different weights are applied to the output of multiple GAN processors to compensate for known noise sources, such as known noise sources within the NVM array, due to, for example, worn-out blocks. In another example, the multiple GAN processors are trained differently to account for different types of noise. In yet another example, multiple copies of the stored data are read from the NVM array using different read voltages, and the separate copies are applied to the individual GAN ​​processors, aggregating the results.In other examples, the results of separate GAN procedures are combined by determining a median value to use as the combined result or by determining the average by excluding maximum and minimum values.

[0025] Note that the term GAN herein encompasses GAN-like procedures and variants, i.e., procedures or variants that utilize a generative adversarial machine learning procedure or algorithm. An example is the original GAN ​​algorithm developed by Goodfellow et al., described in Generative Adversarial Nets. Proceedings of the International Conference on Neural Information Processing Systems (NIPS 2014), pp. 2672–2680. Other GAN-like procedures or variants include, but are not limited to, conditional GANs; GANs with alternative architectures such as deep convolutional GAN ​​(DCGAN), self-attention GAN (SAGAN), variational autoencoder GAN (VAEGAN), transformer GAN (TransGAN), and flow GAN; and GANs with alternative objectives such as Non-Saturating Loss GANs, Maximum Likelihood GANs, Least Squares GANs, etc. and many other variants.

[0026] Some GANs are configured for use with specific data types, such as image data or audio data. Therefore, if the DSD is a dedicated device designed to process a specific data type, such as still images, video, text, or audio, a GAN suitable for use with that particular data type can be chosen during the DSD design phase. As mentioned earlier, SRGAN is particularly useful for image data or video images. If the DSD is designed to process different data types, such as audio and video, the DSD can be equipped with different GAN types, allowing the device to route the data to the associated GAN. It should be noted that some data types, such as encrypted or scrambled data, are not suitable for GANs.However, in some DSDs, data can be stored in encrypted or ciphered form in the NVM as long as it is decrypted or decrypted before being applied to the GAN.

[0027] These and other functions are described below. Example data storage device with GAN-based processing

[0028] Fig. 1 is a block diagram of a system 100 including an exemplary SSD (or DSD) with components configured for use with GAN. The system 100 includes a host 102 and an SSD 104 or other DSD coupled to the host 102. The host 102 provides commands to the SSD 104 to transfer data between the host 102 and the SSD 104. For example, the host 102 may provide a write command to the SSD 104 to write data to the SSD 104 or send a read command to the SSD 104 to read data from the SSD 104. The host 102 may also provide GAN-based parameters for use in controlling GAN processing, such as the parameters of a GAN trained by the host 102. The host 102 may be any system or device requiring data storage or retrieval and having a compatible interface for communicating with the SSD 104.For example, host 102 may be a computing device, a personal computer, a portable computer, a workstation, a server, a PDA, a digital camera, or a digital telephone, to name a few examples. Additionally or alternatively, host 102 may be a system or device requiring neural network processing, such as image or speech processing. For example, host 102 may be a component of a video game system.

[0029] The SSD 104 includes a host interface 106, an SSD controller 108, a volatile memory 110 (such as DRAM) or other memory, an NVM interface 112 (which may be referred to as a flash interface), and an NVM array 114. The NVM array 114 includes one or more NAND dies, one or more of which are configured with (a) a main memory partition 115 (e.g., regular flash memory for storing data with ECC parity bits) and (b) a GAN memory partition 117 (e.g., alternative memory for use with a GAN that stores data with fewer or no parity bits). In some examples, data to be processed with a GAN is stored with reduced parity (or no parity) in the GAN memory partition 117. However, in other examples, the data to be processed with a GAN is stored with standard parity in the main memory partition 115.In still other examples, there is no separate GAN memory partition.

[0030] The SSD controller 108 includes a GAN-based processing controller 116 for controlling GAN-based functions (such as GAN-based decoding / reconstruction of data acquired from the GAN memory partition 117) and an ECC-based processing controller 118 for controlling various ECC-based functions (such as LDPC-based decoding of data acquired from the main memory partition 115). Note that, although primarily described with respect to examples in which the GAN-based processing controller 116 is a component of the SSD controller 108, the GAN-based processing controller 116 may instead be separate from the SSD controller 108. Some functions of the GAN-based processing controller 116 may be implemented in the host 102 or another device, separate from the SSD 104.It should also be noted that SSD and SSD controller are used for illustrative purposes only throughout the figures, and the various embodiments may include the use of the disclosed GAN-based data processing techniques in the controller for other types of storage devices such as HDDs and tape drives.

[0031] The host interface 106 is coupled to the SSD controller 108 and facilitates communication between the host 102 and the SSD controller 108. The SSD controller 108 is coupled to the volatile memory 110 and the NVM array 114 via the NVM interface 112. The host interface 106 may be any suitable communication interface, such as an NVMe (Non-Volatile Memory Express) interface, a USB (Universal Serial Bus) interface, an SP (Serial Peripheral) interface, an ATA or SATA (Advanced Technology Attachment) interface, a SCSI (Small Computer System Interface), an IEEE 1394 (Firewire) interface, or the like. In some embodiments, the host 102 includes the SSD 104. In other embodiments, the SSD 104 is remote from the host 102 or included in a remote computing system communicatively coupled to the host 102.For example, the host 102 may communicate with the SSD 104 via a wireless communication link.

[0032] Although in the Fig. 1, the SSD 104 includes a single channel between the SSD controller 108 and the NVM die(s) 114 via interface 112, the subject matter described herein is not limited to a single memory channel. For example, in some NAND memory system architectures, two, four, eight, or more NAND channels couple the controller and the NAND memory device, depending on controller capabilities. In all embodiments described herein, more than one channel may be used between the controller and the memory die, even though only a single channel is shown in the drawings. The SSD controller 108 may be implemented on a single integrated circuit chip and may communicate with different layers of memory in the NVM die(s) 114 via one or more command channels.

[0033] The SSD controller 108 controls the operation of the SSD 104. In various aspects, the SSD controller 108 receives commands from the host 102 via the host interface 106 and executes the commands to transfer data between the host 102 and the NVM array 114. The SSD controller 108 and / or the NVM interface 112 may include flash management components, such as a Flash Translation Layer (FTL).

[0034] The SSD controller 108 may include any type of processing device, such as a microprocessor, a microcontroller, an embedded controller, logic circuitry, software (SW), firmware (FW), hardware (HW), or the like. In some aspects, some or all of the functions described herein as being performed by the SSD controller 108 may instead be performed by another element of the SSD 104. For example, the SSD 104 may include a microprocessor, a microcontroller, an embedded controller, logic circuitry, software, FW, or any type of processing device to perform one or more of the functions described herein as being performed by the SSD controller 108. In other aspects, one or more of the functions described herein as being performed by the SSD controller 108 are instead performed by the host 102.In still further aspects, some or all of the functions described herein as being performed by the SSD controller 108 may instead be performed by another element, such as a controller in a hybrid drive that includes both non-volatile memory elements and magnetic storage elements.

[0035] The volatile memory 110 may be any suitable memory, computing device, or system capable of storing data. For example, the volatile memory 110 may be ordinary RAM, DRAM, Double Data Rate (DDR) RAM (DRAM), Static RAM (SRAM), Synchronous Dynamic RAM (SDRAM), Flash memory, Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), or other fast non-volatile memory such as storage-class memory (e.g., MRAM, ReRAM, PCM) or the like. In various embodiments, the SSD controller 108 uses the volatile memory 110, or a portion thereof, to store data during data transfer between the host 102 and the NVM array 114. For example, the volatile memory 110, or a portion of the volatile memory 110, may be cache memory.NVM array 114 receives data from SSD controller 108 via NVM interface 112 and stores the data in NVM array 114. NVM array 114 may be any suitable type of non-volatile memory, such as NAND-type flash memory or the like. In some embodiments, volatile memory 110 may be replaced with non-volatile memory such as MRAM, PCM, ReRAM, etc., to serve as memory for the device as a whole.

[0036] Although Fig. 1 shows an exemplary SSD, and an SSD is generally used as an illustrative example throughout the description, the various disclosed embodiments are not necessarily limited to an SSD application / implementation. As an example, the disclosed NVM die and associated processing components may be implemented as part of a package that includes other processing circuitry and / or components. For example, a processor may include or otherwise be coupled to embedded NVM and associated circuitry and / or components for the GAN processing described herein. As an example, the processor could offload certain GAN processing tasks from the controller 108 to other circuitry and / or components.

[0037] Fig. Figure 2 illustrates a block diagram of an exemplary NVM array 202 (consisting of one or more NVM dies) configured to store some data in a main memory partition 204 (with a default number of parity bits) and other data (e.g., images) in a GAN memory partition 206 (with fewer parity bits or no parity bits). The figure does not illustrate all circuit or memory components that might be used in a practical NVM array, such as input / output components, voltage regulation components, clock and timing components, etc. Rather, only some components and circuits are shown, summarized as block or schematic diagrams.

[0038] The SSD controller 208 from Fig. 2 includes components configured to perform or control GAN-based operations and other non-GAN operations, such as the otherwise standard ECC processing. In the example of Fig. 2, the example SSD controller components include: one or more GAN processors (or GAN-like processors) 212 configured to perform at least some GAN processing operations (e.g., SRGAN inference operations) on data obtained from the GAN memory partition 206; one or more ECC processors 214 configured to perform at least some ECC operations on data obtained from the main memory partition 204 (which may include LDPC-based processing); one or more GAN output aggregation components 219 configured to combine or otherwise aggregate the outputs of the GAN processors 212, as described below; and one or more GAN training components 220 configured to train or retrain the GAN processors 212.It should be noted that GAN training can be computationally intensive, and therefore, training of the GANs may instead be performed using host processors (with the trained GAN networks being loaded into the SSD controller 208). However, in some examples, GAN training may be performed in the SSD controller, which is why GAN training components 220 are shown. Multiple instances of each of these components (e.g., 212, 214, 219, and 220) are shown because, in some examples, a plurality of such devices may operate in parallel. Furthermore, at least some of these components (such as 212, 214, 219, and 220) may be configured on-chip, i.e., on an NVM die of the NVM array 202, which stores the data, for example, using circuitry below or adjacent to the array.

[0039] In the example of Fig. 2, the SSD controller 208 of Fig. 2 also includes a GAN-based processing controller 216 and an ECC-based processing controller 218. The GAN-based processing controller 216 includes one or more GAN-based encode / decode controllers 222 configured to control the storage of data to the GAN memory partition (e.g., as part of a write operation) and to control the decoding (reconstruction) of data acquired from the GAN memory partition 206 (e.g., as part of a read operation) using a GAN procedure (or a GAN-based inference procedure) performed by one or more of the GAN processors 212.As mentioned above, data can be stored in the GAN memory partition 206 with few (if any) parity bits, yet by employing a GAN-based decoding procedure, errors in the stored data—such as blurriness in heavily compressed images or noise in uncompressed images read from a heavily worn NVM—can be "corrected," for example, to produce sharper versions of the images. For images, the GAN-based procedure can be an SRGAN procedure.

[0040] The ECC-based processing controller 218 includes one or more ECC-based encode / decode controllers 224 configured to control the storage of data to the main memory partition (e.g., as part of a write operation) and to decode the data acquired from the main memory partition 204 (e.g., as part of a read operation) using an ECC procedure (e.g., an LDPC procedure) performed by one or more of the ECC (LDPC) processors 214. In some examples, a standard (first) number of parity bits may be used for data stored in the main memory partition 204, while a smaller (second) number of parity bits may be used for data stored in the GAN memory partition 206. It should be noted that the ECC-based encode / decode controller(s) 224 may also be used to attempt to decode data read from the GAN memory partition 206.As mentioned above, in some examples, the ECC decoding procedure is initially applied to attempt to decode the data. If the data is successfully decoded using ECC, it is sent to a host. However, if the ECC-based decoder 224 cannot successfully decode the data, the data is provided to the GAN-based decoder 222. In other examples, the CRC processor(s) 236 is / are used to detect errors in the data read from the GAN memory partition 206 (without attempting to correct the errors). If no errors are detected, the data is sent to a host. If errors are detected, the data is then provided to the GAN-based decoder 222 for GAN-based decoding.

[0041] The SSD controller 208 also includes one or more GAN-based soft bit controllers 226 configured to control the generation and processing of soft bits using GAN, and one or more ECC-based soft bit controllers 228 configured to control the generation and processing of soft bits using ECC. For example, under the control of the GAN-based soft bit controller(s) 226, data may be read from the GAN storage partition 206 by sensing bits using a hard bit sensing threshold. The sensed data is processed using the GAN processors 212 to generate GAN-based soft bit information from the data, which is then applied to the ECC processors 214 to use an LDPC procedure to decode the data using the GAN-based soft bit information.As previously mentioned, in an image-based example, pixels may be enhanced to include GAN-based soft bit information in addition to the usual pixel color information. The pixels with the soft bit information may be decoded using an LDPC procedure configured to utilize the GAN-based soft bit information. In some examples, the GAN-based soft bit controller(s) 226 use dissimilarity matrices, described in detail below, to modify codewords for LDPC processing. Additional soft bits may be obtained under the control of the ECC-based soft bit controller 228, for example, by performing additional detections on opposite sides of the hard bit detection threshold. The LDPC procedure of the ECC processor(s) 214 may be configured to utilize both the GAN-based soft bits and the additional soft bits.

[0042] One or more BER estimators 230 estimate the BER for data read from the GAN memory partition 206 (and for data read from the main memory partition 204). In some examples, GAN-based soft bit information is generated by the GAN-based soft bit controller(s) only if the BER exceeds the threshold. Otherwise, the ECC processor(s) 214 decodes the data without GAN-based soft bit information.

[0043] It should be noted that when using GAN-based soft bits, the GAN processors 212 can be trained offline to use the soft bits in the GAN process. For example, the GAN processors 212 can be trained based on images whose pixels include the soft bits.

[0044] The SSD controller 208 also includes one or more GAN-based confidence information controllers 232 configured to control the generation and processing of confidence information using GAN, and one or more ECC-based confidence information controllers 234 configured to control the generation and processing of confidence information using ECC. For example, under the control of the GAN-based confidence information controller(s) 226, pixel data is read from the GAN memory partition 206, and confidence information is generated that represents confidence in the reliability of reading the pixel data from the GAN memory partition 206. The pixel data is then decoded (reconstructed) using a GAN processor 212 configured to use the confidence information in decoding the pixel.

[0045] In some examples, the confidence information is generated as soft bits, based, for example, on LLRs. In other examples, pixel data is initially decoded in a preliminary processing stage using at least two different GAN procedures of the GAN processors 212 (without using confidence information). The results are compared by the GAN-based confidence information controller 232, and if the different GAN procedures yield the same preliminary decoded (reconstructed) pixel data, the confidence is classified as high. On the other hand, if there are numerous bit flips between the resulting decoded pixels, the confidence is classified as low. In this way, reliability is quantified by the number of bit flips. Furthermore, the confidence level can be quantified using both LLR and GAN.After quantification, the confidence level may be represented as soft bits for use in further decoding using the GAN-based soft bit controllers 226 and / or the ECC-based soft bit controllers 228. In some examples, when the GAN inverts data (e.g., a pixel) that was classified as high confidence, that data is inverted again so that only low-confidence data is modified by the GAN, while high-confidence data remains. After the data is inverted again in the latest GAN output, the GAN output is compared to the previous version of the GAN output (e.g., a previous GAN output image). If the differences between the new version of the GAN output and the previous version of the GAN output are acceptable (e.g., below a difference threshold), the process is complete, and the data (e.g., the processed image) is output to a host.If, however, the differences are unacceptable (e.g., they are above the difference threshold), the latest version of the GAN output can be sent back to the ECC-based soft-bit controller(s) 228 for further processing to generate updated confidence information for application to the GAN-based soft-bit controller(s) 226, and the process is repeated until sufficient convergence is achieved. This is described in detail below.

[0046] Furthermore, as explained in detail below, the GAN-based confidence information controllers 232 can utilize talbits (i.e., bits that reverse their state when the read voltages are slightly adjusted) to evaluate confidence in an iterative process using high-speed AND and XOR operations, thus accelerating processing. One or more CRC processors 236 can be provided.

[0047] It should be noted that not all features from Fig. 2 are required. For example, in some embodiments, only single instances of the various processors or components (212, 214, 219, and 22) are provided instead of multiple instances. Also, not all of the controllers 222, 224, 226, 228, 230, 232, 234, and 236 are required. Furthermore, some of the controllers or processors may instead be implemented in memory dies of the NVM array 202, e.g., using switching logic beneath the array.

[0048] Example GAN-based Procedures That Replace or Complement ECC Procedures This section describes systems and procedures that provide a GAN-based storage system where ECC protection is either performed as a backup for GAN and / or replaced by GAN- or SRGAN-like models. GAN / SRGAN reconstruction is performed during the read path and, in some examples (i.e., in addition to standard LDPC / ECC decoding), is implemented on top of or replaces the otherwise standard ECC protection. Examples using visual data (e.g., images / videos) are provided below, but it should be understood that these features are not limited to visual data.

[0049] For example, a dedicated DSD is provided that is (primarily) intended for storing visual data, or the DSD has a dedicated die or partition for such data. The stored data preferably belongs to a specific distribution (such as specific computer game types), so that a GAN training procedure can be initially executed offline using large amounts of source data (e.g., source images) and specialized generative learning capabilities to create a GAN-like or SRGAN-like model. The model is trained to be able to reproduce the source images from distorted / blurred / corrupted images.

[0050] In a first example, the amount of memory allocated to ECC parity is reduced (or eliminated) by a factor of five, for example, which can lead to significant cost reductions. In general, LDPC / ECC is expected to successfully decode approximately 99.99% of the images read from an NVM when implemented using Ultra-Low Processing (ULP) or Low Processing (LP) decoding modes. Full Performance (FP) is even more effective. In an illustrative example, the DSD is configured to perform ULP / LP ECC but not FP ECC. Instead, the FP decoding mode is replaced by an SRGAN inference reconstruction module, implemented within the DSD either with a dedicated hardware module or by using the DSD's existing compute engines. Therefore, in this example, memory costs are reduced (due to the reduced parity allocated to ECC) and the LDPC hardware is simplified (by omitting the FP decoding mode).In this example, if data is determined to be suitable for storage in the GAN partition (as specified by the host in metadata properties sent to the DSD) or is derived from the DSD (e.g., from characteristics of the data), the data is encoded with a small parity size. In some examples, metadata is preserved with some of its properties in the header. For example, the header can include information about the data type, such as image or video frame. For videos, the order of frames can be tracked and utilized by the GAN when reading a video.

[0051] In a second example, data is written (programmed) to the NVM array without any encoding and then read without decoding. No parity is assigned. This example is particularly useful within a dedicated visual die / device intended for storing images. The read data can be processed by a fast SRGAN inference module (or reconstructed by the host instead). In this second example, the programming (write) path is significantly reduced (as is the storage cost), although longer read durations may be required, which can be mitigated by using a dedicated SRGAN HW module with fast inference. Therefore, the DSD in this example does not need to have LDPC circuitry.

[0052] In the second example, a caching component (IS component) that could otherwise be provided in the DSD can be omitted. (A caching component is often used as a temporary buffer in high-quality SLC memory, allowing fast, error-free data writing. The usual storage goal is then to fold the data from the cache into regular storage in quad-level cell (QLC) memory or another MLC memory.) In this second example, caching can be omitted because errors on the read path can be tolerated or corrected using the GAN.

[0053] In the second example, the program path (write path) can also be configured to use a faster trim in the NVM (NAND) memory to speed up write operations. This faster trimming may result in more errors, but since the images are subsequently reconstructed using GAN, such errors can be tolerated. Furthermore, CRC can be used for each image to provide some degree of error detection. Images can therefore be read very quickly and without error correction. For corrupted images, GAN is applied during reading (either by the DSD or, in some examples, by the host). Corruption can be detected, for example, using syndrome weights.

[0054] Fig. 3 is a flowchart providing a high-level overview of a write path method for storing data in a GAN memory partition according to the above examples. In block 302, a processor (such as the GAN-based encode / decode controller 222 of Fig. 2) Data to be stored with a reduced number of ECC parity bits (e.g., fewer parity bits than usual with standard ECC according to the first example) or no parity bits are used (e.g., no encoding according to the second example). In block 304, the processor inserts metadata into a header of the data to indicate properties of the data (e.g., data type, images, sound recordings, etc., or more specifically, whether images are part of an animated movie or a real-world movie). In block 306, the processor writes the data (including the header) to the GAN memory partition (either directly to a physical address in an NVM die or to a logical namespace for subsequent storage at a physical address in the NVM die).

[0055] Fig. Figure 4 is a flowchart providing a high-level overview of a read path method for decoding a data read from a GAN memory partition according to the first example. In block 402, a processor (such as the ECC-based encode / decode controller 224 of Fig. 2) Data from the GAN memory partition (e.g., in response to a host read command) and, in block 404, attempts to read the data using a ULP / LP ECC procedure or engine (such as ECC processors 214 of Fig. 2) to decode. If the ECC-based decoding is successful, as determined in block 406, the decoded data is sent (e.g., extracted and sent) to the host that requested the data in block 408. If the decoding is unsuccessful, the data is reconstructed in block 410 using GAN, e.g., under the control of the GAN-based encoding / decoding controller 222 of Fig. 2.

[0056] Fig. 5 is a flowchart illustrating an alternative read path method according to the second example. In block 502, the SSD controller reads (acquires) data from the GAN storage partition (e.g., in response to a host read command) and, in block 504, detects the read / write status using CRC (e.g., using the CRC processor(s) 236 of Fig. 2) any errors in the data. If no errors are detected, as determined in decision block 506, the data (which does not require further decoding) is sent (e.g., extracted and sent) to the host that requested the data in block 508. If errors are detected, the data is reconstructed in block 510 using GAN, e.g., under the control of the GAN-based encoding / decoding controller 222 of Fig. 2.

[0057] Fig. 6 is a flowchart illustrating additional or alternative write path method features according to the first example. In block 602, the DSD receives data from a host to be written to an NVM array (including headers with metadata). At block 604, a component of the DSD (e.g., GAN-based encoding / decoding controller 222 of Fig. 2) whether the data is of a first type (e.g., non-image data to be subsequently decoded with ECC) or of a second type (e.g., image data to be subsequently decoded first with ECC and then, if necessary, with GAN), the determination being based on metadata indicators such as the logical block address (LBA) range, a namespace identifier (ID), a host ID, or a queue ID. For example, some LBA ranges may be designated for use with image data, while other LBA ranges are designated for use with non-image data, or some host queues may be designated for use with image data, while other host queues are designated for use with non-image data. If the data is of the first type, in block 606, a component of the DSD (e.g., ECC-based encoding / decoding controller 224 of Fig. 2) the data with a standard number of ECC parity bits (e.g., three bits per byte). In block 608, the encoded data is then stored in the main memory partition of the NVM array, e.g., within an SLC block (which generally exhibits fewer read / write errors than MLC blocks). If, however, the data is of the second type, in block 610, a component of the DSD (e.g., GAN-based encoding / decoding controller 222 of Fig. 2) the data with a smaller number of ECC parity bits (e.g., only one). In block 612, the encoded data is then stored in the GAN memory partition of the NVM array, e.g., within an MLC block. If the data is image data that will ultimately be reconstructed using GAN, the read / write errors that occur in MLC blocks can be more easily tolerated.

[0058] It should be noted that the procedures of Fig. 3-6 may be performed with multiple GANs in which GAN results are aggregated or otherwise combined. For example, read path operations may be performed in parallel (e.g., in block 402 of Fig. 4 and Block 502 of Fig. 5) to read multiple copies of data from the GAN memory partition using different read voltages, and the separate copies are applied to the separate GAN processors (e.g., in block 410 of Fig. 4 and Block 510 of Fig. 5), where the results are aggregated, for example, by determining a median value for use as a combined result or by determining the average excluding maximum and minimum values, as discussed above.

[0059] Example GAN-Based Procedures Leveraging GAN Preprocessing Before LDPC This section describes systems and procedures that provide a GAN-based storage system in which a GAN or a GAN-like system is used to generate soft bit information that can be used by an ECC / LDPC decoder to subsequently correct data errors (alone or in combination with other soft bit information). That is, a GAN is used to preprocess data to obtain soft bit information before LDPC is used to correct the data. The GAN is trained to relate the metric optimized by the GAN to the ECC / LDPC metric. In this way, soft bits generated by the GAN are used to improve the error correction capabilities of the ECC / LDPC decoder.It should be noted that although a standard GAN is not bit-based, soft bit information can still be generated or utilized by GAN-based systems, for example, by converting standard bits into pixels for processing by the GAN.

[0060] Here, the ECC / LDPC decoder can also use standard soft bit information obtained by performing additional acquisitions. In this regard, one of the main advantages of LDPC codes is the ability to utilize soft reliability information, which roughly doubles the decoder's correction capabilities. Soft information can be generated in the form of "soft bits," which are (traditionally) additional acquisitions (e.g., flash NAND acquisitions) performed above and below the hard bit voltage read threshold, indicating whether each bit is close to the hard bit threshold, making it less reliable, or far from the threshold, making it more reliable. The procedures described in this section can utilize both GAN-generated soft bits and conventional soft bits obtained by performing additional acquisitions.An ensemble of GANs can be used.

[0061] It should be noted that when applying these GAN-based procedures to images, a "spacing" between pixels can be defined based on visual metrics rather than bit flips. Depending on the specific GAN model, some examples may also involve converting standard bits to pixels so the GAN can process the data. Examples using visual data (e.g., images / videos with pixels) are provided herein, but it should be understood that some features are not limited to visual data but can also be applied to other data types.

[0062] Fig. Figure 7 is a flowchart providing a high-level overview of a read path method for reading data from an NVM array while selectively utilizing GAN-based soft bits. In block 702, a processor (such as the GAN-based encoder / decoder controller 222 of Fig. 2) Data from an NVM array by capturing it using a hard-bit read threshold. It should be noted here that the data does not need to be located in a GAN partition that uses fewer parity bits (or no parity bits). Rather, the data can be stored in an otherwise standard NVM as codewords with parity bits for LDPC error detection and correction. At block 704, the DSD estimates the BER (using, for example, the BER estimator 230 of Fig. 2) and compares the BER to an error threshold to determine whether soft bits should be generated. If the BER does not exceed the threshold, as determined in block 706, a processor (such as the ECC-based encoder / decoder controller 224 of Fig. 2) the data with LDPC without soft bits in block 708. However, if the BER exceeds the threshold, as determined in block 706, conventional soft bits and GAN-based soft bits are generated. That is, in block 710, a processor (such as the ECC-based soft bit controller 228 of Fig. 2) conventional soft bits by performing additional detections at voltages above and below the hard bit read threshold, and in block 712, a processor (such as the GAN-based soft bit controller 226 of Fig. 2) GAN-based soft bits using a trained GAN (e.g., one of the GAN processors 212 from Fig. 2). The operations of blocks 710 and 712 may be performed concurrently. Thereafter, in block 714, a processor (such as the ECC-based encoder / decoder controller 224 of Fig. 2) the acquired data using LDPC with traditional soft bits and GAN-based soft bits. (Here, the traditional soft bits may also be referred to as "additional soft bits" because they are used in addition to the GAN-based soft bits in this example.) For example, the soft information bits generated using GAN (or an ensemble of GANs) may be integrated with the additional soft bit information in the LDPC decoding process to improve the correction capabilities of the LDPC decoder. Note that in other examples, the processor may perform either block 710 or block 712, but not both, depending on device programming.

[0063] Although a variety of GAN architectures can be used, the super-resolution tasks of SRGAN are not utilized in the generation of the GAN-based soft bits. Rather, GAN tasks associated with (for example) standard GAN reconstruction of noisy images are utilized. That is, the goal of the GAN of block 712 is to "clean" a noisy image of NAND-related noise (e.g., programming noise / data retention (DR) noise, etc.), not requiring an exact reconstruction of an image, but only performing a partial reconstruction to remove the noise. As explained below, the GAN-based soft bits are derived from this.

[0064] Note that in these examples, the image data stored in the NVM array is encoded using an LDPC process that generates codewords that include the "message" (i.e., the bits representing a particular pixel) in one part of the codeword and various error detection and correction bits for use by LDPC in another part of the codeword. This preserves the bits of the image pixels so they can be read without LDPC error correction. Therefore, codewords can be read from memory to first obtain only the stored image (which may contain various errors due to bit flips), which can be applied to a GAN to generate the GAN-based soft bits. Then, the entire codeword, along with the GAN-based soft bit information, can be applied to an LDPC decoder to error-correct the image.

[0065] The overall process comprises the following steps. The initial stage (stage 0) is performed offline by a host processor. The subsequent steps are performed by a DSD. •Level 0 - Initial offline training: A selected GAN is trained on a large collection of data samples (e.g., images), preferably on data samples from training sets all taken from a uniform distribution (e.g., cartoons / nature videos / family photos, etc.). The GAN's cost function is configured to include a penalty term for the number of original bit flips / pixel flips, in order to incentivize the GAN model to provide a (partially) reconstructed image that preserves much of the original data (or at least hints of the original data). The advantage here is that the reconstructed image does not differ significantly from the original. Therefore, the total number of bit flips can be restricted or limited to a certain maximum. This is done so that the GAN only flips bits when there is high confidence that the bit needs to be flipped. The GAN therefore has high confidence that bits that are not flipped are correct.The trained GAN is loaded into the DSD. •Level 1 - GAN Inference: In use, an initial image (previously stored in the NVM array as codewords encoded with a coding matrix (M)) is read from the NVM array by the DSD's SSD controller, which reads the codewords and extracts the bits representing the image's pixels. This initial image is then applied to the trained GAN, allowing the GAN generator to perform inference operations on the initial image to generate a partially reconstructed modified image. Note that if the image data is stored encrypted or ciphered, the data should be decrypted or deciphered before being applied to the GAN to ensure the GAN obtains a coherent image. •Step 2 - Calculate similarity score matrix: The GAN result (i.e., the modified image from stage 1) is compared with the initial image read from the NVM array, and a dissimilarity matrix (with the dimensions of the original image) is generated to represent the visual differences between them (e.g., at the pixel level or at the pixel group level). For example, a group of pixels that have changed significantly between the original image read from memory and the reconstructed image created by the GAN is labeled with a high dissimilarity score, while a group of pixels that have not changed significantly receives a low dissimilarity score. To set an image similarity metric (i.e.,A variety of methods can be used to calculate the dissimilarity scores between the original image and the GAN-reconstructed image, such as Root Mean Square Error (RMSE), Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index (SSIM), Feature-Based Similarity Index (FSIM), etc. The dissimilarity scores of the dissimilarity matrix represent the GAN-based soft bit information. •Step 3 - Assigning the similarity scores to the original codeword: The resulting dissimilarity matrix is ​​mapped to the bit placement within the relevant codewords of the initial image. The mapping between pixels (or groups of pixels) and codeword bits is calculated by the SSD controller based on the coding matrix M used to initially encode the data. •Stage 4 - Assigning soft bits / adjusting LLR for relevant bits: LLR values ​​are obtained for all codeword bits, and the LLR values ​​with high dissimilarity scores are reduced to indicate to the LDPC decoder that these bits in the image are less reliable, e.g., due to noise in the NVM array. •Level 5 - LDPC error correction: The image codewords are error-corrected using LDPC based on adjusted LLR values ​​(which incorporates the GAN-based soft bit information—i.e., the dissimilarity scores—that are intended to aid decoding, making decoding more precise and efficient). As discussed above, the number of bit flips performed by GAN can be limited to preserve much of the original image, and this has the effect of preventing LDPC from flipping bits during error correction that the GAN has high confidence are correct in stage 5. Note that LDPC can also use otherwise standard soft bits obtained by additional sensing above and below a hard read voltage.

[0066] Fig. Figure 8 is a flowchart illustrating the processing of the dissimilarity matrix to generate the GAN-based soft bit information. In block 802, the SSD controller reads (acquires) image data from the NVM array that was previously stored using codewords to be error-corrected by an LDPC procedure. As discussed above, at this point, the SSD controller only reads the bits of the codewords that correspond to the pixels of the image and does not attempt to perform LDPC error correction. At block 804, the SSD controller (e.g., the GAN-based soft bit controller 226 of Fig. 2) applies a trained generative inference procedure of the GAN to the initial image read from the GAN memory partition to obtain a GAN-modified image, i.e., a reconstructed version of the image in which some bits have been reversed by the GAN. At block 806, the SSD controller generates a dissimilarity matrix from the differences between the initial image and the GAN-modified image. The matrix has the same size as the image and consists of dissimilarity scores representing the amount by which each pixel (or group of pixels) in the GAN-modified image differs from the corresponding pixels (or groups of pixels) in the initial image. At block 808, the SSD controller maps the dissimilarity matrix to the codewords of the initial image to be error-corrected using LDPC. At block 810, the SSD controller assigns the values ​​in the matrix to the bits within the codewords of the initial image so that the LDPC procedure (e.g.,performed by the ECC-based encoder / decoder 224 of . Fig. 2) Place relatively less emphasis on bits with relatively high dissimilarity scores and relatively more emphasis on bits with relatively low dissimilarity scores, e.g., by obtaining LLR values ​​for the codeword bits and then reducing the LLR values ​​with high dissimilarity scores to indicate to the LDPC decoder that these bits within the initial image are less reliable, e.g., due to noise in the NAND. As previously explained, when LDPC is applied to the image codewords, LDPC can also utilize conventional soft bits to further improve error correction capabilities.

[0067] In some examples, the optimal LLR values ​​used for initializing the LDPC decoder depend on an underlying memory error model with bins r as follows: E r = number or erroneous bits read in bin r; C r= number of bits read in bin r |LLRr|=log21−BERrBERr=log2Cr−ErEr where BER r = E r / C r . That is, LLR can be calculated from BER, which in turn is calculated from C r and E r can be calculated as shown.

[0068] Using these techniques, depending on the particular embodiment, improved correction capabilities for GAN-based domains can be achieved. A reduction in uncorrectable ECC (UECC) rates and an improvement in device lifetime can also be achieved. These correction capabilities can also be used to allow for reduced parity allocations, thus reducing device costs or increasing the usable memory within them. Furthermore, these techniques can be used to store data in worn-out blocks of a drive. Example GAN-based procedures that use soft bit confidence information

[0069] This section describes systems and procedures that provide a GAN-based storage system in which GANs or GAN-like systems use soft-bit confidence information to correct images during a GAN procedure. While the procedures of the previous section use a GAN to generate GAN-based soft-bit information for subsequent use by an LDPC decoder to enhance LDPC processing, the procedures of this section involve generating soft-bit information that is applied to a GAN to enhance GAN processing. The applied soft information can be obtained using conventional means (e.g.,Soft bits obtained from multiple acquisitions around a hard read threshold) and / or generated by running two or more GANs in parallel to identify bits that are reversed by some GANs but not by others (and are therefore less reliable and have lower confidence), as opposed to bits that are not reversed between GANs (and are therefore more reliable and have higher confidence).

[0070] In a GAN, each pixel is usually represented by a certain number of bits (e.g., 8 or 16). The bits represent the color of the pixel. Procedures described herein increase the number of bits per pixel by adding bits that encode the pixel's reliability (confidence). Reliability can be quantified using LLR or similar metrics. In one example, a pixel may include a first set of (standard) bits indicating the pixel's color and a second set of (added) bits that quantify the reliability of the first set of bits. For example, reliability can be quantified on a scale of 1 to 64, where 64 indicates maximum LLR reliability. In a specific example, the first set of bits may indicate that the pixel has a pure red color (e.g., #FF0000 in hexadecimal), and the second set of bits may indicate a reliability of 64 (i.e.,the system is very confident that the pixel actually corresponds to a pure red color). In another specific example, the first bit set may also indicate that the pixel has a pure red color, but the second bit set may indicate a confidence of 1 (e.g., relatively poor confidence).

[0071] In some examples, maximum reliability corresponds to the case where all "physical soft bits" match the pixel's bit values, i.e., the soft bits captured by the NVM are such that an LDPC procedure would determine high confidence in the bits. (Here, the term "physical soft bits" refers to soft bits read from the NVM at voltages different from the hard read voltage. They are physical in the sense that an actual capture operation is performed in physical memory to obtain the value.) On the other hand, minimum reliability corresponds to the case where the physical soft bit information indicates that the hard bits were read close to the read voltage, i.e., the physical soft bits are such that an LDPC procedure would determine low confidence in the bits.

[0072] As another example, maximum reliability corresponds to the case where a set or ensemble of GANs agrees that none of the pixel's bits should be reversed in a GAN-based reconstruction of the image containing the pixel, while minimum reliability corresponds to the case where the group or ensemble of GANs disagrees on whether to reverse the pixel's bits, for example, when half of the GANs reversed the bits and the other half did not. (Here, a soft bit derived by comparing GAN outputs is not considered a "physical soft bit" because it is not obtained directly from memory like physical soft bits.) Note that when multiple GANs are used, each GAN can be referred to as a "layer" of the entire GAN.

[0073] Therefore, in some examples, two or more GANs are used to process the image data, and the output of the GANs is compared to identify pixels that differ. The bits within the differing pixels are considered lower confidence bits, and this information can be applied during a final reconstruction of the image by a GAN trained or configured to use the confidence information.

[0074] Therefore, the reliability (confidence) of an image's pixel bits can be quantized and encoded in several different ways. For a given scheme, a GAN can be configured to process pixel data, incorporating reliability information to incorporate this information into image reconstruction, so that high-confidence pixels are preserved by the GAN while low-confidence pixels can be modified. Since low-confidence pixels are more likely to be corrupted by noise and high-confidence pixels are less likely to be corrupted by noise, the GAN then reconstructs the image, correcting the noise to produce a sharper image. In one example, the GAN extracts the pixel's extra bits that encode reliability and uses these bits as weights applied to the pixel during image reconstruction.

[0075] In yet another example, the processor may be provided with a supervisor function to control the operation of two GANs based on "talbits," where the GANs are controlled to modify only pixels that have talbits during image reconstruction. As explained below, talbits are low-confidence bits because they tend to reverse with a slight change in the read voltage. The supervisor function iteratively applies image data read at different voltages to the two GANs until the results of the two GANs converge on the same (or similar) output image, which is then used as the reconstructed output image.

[0076] In still other examples, the supervisor may iteratively apply data to an ECC and a GAN, with the output from the GAN being fed back into the ECC. In still other examples, depending on the data, the supervisor may be applied to other inference-based procedures or engines, or rule-based procedures or engines, such as spell checkers, autocorrect, predictive text models, and AI. (A rule-based procedure is one that applies specified rules to generate outputs; an inference-based procedure, such as AI, infers results from a set of data using, for example, a neural network, and generates outputs based on the results of the inference.)In some examples, the monitoring and iteration procedures or topologies described herein may be applied using a variety of means to determine which bits within the data should be reversed (or not reversed) (or means for reasoning, or means for guessing, etc.). These inference or rule-based engines may be used in parallel with or in conjunction with GANs to provide confidence to various data correction procedures.

[0077] Fig. Figure 9 is a flowchart providing an overview of a method for GAN image reconstruction (correction) using soft-bit reliability information. In block 902, a processor (such as the GAN-based encoder / decoder controller 222) reads (captures) Fig. 2) the pixels from an image from an NVM array, e.g., by capturing them using a hard-bit read threshold. Note that the image data may be stored in a GAN partition, but need not be stored in a GAN partition and may be stored in otherwise standard NVMs (as codewords with parity bits for LDPC error detection and correction, as discussed above). In block 904, the SSD controller generates soft-bit information indicating the reliability (confidence) of the bits of the image pixels to add to the pixels (where the soft-bit information is obtained either from LLR from physical soft reads or by applying the image to two or more GANs to detect the bit flips between them, as discussed above). In block 906, the SSD controller integrates the soft-bit information into the pixels of the image.As discussed above, the soft bit information may be added to the bits of a pixel, with the additional bits representing the reliability of the other bits of the pixel. At block 908, the SSD controller applies the image (with the modified pixels) to a GAN trained to utilize the soft bit information to reconstruct the image, correct the image, reduce noise, and / or sharpen the image. In this way, a noisy or blurry image may be sharpened by a GAN trained to leverage reliability in the GAN inference process to retain pixels with high read confidence and modify pixels with low read confidence (which are therefore more likely to be corrupted by noise). At block 910, in some examples, the SSD controller also recalibrates the read voltages, if warranted, based on the soft bit confidence / reliability information.That is, if the soft-bit confidence / reliability information obtained in block 904 indicates a relatively high level of noise (e.g., above a certain noise threshold) in the images read from the NVM, the read voltages may be recalibrated to reduce the noise.

[0078] The following describes efficient techniques for exploiting low-confidence “talbits” (i.e., bits whose states reverse when the read voltages are slightly adjusted).

[0079] Fig. Figure 10 provides a pictorial representation of the read voltage levels that can be used to define and identify valley bits. Briefly, a first diagram at 1000 shows an example of a two-state memory where the cells are either zero or one. The x-axis represents the voltage threshold (VT), and the y-axis shows the number of cells in the memory at that voltage. Note that there is some overlap between the voltage states. This means that at least some cells that should represent 1 overlap with some cells that should represent 0. The read level of diagram 1000 is the optimal hard read level because it is directly between the two states. Nevertheless, due to the overlap, some bits near the voltage threshold are misread and are therefore unreliable bits.Diagram 1002 shows an example where the read level is increased to a voltage at which none of the bits representing 1 are erroneously read as 0. Nevertheless, a larger number of bits that should be 0 are erroneously read as 1. Diagram 1004 shows a counterexample where the read level is decreased to a voltage at which none of the bits representing 0 are erroneously read as 1. Nevertheless, a larger number of bits that should be 1 are erroneously read as 0. Finally, diagram 1006 shows both the upper and lower adjusted read levels (R1 and R2). The voltage region between these two levels is referred to herein as the valley region, and the bits in between are referred to as valley bits. Bits in the valley region are sensitive to slight changes in the read voltage and easily reverse from one voltage state to the other.This means that the valley bits reverse their state when the read voltages are slightly adjusted. Note that diagram 1006 represents an example where only two acquisitions are made (at R1 and R2). In this regard, if the location (or approximate location) of the optimal read level (diagram 1000) is known, it is possible to skip sensing at this optimal read level to speed up performance at the expense of information, and thus conclude that all bits that have changed between the two acquisitions (R1 and R2) are in the valley, regardless of their probability of being a zero-to-one or one-to-zero transition.

[0080] Fig. 11 illustrates GAN processing 1100 for a "Two Sense" example (Two Sense: Two acquisitions, e.g., R1 and R2), in which various XOR and AND operations are performed on images acquired by an NVM, comprising bit strings (acquired at the two voltages, e.g., R1 and R2). The images are applied to separate GANs (GAN 1 and GAN 2) to control the GANs to modify only valley bits until the GANs converge on a reconstructed version of the image. The operations may be performed by a supervisory function of a processor of the SSD controller (such as the GAN-based confidence information controller 236 of Fig. 2). At block 1102, the processor performs a read operation on a first sense (e.g., sensing at R1) to obtain a string of bits representing an image that has been stored in the NVM. At block 1104, the processor adjusts the read level, e.g., by changing it to R2. At block 1106, the processor performs a read operation on a second sense (e.g., obtaining second read data), e.g., sensing at R2, to obtain a second string of bits representing the image. Because the read voltage is different, some bits are reversed from one bit string to the other. As explained above, R1 and R2 can be chosen to be on opposite sides of an optimal read voltage so that the reversed bits are valley bits.

[0081] In block 1108, the processor performs XORs of the first and second read data to obtain the valley bits. In block 1110, the processor sends or applies the first read data to GAN 1 to obtain reconstructed output data (GAN Out 1 data). In block 1112, the processor performs an XOR between the first read data and the GAN Out 1 data (to obtain GAN Out 1 Diff 1). In block 1114, the processor performs ANDs on the valley bits and the GAN Out 1 Diff 1 data (to obtain Valley-Only GAN Out 1 Diff 1). In block 1116, the processor performs XORs of the first read data and the Valley-Only GAN Out 1 Diff 1 data (to obtain Valley-Only GAN-modified first read data). In block 1118, the processor sends or applies the second read data to GAN 2 (to obtain GAN Out 2 data).

[0082] In block 1120, the processor performs XORs of the second read data and the GAN Out 2 data (to obtain GAN Out 2 Diff 1 data). In block 1122, the processor performs ANDs on the valley bits and the GAN Out 2 Diff 1 data (to obtain Valley-Only GAN Out 2 Diff 1 data). In block 1124, the processor performs XORs of the second read data and the Valley-Only GAN Out 2 Diff 1 data (to obtain Valley-Only GAN-modified second read data). In block 1126, the processor performs XORs of the Valley-Only GAN-modified first read data and the Valley-Only GAN-modified second read data (to obtain Valley bits 2).

[0083] In decision block 1128, the processor determines whether the number of Talbits 2 is below a (predetermined or adaptively set) threshold. If so, in block 1130, the processor determines that the Valley-Only GAN-modified first read is sufficient and no further reads are needed. That is, the Valley-Only GAN-modified first read image data represents an acceptable reconstruction of the image read from memory. Therefore, the processing of FIG. 1100 has occurred. On the other hand, if the count of Talbits 2 is not below the threshold, processing proceeds to block 1132, where the processor resets the data to: use the Talbits 2 as new Talbits; use the Valley-Only GAN-modified first read as new first read data; and using the Valley-Only GAN-modified second read as new second read data (to obtain new valley bits, first read data, and second read data).Processing then returns to block 1110 to repeat the processing of blocks 1112-1128 and determine whether the procedure has now reached a solution. A maximum number of iterations can be programmed to terminate the procedure if convergence is not achieved.

[0084] Fig. Figure 12 illustrates GAN processing 1200 for another “Two Sense” example. The operations may again be performed by a monitoring function of a processor of the SSD controller (such as the GAN-based confidence information controller 236 of Fig. 2). At block 1202, the processor performs a read operation on a first sense (e.g., sensing at R1) to obtain a string of bits representing an image that has been stored in the NVM. At block 1204, the processor adjusts the read level, e.g., by changing it to R2. At block 1206, the processor performs a read operation on a second sense (e.g., obtaining second read data), e.g., sensing at R2, to obtain a second string of bits representing the image. As explained above, R1 and R2 can be chosen to be on opposite sides of an optimal read voltage so that the inverted bits are valley bits.

[0085] In block 1208, the processor performs XORs of the first and second read data to obtain the valley bits. In block 1210, the processor sends or applies the first read data to GAN 1 to obtain reconstructed output data (GAN Out 1 data). In block 1212, the processor performs an XOR between the first read data and the GAN Out 1 data (to obtain GAN Out 1 Diff 1). In block 1214, the processor performs ANDs on the valley bits and the GAN Out 1 Diff 1 data (to obtain Valley-Only GAN Out 1 Diff 1). In block 1216, the processor performs XORs of the first read data and the Valley-Only GAN Out 1 Diff 1 data (to obtain Valley-Only GAN-modified first read data). In block 1218, the processor sends or applies the second read data to GAN 2 (to obtain GAN Out 2 data).

[0086] In block 1220, the processor performs XORs of the second read data and the GAN Out 2 data (to obtain GAN Out 2 Diff 1 data). In block 1222, the processor performs ANDs on the valley bits and the GAN Out 2 Diff 1 data (to obtain Valley-Only GAN Out 2 Diff 1 data). In block 1224, the processor performs XORs of the second read data and the Valley-Only GAN Out 2 Diff 1 data (to obtain Valley-Only GAN-modified second read data). In block 1226, the processor performs XORs of the Valley-Only GAN-modified first read data and the Valley-Only GAN-modified second read data (to obtain Valley bits 2).

[0087] In decision block 1228, the processor determines whether the number of valley bits 2 is below a (predetermined or adaptively set) threshold. If so, processing proceeds to a second decision block 1229, where the processor determines whether Valley-Only GAN Out 1 Diff 1 is greater than Valley-Only GAN Out 2 Diff 1. If not, the processor determines in block 1230 that the Valley-Only GAN-modified first read is sufficient and no further reads are needed. Otherwise, the processor determines in block 1231 that the Valley-Only GAN-modified second read is sufficient and no further reads are needed. In either case, the processing of FIG. 1200 has occurred.On the other hand, if the count of Talbits 2 is not below the threshold in block 1228, processing proceeds to block 1232, where the processor resets the data to: use the Talbits 2 as new Talbits; use the Valley-Only GAN-modified first read as new first read data; and use the Valley-Only GAN-modified second read as new second read data (to obtain new Talbits, first read data, and second read data). Processing then returns to block 1210 to repeat the processing of blocks 1212-1218 and determine whether the procedure has now resulted in a solution. Again, a maximum number of iterations can be programmed to terminate the procedure if convergence is not achieved.

[0088] It should be noted that once the various GAN-based changes that occur during the processing of Fig. 11 or Fig. 12 are generated, once they have converged below the threshold, this can be considered as a convergent result (so no further processing is needed). Fig. 12, the image that required the fewest changes can be taken as the likely correct image, possibly saving one iteration by allowing a larger threshold. Furthermore, the process of Fig. 12 help resolve asymmetries that occur in NAND distributions. It should also be noted that each iteration of either process restricts the GAN's talbits, i.e., the suspect bits, and thus further promotes convergence.

[0089] It should be noted that it is possible that bits outside the valley may be erroneous, and therefore the final output can be run through a GAN again without restricting the result to valley bits alone in an effort to further correct and sharpen the image. The number of such bits is likely small, so using the additional GAN ​​stage may not be justified for practical reasons. However, for systems where such errors are more common, the additional GAN ​​stage may be worthwhile. When the additional GAN ​​stage is executed, the number of bits changed (flipped) in this additional stage can be counted and compared to a threshold to determine whether the valley needs to be recalibrated (e.g., whether R1 and R2 need to be adjusted). In some examples, the valley may be too narrow (e.g.,R1 and R2 are too close together), so recalibrating the valley may be beneficial. A valley bit count can be used to determine if the valley is not wide enough or if the valley is not properly centered around the optimal read voltage. A valley search can also help center the center of the reads in the valley. A larger valley (increasing the spacing between reads) can allow more bits to be in play for the GAN to change. The number of bits can therefore indicate that increasing the valley size is worthwhile.

[0090] It should also be noted that the procedures of Fig. 11 and Fig. 12 treat all transitions equally, but if a transition is more likely 0 -> 1 (i.e., if it involves the "left" side of the valley), then only 0 -> 1 changes in the difference between the GAN's input and output might be allowed, with another iteration performed on this result before it is counted as the difference. This means that better correction can be achieved by factoring the probability of a bit being a zero or a one, rather than just the increased probability of an error compared to most other bits. It should also be noted that with multiple GANs, parts of the process can run in parallel within a single iteration. It should also be noted that XOR operations are usually performed very quickly on storage devices, since NAND flash can perform the operations in the NAND's latches.Additionally, XOR hardware engines for RAID capacity are often available in NAND storage devices and / or SSD / HDD arrays, so such operations do not need to be performed by a host or firmware. Although the description primarily refers to examples where the DSD / SSD performs the GAN operations, the operations could instead be performed by a host, or partially by the host and partially by the DSD / SSD.

[0091] Furthermore, although two or three readings in Fig. 11 and Fig. 12, additional read (capture) iterations are performed. This allows for different confidence levels, such as assigning a zero / one probability to bits closer to zero / one and assigning an increased bias (low confidence) to errors in bits located in the center of the valley.

[0092] Fig. Figure 13 provides a pictorial representation of additional voltage reading levels (at R3 and R4). As explained, this allows for different confidence levels. The bits sensed by R3 can be assigned a higher confidence than the bits sensed by R1. The bits sensed by R4 can be assigned a higher confidence than the bits sensed by R2. The valley between R3 and R4 can be the wider valley mentioned above, while the valley between R1 and R2 is a narrower valley.

[0093] Fig. Figure 14 illustrates a method 1400 in which confidence information is used to "re-flip" any high-confidence data modified by a GAN, so that only low-confidence data is modified by the GAN while high-confidence data is preserved. The procedure also includes an initial ECC decoding operation, which may be repeated in a loop along with the GAN procedure until sufficient convergence is achieved. The operations may, in turn, be controlled by a monitoring function of a processor of the SSD controller. Beginning at block 1402, the processor reads (captures) data (or other information) from the NVM array. At block 1404, the processor applies the data to an ECC decoder to attempt to correct errors in the data. During ECC decoding, confidence information of the type discussed above may be generated.If ECC successfully decodes the data by correcting all (or most) errors, as determined in decision block 1406, processing is complete, as indicated in block 1408. The processor may then output the data to a host or perform other actions. Note that in block 1406, the number of remaining errors in the data may be quantified and compared to an error threshold. If the number of errors is below the threshold, error correction is considered satisfactory. In other examples, all errors must be corrected before processing is considered complete.

[0094] If errors remain in the data, processing continues to decision block 1406, with block 1410 exporting the data from the ECC decoder for processing by one or more GANs. Additionally, any confidence information generated by ECC is exported from the ECC decoder at block 1412.

[0095] Additionally or alternatively, other types of confidence information may be generated in block 1412 using other inference-based or rule-based procedures, such as spell checkers, autocorrect, predictive text models, and AI. It should be noted that other confidence generation procedures in Fig. 14, but these are not limited to use only in the procedure of Fig. 14, but can also be used in many other confidence-based systems and procedures described herein, depending on the particular data type. For example, if the data being processed is text data, text-based procedures such as spell checking, autocorrection, and predictive text models can be used.

[0096] In block 1414, the processor applied the data exported in block 1410 to one or more GANs to generate a new version of the data. For example, if the data is image data, the GAN may reconstruct a previous version of the image to generate a new version of the image. If the data is text data and the GAN is configured to process text data, the GAN generates a new version of the text. As explained above, during a GAN operation, the GAN may attempt to correct (or reconstruct) the data by inverting some of the data, e.g., by inverting individual bits to change pixels, etc. In block 1416, the processor receives the modified data from block 1414 and the confidence information from block 1412 and provides the inversion (e.g.,Restores (flips or reverts) specific portions of the data that were reversed to their previous state when the data was high-confidence data. This is done so that only low-confidence data is modified by the GAN while high-confidence data is preserved. For example, the confidence value associated with a particular portion of data (e.g., a pixel) can be compared to a confidence threshold. If the confidence value exceeds the confidence threshold, the data is considered high-confidence data. Otherwise, it is considered low-confidence data. Any high-confidence data modified by the GAN is returned to its previous state.

[0097] In block 1418, the processor compares the new version of the data (i.e., the output of block 1416) with the previous version (i.e., the output of block 1410) to evaluate the extent of the change. This may be done by quantifying the differences and comparing them to a difference threshold (D). For example, the percentage of pixels in the latest version of the image that differ from the corresponding pixels in the previous version of the image may be calculated and compared to a threshold for the percentage difference, such as 1%. If the amount of change is not less than the difference threshold (D), as determined in decision block 1420, processing may return to block 1404 so that the latest version of the data may be reapplied to the ECC decoder (assuming the data is still in a form that can be processed by ECC). In this way, the entire procedure of Fig. 14 may be repeated in a loop to iteratively process the data until acceptable convergence is achieved. Once this convergence is achieved, which is determined in decision block 1420 when the most recent changes are less than the difference threshold (D), processing terminates in block 1408. For an image processing example, the data corresponding to the image may be repeatedly applied to the GAN(s) until the latest version is sufficiently similar to the previous version that no further processing is justified. In some examples, a maximum number of iterations may be specified such that the process ultimately terminates if convergence is not achieved, at which time an error indicator may be sent to the host or other appropriate action may be taken.It should also be noted that in other examples (not shown), if the change amount is not less than the difference threshold (D) at block 1420, processing returns to 1410 and 1412 rather than block 1404 for further GAN processing. That is, in some examples, blocks 1404 and 1406 may be bypassed in subsequent iterations of the loop. Example GAN-based procedures that use GAN ensembles

[0098] This section describes systems and procedures that apply multiple GANs (e.g., a GAN ensemble) to read data from an NVM to aggregate the results and thus achieve better overall correction capacity. In some examples, the aggregation can use the median between different GAN operations to provide a form of "majority rule."

[0099] In a first illustrative embodiment, when the data is passed to a GAN reconstruction module to support a decoding process, it is sent (e.g., in parallel) to multiple GANs instead of a single GAN, with each GAN initiated with different initial conditions. The data is processed by the GANs in parallel, and the results of the operation of the multiple GANs can be aggregated or otherwise combined. The aggregated result can then be sent to the host (or to a next decoding layer) as the output of the GAN reconstruction module. Aggregation can be performed in several ways. For example, if the input is an image, the median value of all GANs for each pixel of the image can be calculated to produce the final result.In another example, an average of all pixel values ​​can be calculated, excluding the minimum and maximum values ​​from the calculation to avoid extreme values.

[0100] Fig. 15 summarizes these features in flowchart 1500. Briefly, in block 1502, a processor (such as the GAN-based encoder / decoder controller 222 of Fig. 2) Data from the GAN memory partition (e.g., in response to a host read command). In block 1504, the processor initiates multiple GANs (e.g., in parallel) with different initial conditions and, in block 1506, applies the data to the GANs (e.g., in parallel), for example, to allow each to separately reconstruct an image represented by the data. In block 1508, the processor aggregates the results of the multiple GANs (e.g., by calculating the median value of all GANs for each of the image pixels to obtain a final result, or calculating an average of all pixel values, excluding the minimum and maximum values ​​from the calculation to avoid extreme values) or otherwise combines them.

[0101] In another embodiment, the multiple GANs can be trained differently to account for different types of defects / noise in the underlying storage medium. The aggregation may be different in this example because not all outputs are "the same." The system can identify the source of the noise, so the weighting of the outputs of the different GANs is done differently depending on the current noise source. In another embodiment, soft information can be generated based on the aggregated results to be fed to an ECC decoder (LDPC) or another GAN layer. The soft information can be generated in such a way that if a bit has been reversed in a GAN (or GANs), the reliability of the corresponding bit is reduced.The exact extent of reduced reliability (and its impact on the corresponding LLR) can be developed offline (e.g., using a host) during an initial calibration procedure. However, an online measurement (e.g., performed by the DSD) of the reliability impact can be used during decoding itself through a mechanism similar to that described in U.S. Patent 10,554,227 to Sharon et al., which is incorporated herein by reference. In another embodiment, each of the GANs can use data acquired by reading at different voltage levels, as discussed above.

[0102] Fig. 16 summarizes these features in flowchart 1600. Briefly, in block 1602, a processor (such as the GAN-based encoder / decoder controller 222 of Fig. 2) Data from the GAN memory partition (e.g., in response to a host read command). In block 1604, the processor initiates multiple GANs (e.g., in parallel) with different initial conditions and, in block 1606, applies the data to the GANs (e.g., in parallel), for example, to allow each to separately reconstruct an image represented by the data. In block 1608, the processor aggregates the results of the multiple GANs and then, in block 1610, applies the aggregated GAN results as soft information to another GAN or an LDPC engine. In block 1612, the processor decodes the data using the soft information. Blocks 1610 and 1612 may be performed, for example, using the techniques described above with reference to Fig. 7-8.

[0103] Generally speaking, the GAN aggregation procedures described in this section can be used in any of the GAN procedures described above. For example, if a GAN is used, the GAN can be replaced by an ensemble of GANs that aggregate results. Further exemplary procedures and facilities

[0104] Fig. 17 illustrates an embodiment of a device 1700 configured in accordance with one or more aspects of the disclosure. The device 1700 or its components could be embodied in or implemented in any suitable device or apparatus capable of performing the operations such as the DSDs of Fig. 1 or Fig. 2. The device 1700 includes an NVM 1702. The device 1700 also includes a data storage controller 1704 coupled to the NVM 1702 and having a processor configured to: read data from the NVM 1702; and decode the data using a decoding procedure comprising a GAN procedure, as described above, for example, in connection with Fig. 3-6. In some examples, the decoding method includes both a GAN procedure and an ECC procedure. The processor is further configured to decode the data by: applying the ECC procedure to the data read from the NVM to attempt to successfully decode the data using ECC, and in response to successfully decoding the data using ECC, sending the decoded data to a host; and in response to unsuccessful decoding of the data using ECC, decoding the data using the GAN procedure and sending the decoded data to the host.In other examples where the decoding procedure includes both a GAN procedure and an ECC procedure, the processor is further configured to decode the data by: applying the ECC procedure to the data read from the NVM to detect but not correct errors; sending the data to the host if no errors are detected; and decoding the data in response to detected errors using the GAN procedure and sending the decoded data to the host.

[0105] Fig. 18 broadly illustrates a process 1800 according to some aspects of the disclosure. The process 1800 may be performed in any suitable device or apparatus capable of performing the operations and coupled to an NVM, such as the device of Fig. 16 or the DSDs of Fig. 1 and Fig. 2. In block 1802, the device reads data from an NVM of a DSD, and in block 1804, the device decodes the data using a decoding procedure that includes a GAN procedure. In some examples, in block 1806, the device may optionally send the decoded data to a host and / or process the decoded data in the DSD. In some examples, such as the device of Fig. 16, the decoding procedure includes both a GAN procedure and an ECC procedure. The method may further include decoding data by: applying the ECC procedure to the data read from the NVM to attempt to successfully decode the data using ECC, and in response to successfully decoding the data using ECC, sending the decoded data to a host; and in response to unsuccessful decoding of the data using ECC, decoding the data using the GAN procedure and sending the decoded data to the host.In other examples where the decoding procedure includes both a GAN procedure and an ECC procedure, the method may further include decoding the data by: applying the ECC procedure to the data read from the NVM to detect but not correct errors; sending the data to the host if no errors are detected; and decoding the data in response to detected errors using the GAN procedure and sending the decoded data to the host.

[0106] Fig. 19 illustrates an embodiment of a device 1900 configured in accordance with one or more aspects of the disclosure. The device 1900 or its components could be embodied in or implemented in any suitable device or apparatus capable of performing the operations such as the DSDs of Fig. 1 or Fig. 2. The device 1900 includes an NVM 1902. The device 1900 also includes a data storage controller 1904 coupled to the NVM 1902 and having a processor configured to: read data from the NVM 1902; generate GAN-based soft bit information from the data using a GAN procedure; and decode the data by applying an LDPC procedure to the data along with the GAN-based soft bit information generated by the GAN procedure, as described above, for example, in connection with Fig. 7 and Fig. 8. In some examples, the dissimilarity matrix procedures described above are used.

[0107] Fig. Figure 20 broadly illustrates a process 2000 according to some aspects of the disclosure. The process 2000 may be performed in any suitable device or apparatus capable of performing the operations and coupled to an NVM, such as the device of Fig. 18 or the DSDs of Fig. 1 and Fig. 2. In block 2002, the device reads data from the NVM. In block 2004, the device generates GAN-based soft bit information from the data using a GAN procedure. In block 2006, the device decodes the data by applying an LDPC procedure to the data together with the GAN-based soft bit information generated by the GAN procedure, as described above, for example, in connection with Fig. 7 and Fig. 8. In some examples, in block 2008, the device may optionally send the decoded data to a host and / or process the decoded data in the DSD. In some examples, the dissimilarity matrix procedures described above are employed.

[0108] Fig. Figure 21 illustrates an embodiment of a device 2100 configured in accordance with one or more aspects of the disclosure. The device 2100 or its components could be embodied in or implemented in any suitable device or apparatus capable of performing the operations such as the DSDs of Fig. 1 or Fig. 2. The device 2100 includes an NVM 2102. The device 2100 also includes a data storage controller 2104 coupled to the NVM 2102 and having a processor configured to: read data from the NVM 2102; generate confidence information representing confidence in the reliability of reading data from the NVM; and process the data with a GAN procedure configured to use the confidence information to reconstruct the data, for example, as described above in connection with Fig. 9-16.

[0109] Fig. Figure 22 broadly illustrates a process 2200 according to some aspects of the disclosure. The process 2200 may occur in any suitable device or apparatus capable of performing the operations and coupled to an NVM, such as the device of Fig. 18 or the DSDs of Fig. 1 and Fig. 2. In block 2202, the device reads data from the NVM. In block 2204, the device generates confidence information representing confidence in the reliability of reading data from the NVM. In block 2204, the device processes the data using a GAN procedure configured to use the confidence information to reconstruct the data, as described above, for example, in connection with Fig. 9-16. In some examples, in block 2208, the device may optionally send the reconstructed data to a host and / or process the reconstructed data in the DSD. Example setup with an NVM array with a GAN partition

[0110] Fig. 23 illustrates one embodiment of a device 2300 configured in accordance with one or more aspects of the disclosure. Device 2300 or its components could embody or be implemented in a DSD or other type of device that supports data storage. In various implementations, device 2300 or its components could be a component of a processor, a controller, a computing device, a personal computer, a portable device or workstation, a server, a PDA, a digital camera, a digital phone, an entertainment device, a medical device, a control device for self-driving vehicles, or any other electronic device that stores, processes, or uses data. In some examples, device 2300 is a component of a video game system.

[0111] The device 2300 is communicatively coupled to an NVM array 2301 comprising one or more memory dies 2304, each of which may include physical memory arrays 2306, e.g., NAND blocks, where at least one of the memory dies includes a main memory partition 2308 and a GAN memory partition 2309. The physical memory array 2306 may be communicatively coupled to the device 2300 such that the device 2300 may read or capture information from the physical memory array 2306 and may write or program information. That is, the physical memory array 2306 may be coupled to circuitry of the device 2300 such that the physical memory array 2306 is accessible to the circuitry of the device 2300. It should be noted that not all components of the memory dies are shown. The dies may, for example, B. Processing circuits outside the array (e.g.circuits below the array or adjacent to the array), as well as input / output components, etc. The connection between the device 2300 and the memory dies 2304 of the NVM array 2301 may include, for example, one or more buses.

[0112] The device 2300 includes a communication interface 2302 and a data storage controller 2310. These components can be coupled and / or brought into electrical communication with each other and with the NVM array 2301 via suitable components, which is generally represented by the connecting lines in Fig. 23. Although not shown, other circuits may be provided such as timing sources, peripherals, voltage regulators, and power management circuits which are well known in the art and therefore will not be described further.

[0113] The communication interface 2302 provides a means for communicating with other devices over a transmission medium. In some implementations, the communication interface 2302 includes circuitry and / or programming (e.g., a program) suitable for facilitating bidirectional communication of information related to one or more devices in a system. In some implementations, the communication interface 2302 may be configured for wired communication. For example, the communication interface 2302 could be a bus interface, a transmit / receive interface, or another type of signal interface, including circuitry for outputting and / or receiving signals (e.g., outputting signals from and / or receiving signals in an SSD). The communication interface 2302 serves as an example of a means for receiving and / or a means for transmitting.

[0114] The data storage controller 2310 includes modules and / or circuits arranged or configured to receive, process, and / or send data, control data access and storage, issue or respond to commands, and control other desired operations. For example, the various modules / circuits may be implemented as one or more processors, one or more controllers, and / or other structures configured to perform functions. According to one or more aspects of the disclosure, the modules / circuits may be adapted to perform the various features, processes, functions, operations, and / or routines described herein. For example, the various modules / circuits may be configured to perform the functions described with respect to Fig. 1-17. Furthermore, in some embodiments, some features of data storage controller 2310 may be implemented on a memory die 2304 as circuitry below or adjacent to the array.

[0115] The term "adapted" in relation to the processing modules / circuits herein may refer to the modules / circuits being configured, deployed, implemented, and / or programmed to perform a particular process, function, operation, and / or routine according to the various features described herein. The modules / circuits may include a special-purpose processor, such as an application-specific integrated circuit (ASIC), which may serve as a means (e.g., structure) for executing any of the functions described in connection with Fig. 1-21. The modules / circuits serve as an example of a means for processing. In various implementations, the modules / circuits may at least partially provide and / or integrate the functionality described above for the components in various embodiments shown.

[0116] According to at least one example of the device 2300, the data storage controller 2310 may include circuits / modules 2320 configured for GAN processing, such as GAN-based decoding of data from Fig. 3-6 and / or GAN-based reconstruction of data as in Fig. 9-13. The data storage controller 2310 may also include circuits / modules 2322 configured for ECC / LDPC processing, such as ECC / LDPC-based decoding of data with error correction as in Fig. 3-6 and Fig. 9-13. The data storage controller 2310 may also include one or more of: circuits / modules 2324 configured to read data from the GAN partition 2309; circuits / modules 2326 configured to read data from the main memory partition 2308; circuits / modules 2328 configured for CRC processing (e.g., to detect but not correct errors in data); circuits / modules 2330 configured for BER processing, such as estimating the BER and comparing the BER to a threshold to control GAN-based or ECC-based processing; circuits / modules 2332 configured for metadata / hint processing, such as using metadata to determine a data type to be processed by a GAN-based procedure; Circuits / Modules 2334 configured for GAN aggregation processing, see Fig. 14-15; circuits / modules 2336 configured for GAN soft-bit processing, see Fig. 7 and Fig. 9; Circuits / Modules 2338 configured for physical soft-bit processing, see Fig. 9; circuits / modules 2340 configured for dissimilarity matrix processing, see Fig. 8; circuits / modules 2342 configured for GAN training using cost functions with penalty terms, as discussed above in connection with the processing of the dissimilarity matrix; circuits / modules 2344 configured for SRGAN processing; circuits / modules 2346 configured to perform monitoring functions, as discussed above in connection with Fig. 11, Fig. 12 and Fig. 14; Circuits / modules 2348 configured to generate confidence information, e.g., physical soft bits or GAN-based soft bits, as described above in conjunction with Fig. 9 or confidence information from spell checkers, predictive text models, etc.; circuits / modules 2350 configured to calculate LLRs; circuits / modules 2352 configured to compare GAN outputs to detect reversed bits, as discussed above in connection with Fig. 9, Fig. 11 and Fig. 12 and to reverse bits again, as discussed in Fig. 14; and circuits / modules 2354 configured to detect and process talbits as discussed above in connection with Fig. 10-13.

[0117] In at least some examples, means for performing the Fig. 23, and / or other functions illustrated or described herein. For example, the means may include one or more of: means, such as circuits / modules 2320, for GAN processing, such as GAN-based decoding of data, see Fig. 3-6 and / or GAN-based reconstruction of data as in Fig. 9-13; means such as circuits / modules 2322 for ECC / LDPC processing, such as ECC / LDPC-based decoding of data with error correction as in Fig. 3-6 and Fig. 9-13; means such as circuits / modules 2324 for reading data from the GAN partition 2309; means such as circuits / modules 2326 for reading data from the main memory partition 2308; means such as circuits / modules 2328 for CRC processing (e.g., to detect errors in data but not to correct them); means such as circuits / modules 2330 for BER processing, such as estimating BER and comparing BER to a threshold to control GAN-based or ECC-based processing; means such as circuits / modules 2332 for metadata / hint processing, such as using metadata to determine a data type to be processed by a GAN-based method; means such as circuits / modules 2334 for GAN aggregation processing, as in Fig. 14-15; means such as circuits / modules 2336 for GAN soft-bit processing, e.g. as in Fig. 7 and Fig. 9; means such as circuits / modules 2338 for physical soft bit processing, e.g. as in Fig. 9; means, such as circuits / modules 2340, for processing the dissimilarity matrix, e.g., as in Fig. 8; means, such as circuits / modules 2342, for GAN training using cost functions with penalty terms, as discussed above in connection with the processing of dissimilarity matrices; means, such as circuits / modules 2344, for SRGAN processing; means, such as circuits / modules 2346, for performing monitoring functions, e.g., as discussed above in connection with Fig. 11, Fig. 12 and Fig. 14; means such as circuits / modules 2348 for generating confidence information, e.g., physical soft bits or GAN-based soft bits, e.g., as discussed above in connection with Fig. 9; means, such as circuits / modules 2350 for calculating LLRs; means, such as circuits / modules 2352 for comparing GAN outputs to detect reversed bits, e.g., as described above in connection with Fig. 9, Fig. 11 and Fig. 12 and to reverse bits again as in Fig. 14; and means such as circuits / modules 2354 for detecting and processing talbits, e.g. as described above in connection with the Fig. 10-13 discussed.

[0118] In another aspect of the disclosure, a non-transitory computer-readable medium is provided having one or more instructions that, when executed by a processing circuit or software module in a DSD controller, cause the controller to perform one or more of the functions or operations listed above.

[0119] In at least some examples, software code for performing the Fig. 23, and / or other functions illustrated or described herein.

[0120] Although the present disclosure is primarily described with reference to DSDs with NVM, aspects of the present disclosure may also be implemented in other devices, such as host computing devices. Furthermore, aspects of the present disclosure are not limited to NVM, and at least some of the GAN-based functions described herein may be applied to data stored in volatile memory. Further aspects

[0121] At least some of the processing circuitry described herein may be generally adapted for processing, including the execution of program code stored on a storage medium. As used herein, the terms "code" or "programming" should be interpreted broadly and include, without limitation, instructions, instruction sets, data, code, code segments, program code, programs, programming, subroutines, software modules, applications, software applications, software packages, routines, subroutines, objects, executable files, threads of execution, procedures, functions, etc., whether referred to as software, firmware, middleware, microcode, hardware description language, or otherwise.

[0122] At least some of the processing circuitry described herein may be configured to retrieve, process, and / or transmit data, control data access and storage, issue commands, and control other desired operations. The processing circuitry may include circuitry configured to implement, in at least one example, the desired programming provided by suitable media. For example, the processing circuitry may be implemented as one or more processors, one or more controllers, and / or other structures configured to execute executable programming.Examples of processing circuitry may include a general-purpose processor, a digital signal processor (DSP), an ASIC, a field-programmable gate array (FPGA) or other programmable logic component, discrete gate or transistor logic, discrete hardware components, or any combination thereof configured to perform the functions described herein. A general-purpose processor may include a microprocessor, as well as any conventional processor, controller, microcontroller, or state machine. At least some of the processing circuitry may also be implemented as a combination of computational components, such as a combination of a controller and a microprocessor, a number of microprocessors, one or more microprocessors in conjunction with an ASIC and a microprocessor, or in any number of other varying configurations.The various processing circuit examples provided herein are for illustrative purposes, and other suitable configurations are contemplated within the scope of the disclosure.

[0123] Aspects of the subject matter described herein may be implemented in any suitable NAND flash memory, such as 3D NAND flash memory. Semiconductor memory devices include volatile memory devices such as DRAM or SRAM devices, NVM devices such as ReRAM, EEPROM, flash memory (which can also be considered a subset of EEPROM), ferroelectric random access memory (FRAM), and MRAM, as well as other semiconductor elements capable of storing information. Each memory device type may have different configurations. For example, flash memory devices may be configured in a NAND or NOR configuration.

[0124] The memory devices can be formed from passive and / or active elements in any combination. As a non-limiting example, passive semiconductor memory elements include ReRAM device elements, which in some embodiments include a resistive switching memory element such as an anti-fuse, phase-change material, etc., and optionally a steering element such as a diode, etc. As another non-limiting example, active semiconductor memory elements include EEPROM and Flash memory device elements, which in some embodiments include elements with a charge storage region such as a floating gate, conductive nanoparticles, or a dielectric charge storage material.

[0125] For the operation of the memory elements and for communication with the memory elements, associated circuitry is typically required. As non-limiting examples, memory devices may include circuitry used to control and drive memory elements to accomplish functions such as programming and reading. This associated circuitry may be located on the same substrate as the memory elements and / or on a separate substrate. For example, a controller for memory read / write operations may be located on a separate controller chip and / or on the same substrate as the memory elements. One of ordinary skill in the art will appreciate that the subject matter described herein is not limited to the described two-dimensional and three-dimensional example structures, but covers all relevant memory structures within the spirit and scope of the subject matter described herein and understood by one of ordinary skill in the art.

[0126] The examples set forth herein serve to illustrate certain concepts of the disclosure. The devices, apparatus, or components illustrated above may be configured to perform one or more of the methods, features, or steps described herein. Those of ordinary skill in the art will understand that these are merely illustrative in nature and that other examples may fall within the scope of the disclosure and the appended claims. Based on the teachings set forth herein, those of ordinary skill in the art should appreciate that an aspect disclosed herein may be implemented independently of all other aspects, and that two or more of these aspects may be combined in various ways. For example, a device may be implemented or a method practiced using any number of the aspects set forth herein.Additionally, such a device may be implemented or such a method may be practiced using other structures, functionality, or structures and functionality in addition to one or more of the aspects set forth herein or others.

[0127] Aspects of the present disclosure have been described above with reference to schematic flowcharts and / or schematic block diagrams of methods, devices, systems, and computer program products according to embodiments of the disclosure. It should be understood that each block of the schematic flowcharts and / or schematic block diagrams, and combinations of blocks in the schematic flowcharts and / or schematic block diagrams, may be implemented by computer program instructions.These computer program instructions may be provided to a processor of a computer or other programmable data processing device to produce a machine, such that the instructions, executing via the processor or other programmable data processing device, create means for implementing the functions and / or actions specified in the schematic flowcharts and / or schematic block diagrams, block or blocks.

[0128] The subject matter described herein may be implemented in hardware, software, firmware, or any combination thereof. Therefore, as used herein, the terms "function," "module," and the like may refer to hardware, which may also include software and / or firmware components for implementing the described feature. In one example implementation, the subject matter described herein may be implemented using a computer-readable medium having stored thereon computer-executable instructions that, when executed by a computer (e.g., processor), direct the computer to perform the functionality described herein.Examples of computer-readable media suitable for implementing the subject matter described herein include non-transitory computer-readable media such as disk storage devices, on-chip memory devices, programmable logic devices, and application-specific integrated circuits. Additionally, a computer-readable medium implementing the subject matter described herein may reside on a single device or computing platform or may be distributed across multiple devices or computing platforms.

[0129] It should also be noted that in some alternative implementations, the functions indicated in the block may occur in a different order than indicated in the figures. For example, two blocks shown in succession may actually execute substantially concurrently, or the blocks may sometimes execute in reverse order, depending on the functionality involved. Other steps and methods are contemplated that correspond in function, logic, or effect to one or more blocks, or portions thereof, of the illustrated figures. Although various types of arrows and lines may be employed in the flowcharts and / or block diagrams, it should be understood that these do not limit the scope of the corresponding embodiments. For example, an arrow may indicate a wait or monitoring period of indefinite duration between the enumerated steps of the illustrated embodiment.

[0130] The various features and processes described above may be used independently or combined in various ways. All possible combinations and sub-combinations are intended to be within the scope of this disclosure. Moreover, in some implementations, certain method, event, state, or process blocks may be omitted. The methods and processes described herein are also not limited to any particular sequence, and the associated blocks or states may be performed in other suitable sequences. For example, described tasks or events may be performed in a different order than the expressly disclosed order, or several of them may be combined into a single block or state. The example tasks or events may be performed serially, in parallel, or in any other suitable manner.Tasks or events may be added to or removed from the disclosed embodiments. The example systems and components described herein may be configured differently than described. For example, elements may be added, removed, or rearranged compared to the disclosed embodiments.

[0131] Those skilled in the art will recognize that information and signals can be represented using a variety of different technologies and techniques. For example, the data, instructions, commands, information, signals, bits, symbols, and chips referred to in the above description can be represented by voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields or particles, or any combination thereof.

[0132] The word "exemplary / example" is used herein to mean "serving as an example, instance, or illustration." Any aspect described herein as "exemplary / example" is not necessarily to be construed as preferential or advantageous over other aspects. Likewise, the term "aspects" does not require that all aspects include the discussed feature, advantage, or mode of operation.

[0133] Although the above descriptions include many specific embodiments of the invention, these should not be construed as limitations on the scope of the invention, but rather as examples of specific embodiments thereof. Accordingly, the scope of the invention should be determined not by the illustrated embodiments, but by the appended claims and their equivalents. Furthermore, reference throughout this specification to "one embodiment," "an embodiment," or similar language means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present disclosure.Therefore, the occurrences of the phrases "in one embodiment," "in embodiments," and similar phrases throughout this specification may or may not refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless expressly stated otherwise.

[0134] The terminology used herein is for the purpose of describing particular aspects only and is not intended to be limiting. As used herein, the singular forms "a," "an," and "the" are intended to include the plural forms (i.e., one or more) unless the context clearly indicates otherwise. Unless expressly stated otherwise, a list of elements is not intended to exclude and / or include some or all of the elements. It is further understood that the terms "comprises," "comprising," "includes," "including," "having," and variations thereof, when used herein, mean "including, but not limited to," unless expressly stated otherwise.That is, these terms may specify the presence of named features, integers, steps, operations, elements, or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, or groups thereof. Furthermore, it is understood that the word "or" has the same meaning as the Boolean operator "OR," that is, encompasses possibilities of "either" and "both" and is not limited to "exclusive or" ("XOR") unless expressly stated otherwise. It is also understood that the symbol " / " between two adjacent words has the same meaning as "or" unless expressly stated otherwise. Furthermore, expressions such as "connected to," "coupled with," or "in communication with / communicating with" are not limited to direct connections unless expressly stated otherwise.

[0135] Any reference to an element herein using a label such as "first," "second," etc., generally does not limit the quantity or order of such elements. Rather, these labels may be used herein as a convenient method of distinguishing between two or more elements or instances of an element. Therefore, reference to first and second elements does not imply that only two elements may be used therein, or that the first element must precede the second element in any way. Unless otherwise noted, a set of elements may also include one or more elements. Additionally, terminology of the form "at least one of A, B, or C" or "A, B, C, or any combination thereof" used in the description or claims means "A, B, or C, or any combination of these elements."This terminology may include, for example, A, B, C, A and B, A and C, A and B and C, 2A, 2B, 2C, 2A and B, etc. As another example, "at least one of: A, B, or C" is intended to cover A, B, C, AB, AC, BC, and ABC, as well as multiples of the same members (e.g., any list that includes AA, BB, or CC). Likewise, "at least one of: A, B, or C" is intended to cover A, B, C, AB, AC, BC, and ABC, as well as multiples of the same members. Similarly, language referring to a list of items linked with "and / or," as used herein, refers to any combination of the items. As an example, "A and / or B" is intended to cover A alone, B alone, or A and B together. As another example, “A, B and / or C” is intended to cover A alone, B alone, C alone, A and B together, A and C together, B and C together, or A, B and C together.

[0136] As used herein, the term "determine" encompasses a wide variety of actions. For example, "determine" can include calculating, computing, processing, deriving, examining, looking up (e.g., looking up information in a spreadsheet, database, or other data structure), determining, and the like. Furthermore, "determine" can include receiving (e.g., receiving information), accessing (e.g., accessing data in a memory), and the like. Furthermore, "determine" can include solving, selecting, choosing, creating, and the like. QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited patent literature

[0000] US 18 / 232,155

[0001] US 63 / 457,951

[0001] US 10,554,227

[0101] Cited non-patent literature

[0000] Goodfellow et al. was developed and in Generative Adversarial Nets. Proceedings of the International Conference on Neural Information Processing Systems (NIPS 2014). pp. 2672-2680

[0025]

Claims

[1] Data storage device comprising: a non-volatile memory (NVM); a data storage controller coupled to the NVM, the data storage controller comprising a processor configured to: Reading data from the NVM; Generating confidence information that represents confidence in the reliability of reading data from the NVM array; and Processing the data with a generative adversarial network (GAN) procedure configured to use the confidence information to reconstruct the data. [2] The data storage device of claim 1, wherein the processor is further configured to generate the confidence information as soft bits. [3] The data storage device of claim 2, wherein the processor is further configured to generate the soft bits by being further configured to determine a Log Likelihood Ratio (LLR) from the read pixel data. [4] The data storage device of claim 1, wherein the processor is further configured to generate the confidence information by being further configured to: Performing a preliminary reconstruction of the data using a variety of different GAN procedures without using confidence information; and Determine whether the multitude of different GAN procedures resulted in the same reconstructed data. [5] The data storage device of claim 4, wherein the processor is further configured to determine whether the plurality of different GAN procedures resulted in the same reconstructed data by being further configured to detect flip bits among the reconstructed data output by the GANs. [6] The data storage device of claim 4, wherein the processor is further configured to quantify the confidence by being further configured to count a number of flip bits. [7] The data storage device of claim 4, wherein the processor is further configured to perform the preliminary reconstruction of the data by being further configured to execute the plurality of different GAN procedures in parallel. [8] The data storage device of claim 1, wherein the processor is further configured to generate the confidence information by being further configured to: Performing first and second acquisitions of bits of the data at first and second different read voltages to obtain first and second read operations of the data; and Identifying a first set of talbits within the first and second reads of the data. [9] The data storage device of claim 8, wherein the processor is further configured to process the data using the GAN procedure by being further configured to: applying the first and second reading of the data to first and second GAN procedures to obtain first and second GAN-modified data; Obtaining a second set of talbits from the first and second GAN output data; Counting the bits within the second set of talbits; Determining whether a count of the second set of talbits is below a threshold; Outputting the first GAN-modified data as a reconstructed image if the count value is below the threshold; and Repeat at least part of the GAN procedure if the count value is not below the threshold to generate additional GAN-modified data. [10] The data storage device of claim 9, wherein the processor, in response to the count value not being below the threshold, is further configured to: reapplying the data to at least one of the GAN procedures to obtain additional GAN ​​output data and thereby obtain additional talbits; Updating the count of Talbits; and Updating the determination of whether the count is below the threshold. [11] The data storage device of claim 1, wherein the processor is further configured to: Representing a pixel of the data, where a first set of bits encodes the color of the pixel and a second set of bits represents the confidence. [12] The data storage device of claim 1, wherein the processor is further configured to: Estimating a bit error rate (BER) from the pixel data read from the NVM; Comparing the BER with a BER threshold; in response to the BER exceeding the BER threshold, generating the confidence information and reconstructing the image data using the GAN procedure configured to use the confidence information; and in response to the BER not exceeding the BER threshold, reconstructing at least some of the data without using the confidence information. [13] The data storage device of claim 1, wherein the processor is further configured to generate information representing the effectiveness of using the confidence information in decoding the data. [14] The data storage device of claim 13, wherein the processor is further configured to adjust read voltages based on the effectiveness representative information. [15] The data storage device of claim 1, wherein the processor is further configured with a plurality of GAN processors configured to perform separate GAN procedures and combine results of the separate GAN procedures. [16] The data storage device of claim 15, wherein the processor is further configured to apply different weights to outputs of the GAN processors for noise compensation. [17] The data storage device of claim 15, wherein the plurality of GAN processors are trained differently to adapt to different types of noise. [18] The data storage device of claim 15, wherein the processor is further configured to read a plurality of copies of the data from the NVM using a plurality of different read voltages and apply respective copies of the data to a corresponding one of the plurality of GAN procedures. [19] The data storage device of claim 15, wherein the processor is further configured to combine the results of the separate GAN procedures by determining a median value for use as the combined result. [20] The data storage device of claim 15, wherein the processor is further configured to combine the results of the separate GAN procedures by determining an average of the results for use as the combined result while excluding maximum and minimum values. [21] The data storage device of claim 1, wherein the processor is further configured to generate the confidence information using one or more of a rule-based procedure or an inference-based procedure. [22] The data storage device of claim 21, wherein the processor is further configured to generate the confidence information using one or more of a spell checker, an autocorrect procedure, a predictive text model, or an artificial intelligence (AI) procedure. [23] Method comprising: Reading data from a non-volatile memory (NVM) of the data storage device; Generating confidence information that represents confidence in the reliability of reading data from the NVM array; and Processing the data with a generative adversarial network (GAN) procedure configured to use the confidence information to reconstruct the data. [24] A device for use in a data storage device, the device comprising: Means for reading data from a non-volatile memory (NVM) of the data storage device; Means for generating confidence information representing confidence in the reliability of reading data from the NVM array; and Means for processing the data with a generative adversarial network (GAN) procedure configured to use the confidence information to reconstruct the data.

Images