Control and management of buffers for wide IO solid state storage

Abstract

A system and method for interfacing a wide IO solid state storage is disclosed. The buffer is configured to store data corresponding to the solid state storage. The buffer includes a first-in first-out (FIFO). The metadata structure is configured to store metadata information including a usage scheme related to the data stored in the buffer. The buffer manager is configured to manage the buffer and the metadata structure based on the metadata information in response to an access request having an access address. The buffer manager performs an access response to the buffer including one of a write access or a read access. The access request is one of a miss or a hit.

Description

[0001] Cross-reference to related applications

[0002] This application claims priority to U.S. Provisional Patent Application Serial No. 63 / 678,529, filed August 1, 2024, the disclosure of which is incorporated herein by reference in its entirety as if fully set forth herein. Technical Field

[0003] This disclosure generally relates to solid-state storage. More specifically, the subject matter disclosed herein relates to buffer control and management for wide I / O solid-state storage. Background Technology

[0004] This background section is intended to provide background information only, and the disclosure of any concepts in this section does not constitute an admission that the concepts are prior art.

[0005] Advances in data science, artificial intelligence (AI), and machine learning (ML) have led to technological transformations across various industries. To adapt to these changes, semiconductor devices and systems have also been developed with new technologies encompassing computing architecture, processor and memory design, network security, and communication interfaces. Among these developments, memory design, or interfaces, has become increasingly important, especially in applications such as mobile devices that require low power consumption and limited physical space.

[0006] In advanced memory design and interface technologies, wide input / output (I / O) interfaces have become popular for three-dimensional (3D) or high-density integrated circuits (ICs), such as low-power double data rate (LPDDR) dynamic random access memory (DRAM) (e.g., LPDDR6). Furthermore, advancements in solid-state drive (SSD) technology for flash memory have enabled high storage capacities for non-volatile memory devices. NAND designs have become the most commonly used type of SSD. However, adapting NAND devices to designs with wide I / O interfaces presents numerous challenges. These challenges include granularity incompatibility, low bandwidth utilization, long latency, high power consumption, high write amplification, and inefficient data buffering.

[0007] The information disclosed in this background section is only intended to enhance the understanding of the background of this disclosure, and therefore may contain information that does not constitute prior art. Summary of the Invention

[0008] To overcome these problems, this paper describes a system and method for data buffering technology for wide I / O interfaces. This technology aims to provide an efficient structure for interfacing with wide I / O solid-state storage. Advantages of this technology include high bandwidth utilization, low latency, low power consumption, and efficient control over the data buffer. In an embodiment, a buffer is configured to store data corresponding to the solid-state storage. The buffer includes a first-in, first-out (FIFO) architecture. A metadata structure is configured to store metadata information, including usage scenarios associated with the data stored in the buffer. A buffer manager is configured to manage the buffer and metadata structure based on the metadata information in response to an access request with an access address. The buffer manager performs an access response to the buffer, including either a write access or a read access. The access request is either a miss or a hit. Attached Figure Description

[0009] In the following sections, aspects of the subject matter disclosed herein will be described with reference to exemplary embodiments shown in the accompanying drawings, wherein:

[0010] Figure 1 This is a block diagram illustrating a system according to an embodiment.

[0011] Figure 2 This is a diagram illustrating a low-power wide I / O circuit according to an embodiment.

[0012] Figure 3 This is a diagram illustrating the buffer control and management circuitry according to an embodiment.

[0013] Figure 4 This is a diagram illustrating the buffer and metadata structure according to an embodiment.

[0014] Figure 5 This is a flowchart illustrating a process for responding to an access request according to an embodiment.

[0015] Figure 6 This is a flowchart illustrating a process for metadata manipulation according to an embodiment.

[0016] Figure 7 This is a diagram illustrating a processing system according to an embodiment. Detailed Implementation

[0017] In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of this disclosure. However, those skilled in the art will understand that the disclosed aspects can be practiced without these specific details. In other instances, well-known methods, processes, components, and circuits have not been described in detail so as not to obscure the subject matter of this disclosure.

[0018] Throughout this specification, references to "an embodiment" or "an embodiment" mean that a particular feature, structure, or characteristic described in connection with that embodiment may be included in at least one embodiment disclosed herein. Therefore, the phrases "in one embodiment," "in an embodiment," or "according to an embodiment" (or other phrases with similar meanings) appearing in various places throughout this specification may not necessarily refer to the same embodiment. Furthermore, particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In this regard, as used herein, the word "exemplary" means "serving as an example, instance, or illustration." Any embodiment described herein as "exemplary" should not be construed as necessarily preferred or advantageous over other embodiments. Additionally, particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. Moreover, depending on the context of the discussion herein, singular terms may include corresponding plural forms, and plural terms may include corresponding singular forms. Similarly, hyphenated entries (e.g., "two-dimensional", "reservation", "pixel-specific", etc.) may occasionally be used interchangeably with their corresponding non-hyphenated versions (e.g., "two-dimensional", "reservation", "pixel-specific", etc.), and uppercase entries (e.g., "Counter Clock", "Row Select", "PIXOUT", etc.) may be used interchangeably with their corresponding non-uppercase versions (e.g., "counter clock", "row select", "pixout", etc.). This occasional interchangeability should not be considered inconsistent with each other.

[0019] Furthermore, depending on the context of the discussion herein, singular terms may include corresponding plural forms, and plural terms may include corresponding singular forms. It should also be noted that the various figures shown and discussed herein (including component diagrams) are for illustrative purposes only and are not drawn to scale. For example, the dimensions of some elements may be exaggerated relative to others for clarity. Additionally, reference numerals are repeated in the figures where deemed appropriate to indicate corresponding and / or similar elements.

[0020] The terminology used herein is for the purpose of describing some exemplary embodiments only and is not intended to limit the claimed subject matter. As used herein, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that, when used in this specification, the terms “comprising” and / or “including” specify the presence of the stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0021] It should be understood that when an element or layer is referred to as being on, "connected to," or "coupled to" another element or layer, it can be directly on, connected to, or coupled to the other element or layer, or there may be intermediate elements or layers. Conversely, when an element is referred to as being "directly on," "directly connected to," or "directly coupled to" another element or layer, there are no intermediate elements or layers. The same reference numerals always denote the same elements. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0022] As used herein, the terms “first,” “second,” etc., serve as labels for nouns that follow them and do not imply any kind of ordering (e.g., spatial, temporal, logical, etc.) unless explicitly defined as such. Furthermore, the same reference numerals may be used across two or more figures to refer to parts, components, blocks, circuits, units, or modules having the same or similar functions. However, such usage is merely for the sake of simplicity and ease of discussion; it does not imply that the construction or architectural details of such components or units are identical in all embodiments, or that such commonly referenced parts / modules are the only way to implement some of the exemplary embodiments disclosed herein.

[0023] Unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this subject pertains. It will be further understood that terms (such as those defined in common dictionaries) should be interpreted as having the same meaning as they have in the context of the relevant field and will not be interpreted in an idealized or overly formal sense unless expressly defined herein.

[0024] As used herein, the term "module" means any combination of software, firmware, and / or hardware configured to provide the functionality described herein in conjunction with modules. For example, software may be embodied as a software package, code, and / or instruction set or instructions, and the term "hardware" as used in any implementation described herein may include, for example, components, hardwired circuitry, programmable circuitry, state machine circuitry, and / or firmware storing instructions executed by programmable circuitry, either individually or in any combination. Modules may be embodied collectively or individually as circuitry forming part of a larger system, such as, but not limited to, integrated circuits (ICs), system-on-a-chip (SoCs), assemblies, etc.

[0025] As used herein, in the context of storage, the term "solid-state" refers to storage technology that uses integrated circuits instead of moving parts (e.g., spinning disks, platters, read / write heads) to store data. The term "flash memory" refers to a type of non-volatile memory that retains data even when power is off. It is commonly used in solid-state drives (SSDs). Flash memory comes in two types: NAND flash and NOR flash. NAND flash memory offers high storage density and low cost per bit, making it suitable for SSDs and mobile applications. NOR flash memory is optimized for random access and is typically used in applications requiring fast code execution.

[0026] As used herein, the term "buffer" in the context of memory refers to a memory device that temporarily stores data or information as part of an operation involving moving data from one location to another. Buffers are typically implemented using static random access memory (RAM) for fast access. Buffers can be organized as standard SRAM or first-in-first-out (FIFO) organization.

[0027] In this embodiment, the buffer is configured to store data corresponding to a solid-state storage device (SSD). The SSD is a wide input / output (WIO) NAND flash memory. The buffer includes a first-in, first-out (FIFO) architecture. A metadata structure is configured to store metadata information, including usage scenarios associated with the data stored in the buffer. The buffer manager is configured to manage the buffer and metadata structure based on the metadata information in response to an access request with an access address. The buffer manager performs an access response including either a write access or a read access to the buffer. The access request is either a miss or a hit. In one embodiment, the usage scenario corresponds to a buffer data item and includes at least one of a validity indicator indicating a valid state, a dirty indicator indicating a modified state, and a relocation indicator indicating a relocation state.

[0028] Figure 1 This is a block diagram illustrating system 100 according to an embodiment. System 100 illustrates the important role of low-power, wide-IO solid-state storage devices in typical AI applications. The AI ​​application in system 100 is a machine learning system with a large language model (LLM). An LLM performs inference and typically comprises two main parts: prompt processing and generating a response to a query. In typical applications, an LLM needs to acquire large amounts of data representing model parameters and forward them to appropriate processing elements, such as a central processing unit (CPU), graphics processing unit (GPU), and neural processing unit (GPU), as well as dedicated processors including application-specific integrated circuits (ASICs). The memory requirements of LLM-based systems include high-bandwidth RAM and wide-IO NAND flash memory devices.

[0029] System 100 includes an internal database 110, a tokenizer 120, an embedded processor 130, a vector database 140, a connection link 145, a context processor 150, a similarity processor 155, a prompting processing unit 160, a large language model (LLM) 170, a response formatter 182, a query processor 184, a user 180, and low-power (LP) wide-I / O storage circuitry 190 (or LP wide I / O circuitry 190). System 100 may include more or fewer components than those described above. System 100 illustrates an exemplary architecture for an artificial intelligence (AI) query and response application. This query and response application receives queries from user 180 and provides responses using LLM 170. This type of application can be implemented in hardware or software, or a combination of both. This application is used as an example to illustrate the role of wide-I / O solid-state storage (e.g., NAND devices) because it utilizes very large computing resources, including large (capacity) storage for data and high (performance) computing. Whether implemented in hardware, software, or a combination of both, the fundamental component of the system is a low-power, wide-I / O solid-state memory circuit 190, which can be used with processing circuitry to perform all or part of the functions of the tokenizer 120, embedded processor 130, context processor 150, similarity processor 155, prompting processing unit 160, LLM 170, response formatter 182, and query processor 184. Some of these components may be part of other components. For example, the tokenizer 120 and embedded processor 130 may be part of the LLM 170.

[0030] Internal database 110 is a database that stores data or information that is private to the organization and not publicly available. Query sessions can be used by company employees, so the data can be company-private or proprietary. If the query is for public information, internal database 110 may not be necessary. Tokenizer 120 processes data from internal database 110 and prepares it for use in subsequent stages. Typical input is text or a sentence. Tokenizer 120 breaks the text into smaller units called tokens, which can be words or phrases, or in other forms that can be processed by other units. Typically, this task may include extracting relevant information from the text and representing that information with meaningful numbers. This can be performed by special programs or special circuitry that can be implemented in an application-specific integrated circuit (ASIC). Such an ASIC requires fast access to memory storing the text and tokens. Wide-IO NAND flash memory devices with interfaces to LPDDR6 devices can be used for this purpose.

[0031] Embedded processor 130 operates on the outputs of the tokenizer and query processor to convert the text representation into a numerical representation conforming to a predetermined format. The embedded representation typically has several numerical fields that can correspond to relevance, relationships, or any characteristics useful for processing. These embedded representations are typically formed as vectors. For example, the text representation “I love New York” can be embedded into a vector with five fields: [0.312, -7.215, 3.126, -0.015, 2.761]. The embedding process can be implemented in hardware using LP-wide I / O circuitry 190, which includes processing circuitry for computing the vector representation and storage elements for storing information retrieved from internal database 110. The resulting vector can be stored in vector database 140 or processed using data read from vector database 140. Vector database 140 stores vectors representing domain knowledge and / or queries. The output of vector database 140 can be passed via connection link 145 to context processor 150 and similarity processor 155 for further processing. The connection link 145 can be a bus, a network connection, or any medium that allows data transfer between the vector database 140 and other devices, including the context processor 150 and the similarity processor 155.

[0032] Context processor 150 provides contextual information to one or more queries. It receives query information from query processor 184. The contextual information expands the meaning of one or more queries to include information related to the content of one or more queries and / or the user's background and experience. For example, queries “What is the capital of California?”, “What to do in Central California?”, and “Where is Yosemite?” can create a travel context. This context will yield vectors related to travel in California, including accommodation information and attractions. Therefore, context processor 150 requires fast computation to perform the search and matching. It also requires a large memory space to store the data. Similarity processor 155 performs matching of candidate vectors with one or more query vectors to locate the vector most relevant to the query. Depending on the format of the query, an appropriate similarity metric can be determined. For example, for vectors with many numerical values, cosine similarity can be used. This similarity metric requires calculating the inner product and magnitude of the two vectors. Thousands of such calculations can be performed when searching for relevant vectors. This number of calculations requires an ASIC dedicated to similarity computation. Therefore, similarity processor 155 can be efficiently implemented by multiple highly integrated circuits including computing elements in the form of ASIC chiplets for fast and parallel computation. In addition, it should have a large memory capacity and a wide I / O interface to provide fast access to vectors. Both the context processor 150 and the similarity processor 155 will also require efficient input / output (I / O) circuitry to perform fast data transfers to and from the vector database 140 and the cue processing unit 160.

[0033] The prompting processing unit 160 receives results from the context processor 150 and the similarity processor 155 to further provide guidance to steer the LLM 170 in the appropriate direction. Due to the large amount of information processed by the LLM 170, it is likely to deviate from the topic area, a phenomenon known as illusion. The prompting processing unit 160 narrows down the search space based on contextual information from the context processor 150 and candidate vectors from the similarity processor 155, as well as additional information such as user profiles, background, or experience. The prompting processing unit 160 can import domain-specific knowledge data to generate appropriate directions for the query. It can interact with the context processor 150 and the similarity processor 155 to generate prompts for the LLM 170. Therefore, highly integrated system or processing elements and localized memory and I / O or interface circuitry, including low-power wide I / O solid-state memory circuitry, will be required.

[0034] The LLM 170 obtains results from the prompting processing unit 160, including the results from the context processor 150 and the similarity processor 155, to generate a response to the query. It also receives query information from the query processor 184. The LLM 170 includes a transformer model with computation partially offloaded to the tokenizer 120, the embedding processor 130, the context processor 150, and the similarity processor 155. It includes encoder and decoder structures for creating and processing contextualized representations of queries, a training model for learning the meaning of queries and processing them, an inference engine for reasoning correct responses, and a fine-tuning structure for refining responses based on the results from the context processor 150 and the similarity processor 155. Typically, the LLM 170 involves a large amount of memory space and computation. Many computations can be performed in parallel with few or no dependencies. Therefore, the LLM 170 will require multiple highly integrated packages with several computational and memory elements with specific algorithms. This is most efficient achieved through multiple ASICs with direct access to local memory devices.

[0035] Response formatter 182 receives one or more responses from LLM 170. These responses correspond to one or more user queries. Response formatter 182 formats these responses into an appropriate format and presentation style, which may include graphics and animations. The results are then delivered to user 180. Due to the computational and I / O interaction volume, response formatter 182 is preferably implemented by a highly integrated subsystem, including multiple processors, memory (e.g., LPDDR6), wide I / O solid-state storage devices, and I / O circuitry.

[0036] Query processor 184 processes queries from user 180. This process may include tokenization and other formatting operations performed by tokenizer 120 to transform the user's query into a form that can be further processed. The results from query processor 184 are delivered to embedded processor 130, context processor 150, and LLM 170. While the computations in query processor 184 may or may not be extensive, they typically require fast processing time and specialized procedures. Therefore, query processor 184 is preferably implemented by a highly integrated subsystem comprising multiple processors, memory (e.g., LPDDR6), low-power wide-I / O solid-state storage circuitry, and I / O circuitry.

[0037] User 180 can be any user of the system and can include individuals, teams, or computerized processes. User 180 can have queries in the public domain and expect results from the public domain. User 180 can also be a user with private queries specific to the platform that User 180 is using. For example, User 180 could be an individual interested in learning about the products offered by Company XYZ. As another example, User 180 could belong to an organization such as a union or association and want to query specific topics that are only relevant to that organization. Under this private setting, the internal database 110 is relevant.

[0038] The LP wide I / O solid-state storage circuit 190 provides highly integrated resources for various storage components in system 100. These resources may include memory for computing, data storage, processing operations, and other dedicated functions. The LP wide I / O solid-state storage circuit 190 can be used in any one of the tokenizer 120, embedded processor 130, context processor 150, similarity processor 155, prompting processing unit 160, LLM 170, resource formatter 182, or query processor 184, or any combination of these elements.

[0039] System 100 is an example illustrating the role of LP wide I / O solid-state memory circuitry in high-performance computing (HC) platforms. Query applications in AI indicate that many HC platforms require several LP wide I / O solid-state memory circuits, including wide I / O NAND SSDs that operate in conjunction with processing units or I / O circuitry. In many cases, the application environment adds additional requirements, including low power consumption, reliable signal integrity, fault tolerance, and reliable operation under extreme conditions, including high temperatures and confined spaces. Examples of other applications that will benefit from highly integrated chip designs include mobile communications (e.g., smartphones, base stations, user equipment), cameras, vehicles, entertainment (e.g., gaming, multimedia, music, movies), technical design (e.g., animation, graphics), medical (e.g., visualization, medical imaging), robotics, drones, automated test equipment, audio processing, speech synthesizers, video and image analytics, vision, automated facial recognition, artificial intelligence (AI) applications, and data centers.

[0040] The following description focuses on several embodiments of a low-power, wide-IO memory circuit 190. These embodiments can be combined to provide a highly integrated and versatile memory circuit.

[0041] Figure 2 This illustrates an embodiment. Figure 1The diagram shows a low-power (LP) wide I / O circuit 190. The LP wide I / O circuit 190 includes a wide I / O memory circuit 210, a main memory circuit 260, a multiplexing circuit (MUX) 270, and a memory controller 280. The LP wide I / O circuit 190 may include more or fewer components than those described above. The LP wide I / O circuit 190 maintains interface compatibility with existing wide I / O DRAM interfaces to minimize modifications and ensure reliable performance. It also improves access time due to the granularity between the main memory in the main memory circuit 260 and the solid-state memory in the wide I / O memory circuit 210.

[0042] Wide I / O memory circuitry 210 includes circuitry that provides wide I / O data access to the SSD memory. It can be referred to as a Level 1 device in a memory expansion organization. It is configured to operate in conjunction with main memory circuitry 260 or existing memory devices in a wide I / O configuration.

[0043] Wide I / O memory circuitry 210 includes a command converter 222, a memory command (MC) queue 224, a solid-state command (SSC) queue 226, a buffer control and management (BCM) circuitry 230, a memory interface 240, and a solid-state storage (SSS) circuitry 250. Wide I / O memory circuitry 210 may include more or fewer of the components described above. Command converter 222 translates commands from memory controller 280 into appropriate commands for SSS circuitry 250. DRAM in main memory circuitry 260 has a small granularity (e.g., 64 bytes), while the granularity in SSS circuitry 250 is larger (e.g., 16kB) due to the wide I / O format. MC queue 224 stores commands translated from command converter 222, formats and arranges them in the appropriate form and order, and schedules their execution. SSC queue 226 stores commands from BCM circuitry 230 and interacts with memory interface 240 to access SSS circuitry 250. The BCM230 provides a structure that allows the SSS circuitry 250 to interface with the main memory circuitry 260 and the memory controller 280 using a wide I / O interface. Furthermore, the BCM 230 utilizes NAND devices to provide a solution for the wide I / O interface, achieving low power consumption, fast latency, and high bandwidth utilization. Figure 3 , Figure 4 , Figure 5 and Figure 6 The BCM 230 is further described below. The memory interface 240 provides an interface to the SSS circuit 250, including receiving commands and data as well as sending data. The SSS circuit 250 is a solid-state memory circuit with a wide I / O configuration. It uses NAND devices as storage elements. It is referred to as high-bandwidth NAND (HBN). As mentioned above, the wide I / O NAND devices in the SSS circuit 250 have large granularity.

[0044] Main memory circuitry 260 includes a memory device that serves as main memory for processing circuitry. It is typically referred to as a level 0 device in memory expansion organization. It may include high-speed DRAM devices, including LPDDR6 devices with speeds of 10.6 Gbps and higher. DRAM devices may have a 24-bit bus width for data. As mentioned above, DRAM devices have a small granularity. DRAM devices may be organized to conform to wide I / O standards. The device may include stacked (3D) or 2.5D integration with logic circuitry to increase bandwidth, reduce latency, and minimize signal interference, making it suitable for mobile applications. Wide I / O can utilize a wide bus width of up to 1024 bits.

[0045] MUX circuitry 270 provides multiplexed control and communication to memory controller 280. MUX circuitry 270 transmits control signals and data including commands, chip select, enable, and data. Memory controller 280 interfaces with a processing device or host 281, including CPU 282, GPU 284, and NPU 286. The interface can be any suitable interface that allows communication via a channel for read and write transactions. In one embodiment, the interface is an Advanced Scalable Interface (AXI). These processing elements can issue command signals to main memory circuitry 260, such as access requests for read and write operations.

[0046] Figure 3 This illustrates an embodiment. Figure 2 A diagram of the BCM circuit 230 is shown. The BCM circuit 230 is configured to address problems in direct access to the HBN, such as long latency and complex control structures. Furthermore, the BCM circuit 230 maintains compatibility with existing wide I / O interfaces, allowing for the inclusion of the HBN in the circuit without requiring modifications. Key features of the BCM circuit 230 include: (1) a cache-like organization for fast access; (2) a metadata structure for providing easy control and response to access requests; (3) a first-in-first-out (FIFO) buffer for providing a simple mechanism for replacement; and (4) a set of functions specifically configured to handle specific problems of the HBN, such as eviction, relocation, and garbage collection. The BCM circuit 230 includes a buffer manager 310, a solid-state (SS) manager 315, a metadata structure 320, a buffer 330, a garbage collection / wear leveling (GC / WL) buffer 340, and a GC / WL controller 350. The BCM circuit 230 may include more or fewer components than those described above.

[0047] Buffer manager 310 is configured to manage buffer 330 and metadata structure 320 based on metadata information in response to access requests with access addresses. It interfaces with MC queue 224 to receive access commands from memory controller 280. Access requests can come from sources such as... Figure 2Any of the processing elements shown, such as CPU 282, GPU 284, or NPU 286. When any of these units executes an access request, the access request is routed to wide I / O memory circuit 210 (in Figure 2 (In the middle), this will be handled by the buffer manager 310. The requested object (data item) may or may not exist in the buffer 230. When the data is not in the buffer, the access request results in a miss. If the data is in the buffer, the access request results in a hit. Depending on whether the access is a miss or a hit and the state of the corresponding data item, the buffer manager will perform appropriate operations to maintain data consistency between the buffer 230 and the SSS circuit 250. The buffer manager 310 manages the operation of the buffering mechanism for cached data from the HBN in the SSS circuit 250. It may include logic circuitry to perform control functions for reading data from the buffer 330 or the SSS circuit 250 and writing data to the buffer 330 or the SSS circuit 250. The buffer manager 310 may perform an access response to the buffer 330, including one of a write access or a read access. In addition, the buffer manager 310 updates metadata information based on the result of each access.

[0048] The SS manager 315 manages access to the SSS (e.g., wide IO NAND devices) in the SSS circuit 250. It interfaces with the SSC queue 226 to provide SS commands to the wide IO NAND devices. The metadata structure 320 is configured to store metadata information related to the status of data items in the buffer 330. The metadata information includes usage schemes related to the data stored in the buffer 330. Each usage scheme corresponds to a buffer data item and includes at least one of a validity indicator indicating a valid status, a dirty indicator indicating a modified status, and a relocation indicator indicating a relocation status. The importance of these indicators or status bits will be explained later.

[0049] Buffer 330 is a low-power (LPW) memory. It is configured to store data corresponding to SSS circuit 250. It comprises a First-In-First-Out (FIFO) cache that stores data on a First-In-First-Out (FIFO) basis. The FIFO is organized as a cache with an N-way set-associative structure. The depth of the FIFO is N, which is the number of ways in the structure. Buffer 330 comprises N×M blocks 335. ijWhere i = 1, ..., N and j = 1, ..., M, N and M are positive integers. FIFO helps reduce complexity and hardware cost, and thus power consumption. Logic circuitry in buffer manager 310 provides control over buffer 330. Examples of control functions include issuing read requests to wide IO NAND devices in SSS circuit 250, returning data from wide IO NAND devices in SSS circuit 250 to any of the hosts 281, and performing access responses to buffer 330.

[0050] As part of the GC / WL operation, the GC / WL buffer 340 stores buffer data items from the buffer 330. GC / WL is an operation specifically for NAND flash memory devices in two cases. In the first case, the NAND needs to erase data blocks before writing new data items to the block. The data to be erased needs to be moved to another location so that its previous location can be erased along with other invalid blocks. All of this data can be collected in the GC / WL buffer 340 so that it can be reused without accessing the SSS circuit 250. In the second case, a relocation bit or flag can indicate that a data item at a location needs to be relocated because the location has been accessed too many times, which can lead to degradation of the data cell. Essentially, when a data cell at a location has received too many program / erase (P / E) cycles, the data cell becomes worn and degraded. Therefore, the data needs to be moved or relocated to another location. In the first case, all data marked with a relocation status or flag will be collected in the GC / WL buffer 340, allowing them to be reused without accessing the SSS circuitry 250. The result is fast processing time and efficient control over data movement.

[0051] The GC / WL controller 350 controls writing to and reading from the GC / WL buffer 340. Control functions may include initiating the GC process, grouping data, issuing read or write requests to the solid-state storage circuitry 250, and communicating with other circuitry or sections. Because GC involves moving data, it is time-consuming and therefore typically performs when the wide I / O storage circuitry 210 is not being used efficiently during a memory cycle. This can be done in a background mode.

[0052] Figure 4 This illustrates the inclusion of, according to an embodiment. Figure 3 The diagram shows structure 400, which includes metadata structure 320 and buffer 330. Structure 400 includes access address 410, buffer 330, and metadata structure 320. Structure 400 is for illustrative purposes only and does not necessarily depict an exact circuit.

[0053] Access address 410 refers to the address of the memory location in the access request issued by host 281. It includes tag 412, index 414, and offset 416, similar to the address field in cache memory. Offset 416 specifies the byte in the cache line. Index 414 identifies the group. Tag 412 identifies the block in the specified group. Tag 412 is compared with tags stored in buffer 330 to determine whether there is a hit or a miss.

[0054] Buffer 330 is shown as having a 4-way group-associative organization. It comprises four arrays corresponding to the four ways 4201, 4202, 4203, and 4204. Each row of each array stores a label field T(i,j) and a data field D(i,j), where i=0, M-1 and j=0, 3. Each row of the four arrays 4201, 4202, 4203, and 4204 corresponds to an index number.

[0055] Metadata structure 320 is organized in the same manner as buffer 330 to store metadata information. The metadata information includes status bits or flags associated with data items in buffer 330. The usage scheme corresponds to the data items in buffer 330 and includes at least one of the following: a validity indicator, bit, or flag indicating a valid state; a dirty indicator, bit, or flag indicating a modified state; and a relocation indicator, bit, or flag indicating a relocation state. A valid state, when asserted, reflects that the data item at the accessed address is valid and has been correctly read, stored, or updated. An invalid state indicates that the data item at the accessed location is invalid or has not been correctly stored or written. A dirty state indicates that the data item has been modified or overwritten, and its value has changed from its original value at the time of initial loading, or from its last updated value while in a valid state. A relocation state indicates that the data item needs to be relocated to another location because its integrity may be compromised due to excessive P / E cycles. Metadata structure 320 has three arrays 430, 440, and 460, respectively corresponding to valid states, dirty states, and relocation states. Each row of each array corresponds to an index as shown in buffer 330. Each array has four columns corresponding to the four paths in the 4-way associative group buffer 330. Column 435 j 445 j and 455 j (Where j=1,…,4) correspond to the V, D, and R states of path j-1, respectively. For example, state 4354 at index 1 is the valid state of path 3 with the value 1. Whenever an access operation causes a state change, the buffer manager 310 updates the state indicator 435. j 's, 445 j 's and 455 j 's.

[0056] Figure 5This is a flowchart illustrating a process 500 for responding to an access request according to an embodiment.

[0057] At the start, process 500 receives an access request from the host (block 510). The access request can be a read access or a write access. This request can be sent from host 281 to MUX 270 and command converter 222, and then to BCM circuit 230. Process 500 then checks metadata information (block 515). Metadata information includes bits for miss / hit, valid, dirty, and relocation. Process 500 then determines whether a hit exists in the access request (block 520). The hit / miss determination is based on a comparison between the tag field of the address in the request and the tag stored in buffer 330. If a match exists, a hit is declared. Otherwise, a miss is declared. If a miss exists (no at block 520), then process 500 issues a read request to the wide I / O memory circuit or HBN (block 525). The read request is executed regardless of whether the access request is a read access or a write access. The reason a read request is still required even when the host requests a write operation is that the data to be written may have a size different from the word size (e.g., 8 bits), and the written data needs to be combined or merged with data of its nominal size. Next, procedure 500 determines whether data has been returned from HBN (block 530). If not, procedure 500 loops back to block 530 to wait for the data to be returned. If no data is returned after a predefined time period, procedure 500 may invoke the error handling procedure.

[0058] If the data is successfully returned, process 500 determines whether the host access request is a read request (block 540). If yes (yes at block 540), process 500 returns the read data to the host (block 550) and continues with normal read / write response execution (block 555). This may include pushing the read data into buffer 330. Next, process 500 updates the metadata information corresponding to the access address (block 560). This may include asserting or deasserting metadata status bits. For example, after a read miss and data being loaded into the buffer, the status bits may be updated from invalid to valid. Process 500 then terminates.

[0059] If the host access request is a write request (No at block 540), process 500 merges the write data into the data read from HBN (for write misses) or the data read from the buffer (for write hits). Then, process 500 proceeds to block 555 to perform the read / write response as described above. Next, process 500 updates the metadata information corresponding to the access address (block 560) and then terminates.

[0060] Figure 6This is a flowchart illustrating a process 600 for metadata operations according to an embodiment. For illustrative purposes, process 600 is shown as a standalone process. In practice, process 600 is performed in conjunction with process 500 or any other process serving access requests (read or write) from the host. When metadata operations (especially dirty and relocation status bits) are involved, process 600 may be incorporated into any other process.

[0061] At the start, process 600 checks the metadata information in the metadata structure (block 610). Then, process 600 determines whether eviction is possible (block 620). Eviction occurs when there is no more space in buffer 330 to accept new data. When this happens, data items in the buffer are evicted to make room for new data. If no eviction occurs (no at block 620), process 600 continues to execute read / write responses as appropriate (block 680), update metadata as needed (block 690), and then terminates.

[0062] If an evict occurs (yes at block 620), process 600 evicts the tail block from the buffer (block 630). Next, process 600 determines whether the evicted data is dirty or relocated. This determination can be performed independently and in parallel. The determination logic or check circuitry includes logic circuitry that can perform logical operations independently and in parallel based on the status bits of the metadata. Although the flowchart may show a sequential process, the operations or blocks in process 600 can be executed in parallel. In particular, relocation and dirty state checks can be performed simultaneously. In some cases, process 600 may perform relocation only due to read interference. In other cases, process 600 may need to write data back to the memory circuitry due to dirty writes. In some rare cases, process 600 may need to handle both relocation and dirty states together. At block 640, the relocation status is checked. If no relocation occurs, process 600 proceeds to block 680. Otherwise (yes at block 640), process 600 writes the tail block from the buffer to the GC / WL buffer (block 650). GL / WL controller 350 (in Figure 3In block 660, the GC operation will be processed. Data marked with relocation status bits that have been moved to the GC / WL buffer will be reused as appropriate, without needing to be stored in the SSS circuit 250. Then, process 600 proceeds to block 680. At block 660, the dirty state in the metadata is checked. If no dirty state exists or the dirty bit is denied or de-asserted, then process 600 proceeds to block 680. Otherwise (yes at block 660), process 600 issues a wide I / O memory write request to the SSS circuit and writes the evicted data to the SSS circuit (block 670) and proceeds to block 680. At block 680, process 600 performs a read / write response as appropriate (block 680). Next, process 600 updates the metadata information (block 690) and then terminates.

[0063] Figure 7 This is a diagram illustrating a computing or processing system 700 according to an embodiment. The computing system 700 may be a system in which wide I / O memory circuitry can be deployed. It can supplement or replace... Figure 1 Any one or more of the blocks shown. It includes a central processing unit (CPU) or processor 710, a bus 720, and a platform controller hub (PCH) 730. The PCH 730 may include a graphics display controller (GDC) 740, a memory controller 750, and an input / output (I / O) controller 760. The processing system 700 may include more or fewer components than those described above. Furthermore, components may be integrated into another component. Figure 7 As shown, all controllers 740, 750, and 760 are integrated into PCH 730. Integration can be partial and / or overlapping. For example, GDC 740 can be integrated into processor 710, I / O controller 760 and memory controller 750 can be integrated into a single controller, and so on.

[0064] Processor 710 is a programmable device capable of executing programs or sets of instructions to perform tasks. It can be a general-purpose processor, a digital signal processor, a microcontroller, or a specially designed processor, such as a design from an application-specific integrated circuit (ASIC). It can include a single core or multiple cores. Each core can have multiplexing and multithreading. Processor 710 can have simultaneous multithreading features to further utilize the parallelism caused by multiple threads across multiple cores. Furthermore, processor 710 can have multiple levels of internal cache. It can be... Figure 2 The CPU 282 in the system.

[0065] Bus 720 can be any suitable bus that connects processor 810 to other devices, including PCH 730. For example, bus 720 can be a direct media interface (DMI).

[0066] The PCH 730 is a highly integrated chipset that includes many functions to provide interfaces to a variety of devices such as memory devices, input / output devices, storage devices, network devices, etc.

[0067] The I / O controller 760 controls input devices 768 (e.g., stylus, keyboard and mouse, microphone, image sensor) and output devices (e.g., audio devices, speakers, scanners, printers) as well as mass storage 764. Mass storage 764 may also include CD-ROMs, hard drives, and SSDs. It also features a network interface card (NIC) 770, which provides interfaces to network and wireless media 775.

[0068] Memory controller 750 controls memory devices, such as main memory 752 and wide I / O storage 754. Main memory 752 includes random access memory (RAM) and / or read-only memory (ROM), as well as other types of memory, such as cache memory or SSD. Main memory 752 may store instructions or programs loaded from mass storage devices that, when executed by processor 710, cause processor 710 to perform the operations described above. It may also store data used in the operation. ROM may include instructions, programs, constants, or data that are retained regardless of whether power is on. Instructions or programs may correspond to the functions described above.

[0069] The GDC 740 controls the display device 745 and provides graphical operation. It can be integrated within the processor 710. It typically features a graphical user interface (GUI) to allow interaction with users who can send commands or activate functions.

[0070] Additional devices or bus interfaces can be used for interconnection and / or expansion. Some examples may include a Peripheral Component Interconnect High Speed ​​(PCIe) bus, a Universal Serial Bus (USB), etc.

[0071] All or part of the embodiments can be implemented according to specific features and functions through various means depending on the application. These devices can include hardware, software, or firmware, or any combination thereof. Hardware, software, or firmware elements can have several modules coupled to each other. Hardware modules are coupled to another module via mechanical, electrical, optical, electromagnetic, or any physical connection. Software modules are coupled to another module via function, procedure, method, subroutine or subroutine call, jump, link, parameter, variable and argument passing, function return, etc. Software modules are coupled to another module to receive variables, parameters, arguments, pointers, etc., and / or generate or pass results, update variables, pointers, etc. Firmware modules are coupled to another module via any combination of the above hardware and software coupling methods. Hardware, software, or firmware modules can be coupled to any one of other hardware, software, or firmware modules. Modules can also be software drivers or interfaces that interact with an operating system running on a platform. Modules can also be hardware drivers for configuring, setting, initializing, sending data to hardware devices, and receiving data from hardware devices. Devices can include any combination of hardware, software, and firmware modules.

[0072] Embodiments of the subject matter and operation described in this specification can be implemented in digital electronic circuits, or in computer software, firmware, or hardware, including the structures disclosed in this specification and their structural equivalents, or combinations thereof. Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a computer storage medium for execution by a data processing apparatus or for controlling the operation of a data processing apparatus. Alternatively or additionally, program instructions can be encoded on artificially generated propagated signals, such as machine-generated electrical, optical, or electromagnetic signals, generated to encode information for transmission to a suitable receiver device for execution by the data processing apparatus. The computer storage medium can be or is included in a computer-readable storage device, a computer-readable storage substrate, a random or serial access memory array or device, or a combination thereof. Furthermore, while the computer storage medium is not a propagated signal, it can be a source or destination of computer program instructions encoded in artificially generated propagated signals. The computer storage medium can also be one or more separate physical components or media (e.g., multiple CDs, disks, or other storage devices), or included therein. Furthermore, the operations described in this specification can be implemented as operations performed by a data processing apparatus on data stored on one or more computer-readable storage devices or received from other sources.

[0073] While this specification may contain numerous specific details of implementation, these details should not be construed as limiting the scope of any claimed subject matter, but rather as descriptions of features specific to particular embodiments. Certain features described in the context of individual embodiments in this specification may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments. Furthermore, although features may be described above as functioning in certain combinations and even initially claimed in this way, one or more features from a claimed combination may be removed from the combination in some cases, and the claimed combination may be for sub-combinations or variations thereof.

[0074] Similarly, although the operations are depicted in a specific order in the accompanying drawings, this should not be construed as requiring these operations to be performed in the specific order shown or sequentially, or to perform all the shown operations to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of the various system components in the above embodiments should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0075] Therefore, specific embodiments of the subject matter have been described herein. Other embodiments are within the scope of the following claims. In some cases, the actions set forth in the claims may be performed in a different order and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific order or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing may be advantageous.

[0076] As those skilled in the art will recognize, the innovative concepts described herein can be modified and altered over a wide range of applications. Therefore, the scope of the claimed subject matter should not be limited to any of the specific exemplary teachings discussed above, but rather is instead defined by the appended claims.

Claims

1. An apparatus comprising: A buffer is configured to store data corresponding to a solid-state storage device, the buffer including a first-in-first-out (FIFO) buffer. A metadata structure is configured to store metadata information, which includes usage schemes related to the data stored in the buffer; and A buffer manager is configured to manage the buffers and the metadata structure based on the metadata information in response to an access request with an access address. The buffer manager executes access responses to the buffer, the access responses including either a write access or a read access, and The access request is either a miss or a hit.

2. The apparatus according to claim 1, wherein, The usage scheme corresponds to a buffer data item and includes at least one of a valid indicator indicating a valid state, a dirty indicator indicating a modified state, and a relocation indicator indicating a relocation state.

3. The apparatus according to claim 1, wherein, The solid-state storage is a wide input / output (wide IO) NAND storage.

4. The apparatus according to claim 1, wherein, In the event of a miss, the buffer manager sends a read request to the solid-state storage to obtain the stored data.

5. The apparatus according to claim 4, wherein, In response to a write access with written data, the buffer manager merges the written data into the stored data and performs the access response to the buffer.

6. The apparatus according to claim 4, wherein, In response to a read access, the buffer manager returns the stored data to the host and executes the access response to the buffer.

7. The apparatus according to claim 1, wherein, During the evicting operation, the buffer manager evicts buffer data items from the tail block of the buffer.

8. The apparatus according to claim 7, wherein, During the eviction operation, the buffer manager also issues a write request to the solid-state storage based on the dirty indicator of the buffer data item and writes the buffer data item to the solid-state storage.

9. The apparatus according to claim 7, wherein, During the eviction operation, the buffer manager also writes the tail block from the buffer to the garbage collector buffer based on the relocation indicator of the buffer data item.

10. The apparatus according to claim 1, wherein, The buffers are organized as N-way group interconnections.

11. A method comprising: The data corresponding to the solid-state storage is stored in a buffer that includes a first-in, first-out (FIFO) mechanism; Metadata information is stored in a metadata structure, the metadata information including usage schemes related to the data in the buffer; as well as In response to an access request with an access address, the buffer and the metadata structure are managed based on the metadata information. The management includes executing access responses to the buffer, the access responses including one of a write access or a read access, and The access request is either a miss or a hit.

12. The method according to claim 11, wherein, The usage scheme corresponds to a buffer data item and includes at least one of a valid indicator indicating a valid state, a dirty indicator indicating a modified state, and a relocation indicator indicating a relocation state.

13. The method according to claim 11, wherein, The solid-state storage is a wide input / output (wide IO) NAND storage.

14. The method according to claim 11, wherein, Management includes issuing read requests to the solid-state storage to obtain stored data when a hit occurs.

15. The method according to claim 14, wherein, Management includes, in response to a write access with written data, merging the written data into the stored data, and performing the access response to the buffer.

16. The method of claim 14, wherein, Management includes returning the stored data to the host in response to a read access and executing the access response to the buffer.

17. The method according to claim 11, wherein, Management includes removing buffer data items from the tail block of the buffer during the evicting operation.

18. The method according to claim 17, wherein, Management also includes issuing a write request to the solid-state storage based on the dirty indicator of the buffer data item during the eviction operation and writing the buffer data item to the solid-state storage.

19. The method of claim 17, wherein, Management also includes writing tail blocks from the buffer to the garbage collector buffer based on the relocation indicator of the buffer data item during the eviction operation.

20. A system comprising: Host processor; Solid state storage; and The buffer control and management circuitry includes: A buffer is configured to store data corresponding to the solid-state storage, the buffer including first-in-first-out (FIFO). Metadata structure, configured to store metadata information, including usage schemes related to the data stored in the buffer; and A buffer manager is configured to manage the buffers and the metadata structure based on the metadata information in response to an access request with an access address. The buffer manager executes access responses to the buffer, the access responses including either a write access or a read access, and The access request is either a miss or a hit.

Images