3D stacked high bandwidth mixed memory control system, method, medium, terminal and program product oriented to CXL memory semantics

Abstract

The application provides a 3D stacked high-bandwidth hybrid memory control system, method, medium, terminal and program product for CXL memory semantics. The system comprises a logic base chip and a NAND Flash stacked structure. A plurality of NAND Flash chips are stacked in the vertical direction and are connected with the logic base chip through corresponding TSV channels. The logic base chip performs on-chip cache hit judgment according to a CXL address. When the cache is not hit, the data heat level of the data to be accessed is determined according to the historical access frequency. The target NAND Flash chip storing the data to be accessed is determined according to the address mapping information corresponding to the CXL address. The CXL memory access request is scheduled in combination with the load state of the target NAND Flash chip and the corresponding TSV channel and the data heat level to perform data access. The application can improve the utilization rate of the back-end storage resource and the TSV channel, improve the cache utilization efficiency, and relieve the problem of mismatch between the CXL front-end bandwidth and the NAND back-end bandwidth.

Description

Technical Field

[0001] This application relates to the field of storage system technology, and in particular to a 3D stacked high-bandwidth hybrid memory control system, method, medium, terminal and program product oriented to CXL memory semantics. Background Technology

[0002] With the development of large language models and artificial intelligence technologies, the demand for memory capacity and access bandwidth in computing systems is constantly increasing, with memory capacity requirements gradually expanding from the GB level to the TB level. CXL (Compute Express Link) technology can provide hosts with low-latency, byte-addressable memory expansion capabilities, and is therefore increasingly being used in the construction of large-capacity memory pools. CXL-based hybrid memory devices typically employ a storage architecture that combines DRAM and NAND Flash, where DRAM is used to cache frequently accessed data, and NAND Flash is used to provide larger data storage capacity, achieving a balance between memory access performance, storage capacity, and usage cost.

[0003] In existing CXL hybrid memory devices, the controller and NAND Flash chips are typically connected via PCB-level interconnects. Due to limitations in the number of parallel channels between the controller and NAND Flash chips, and the bandwidth of the PCB-level interconnect, the data access bandwidth of the back-end NAND Flash is often lower than the data transfer bandwidth provided by the CXL front-end interface. For example, when using a PCIe 5.0 x16 interface, the front-end interface can provide higher data transfer bandwidth, while the limited bandwidth of the back-end NAND access channels is difficult to match with the bandwidth of the front-end interface. When data to be accessed misses the cache and needs to access the NAND Flash, the back-end data access path easily becomes the bandwidth bottleneck in the entire CXL memory access process, making it difficult to fully utilize the high bandwidth capability of the CXL interface.

[0004] On the other hand, existing CXL hybrid memory devices typically use DRAM, independently configured outside the controller, as a cache, combined with general cache management or prefetching mechanisms for data access. The off-chip DRAM needs to be accessed through the data path between the controller and the external cache, and its access latency and transmission bandwidth limit the response speed of CXL.mem requests. Simultaneously, general cache management methods usually do not specifically address the byte-addressable access method and cache-related semantics of CXL. In application scenarios with low data access locality, such as large language model inference, different data may frequently enter and exit the cache, easily causing cache pollution. This prevents the data retained in the limited cache space from effectively matching the actual access frequency, leading to a decrease in cache hit rate.

[0005] Therefore, existing CXL hybrid memory devices still suffer from problems such as a mismatch between the front-end CXL interface and the back-end NAND Flash data access bandwidth, as well as the difficulty in fully adapting the cache access path and cache management method to the characteristics of CXL memory access. It is necessary to further improve the internal storage connection structure, cache structure, and data access scheduling method of CXL hybrid memory devices in order to improve the parallel utilization capability of back-end storage resources and improve the cache utilization efficiency during CXL memory access. Summary of the Invention

[0006] In view of the shortcomings of the prior art described above, the purpose of this application is to provide a 3D stacked high-bandwidth hybrid memory control system, method, medium, terminal and program product for CXL memory semantics, to solve the above problems.

[0007] To achieve the above and other related objectives, a first aspect of this application provides a 3D stacked high-bandwidth hybrid memory control system oriented towards CXL memory semantics, comprising: a logic substrate chip, including a CXL interface unit, a front-end controller, a back-end controller, an on-chip cache array, and a channel scheduling unit; a NAND Flash stacking structure, including multiple NAND Flash chips stacked vertically above the logic substrate chip, each NAND Flash chip being connected to the logic substrate chip via a corresponding TSV channel; the CXL interface unit is used to receive and parse CXL memory access requests sent by a host to obtain the CXL address corresponding to the CXL memory access request; the front-end controller is used to perform cache hit judgment on the on-chip cache array based on the CXL address, accessing the on-chip cache array when a cache hit occurs, and determining the data hotness level of the data to be accessed pointed to by the CXL address based on the historical access frequency of the CXL address when a cache miss occurs; the channel scheduling unit is used to determine the target NAND Flash chip storing the data to be accessed based on the address mapping information corresponding to the CXL address, and obtain the target NAND... The backend controller monitors the load status of the Flash chip and its corresponding TSV channel, and schedules the CXL memory access request based on the load status and the data popularity level of the data to be accessed, directing the CXL memory access request to the target NAND Flash chip; the backend controller controls the target NAND Flash chip to perform data access corresponding to the CXL memory access request through the TSV channel corresponding to the target NAND Flash chip.

[0008] In some embodiments of the first aspect of this application, the step of determining a cache hit in the on-chip cache array based on the CXL address and accessing the on-chip cache array when a cache hit occurs specifically includes: performing byte alignment and cache line mapping on the CXL address to obtain a cache line mapping result; accessing the cache tag RAM based on the cache line mapping result to obtain the corresponding cache tag information; determining whether the data to be accessed pointed to by the CXL address hits the on-chip cache array based on the cache tag information; and when a cache hit occurs, reading the data to be accessed from the on-chip cache array and returning the read response corresponding to the data to be accessed to the host through the CXL interface unit.

[0009] In some embodiments of the first aspect of this application, the load state includes instantaneous load and queue depth; the channel scheduling unit is further configured to, for multiple concurrent CXL memory access requests, determine the target NAND Flash chip corresponding to each CXL memory access request according to the address mapping information corresponding to each CXL memory access request, obtain the instantaneous load and queue depth of each target NAND Flash chip and the corresponding TSV channel, and schedule each CXL memory access request to the corresponding target NAND Flash chip by combining the data heat level of the data to be accessed corresponding to each CXL memory access request.

[0010] In some embodiments of the first aspect of this application, the front-end controller is further configured to select a replacement cache line when a cache miss occurs; the back-end controller is further configured to, after the target NAND Flash chip completes the data reading of the data to be accessed, determine the cache residency policy and cache replacement policy of the data to be accessed in the on-chip cache array according to the data popularity level of the data to be accessed, and update the corresponding cache line according to the cache residency policy and cache replacement policy; wherein, for the data to be accessed with a data popularity level of "hot", the data to be accessed is retained in the on-chip cache array; for the data to be accessed with a data popularity level of "warm", the replacement cache line is cache-replaced according to the cache replacement policy; and for the data to be accessed with a data popularity level of "cold", the residency priority of the data to be accessed in the on-chip cache array is reduced.

[0011] In some embodiments of the first aspect of this application, the logic substrate chip is further configured to control the migration of the data to be accessed among multiple NAND Flash chips according to the data popularity level of the data to be accessed, so as to adjust the storage location of the data to be accessed in the NAND Flash stack structure.

[0012] In some embodiments of the first aspect of this application, when a cache miss occurs, the front-end controller is further configured to create a trace entry corresponding to the CXL memory access request in a missing status holding register used to record cache miss requests; the back-end controller is further configured to, after the target NAND Flash chip completes data reading, retrieve a trace entry related to the logical block address from the missing status holding register according to the logical block address where the data reading was completed, forward the CXL memory access request corresponding to the retrieved trace entry to the front-end controller to continue executing subsequent cache access, and release the trace entry.

[0013] To achieve the above and other related objectives, a second aspect of this application provides a 3D stacked high-bandwidth hybrid memory control method oriented towards CXL memory semantics, applied to a hybrid memory system. The hybrid memory system includes a logic substrate chip and a NAND Flash stacked structure. The NAND Flash stacked structure includes multiple NAND Flash chips stacked vertically above the logic substrate chip, each NAND Flash chip being connected to the logic substrate chip via a corresponding TSV channel. The method includes: receiving and parsing a CXL memory access request sent by a host to obtain the CXL address corresponding to the CXL memory access request; performing a cache hit determination on an on-chip cache array disposed on the logic substrate chip based on the CXL address; accessing the on-chip cache array when a cache hit occurs; determining the data hotness level of the data to be accessed pointed to by the CXL address based on the historical access frequency of the CXL address when a cache miss occurs; determining the target NAND Flash chip storing the data to be accessed based on the address mapping information corresponding to the CXL address, and obtaining the target NAND Flash chip. The load status of the Flash chip and its corresponding TSV channel is monitored, and the CXL memory access request is scheduled according to the load status and the data popularity level of the data to be accessed. The CXL memory access request is scheduled to the target NAND Flash chip, and the data access corresponding to the CXL memory access request is performed on the target NAND Flash chip through the TSV channel corresponding to the target NAND Flash chip.

[0014] To achieve the above and other related objectives, a third aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method.

[0015] To achieve the above and other related objectives, a fourth aspect of this application provides a computer program product comprising computer program code that, when executed on a computer, causes the computer to implement the method.

[0016] To achieve the above and other related objectives, a fifth aspect of this application provides an electronic terminal, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the method.

[0017] As described above, this application has the following beneficial effects:

[0018] This application enables parallel data access to multiple NAND Flash chips through 3D stacking of multiple NAND Flash chips and corresponding TSV channels. It determines the target NAND Flash chip corresponding to each access request based on address mapping information, and schedules access requests by combining the load status of the target NAND Flash chip and the corresponding TSV channel. This improves the back-end storage bandwidth and TSV channel utilization, and alleviates the problem of mismatch between CXL front-end bandwidth and NAND back-end bandwidth.

[0019] This application improves CXL memory access efficiency by setting an on-chip cache array in the logic substrate chip and directly reading cached data when a cache hit occurs, thereby shortening the data access path, reducing cache access latency, and improving CXL memory access efficiency.

[0020] This application determines the data popularity level based on the historical access frequency of the CXL address, and uses the data popularity level for access request scheduling, cache residency, and data migration, so that access requests corresponding to high-popularity data can receive appropriate scheduling processing. At the same time, the cache residency and NAND storage location are adjusted according to the data popularity, thereby improving cache utilization and reducing cache pollution.

[0021] This application integrates CXL interface processing, cache management, TSV channel scheduling, and NAND Flash access into a logic substrate chip and adopts a 3D stacking structure to reduce board-level discrete connections and long-distance data transmission, improve system integration, and reduce the need for host software to participate in data management. Attached Figure Description

[0022] Figure 1 The diagram shown is a structural schematic of a 3D stacked high-bandwidth hybrid memory control system oriented towards CXL memory semantics in one embodiment of this application.

[0023] Figure 2 The diagram shown is a schematic representation of the BEC state machine in one embodiment of this application.

[0024] Figure 3The diagram shown is a flowchart illustrating a 3D stacked high-bandwidth hybrid memory control method for CXL memory semantics in one embodiment of this application.

[0025] Figure 4 The diagram shown is a structural schematic of an electronic terminal according to an embodiment of this application. Detailed Implementation

[0026] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be noted that, unless otherwise specified, the following embodiments and features in the embodiments can be combined with each other.

[0027] In the embodiments of this application, the terms "first" and "second" are used to distinguish identical or similar items with essentially the same function and effect. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and that the terms "first" and "second" do not necessarily imply that they are different.

[0028] It should be noted that, in the embodiments of this application, the words "exemplary" or "for example" indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0029] In this application embodiment, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0030] Before providing a further detailed description of the present invention, the nouns and terms used in the embodiments of the present invention are explained, and the nouns and terms used in the embodiments of the present invention are subject to the following interpretations:

[0031] <1> CXL (Compute Express Link): A protocol for high-speed interconnection between processors, memory, and acceleration devices, used to enable memory access transaction transmission.

[0032] <2> TSV (Through-Silicon Via): An interconnect structure that forms a vertical electrical connection through a silicon substrate, used to enable vertical signal transmission between stacked chips.

[0033] <3> eDRAM (embedded Dynamic Random Access Memory): A type of dynamic random access memory integrated inside a logic chip, which can be used as an on-chip cache.

[0034] <4> PCIe (Peripheral Component Interconnect Express): A serial computer expansion bus interface used for high-speed data transfer between devices.

[0035] <5> MSHR (Miss Status Holding Register): A storage structure used to record and track memory access requests that have been missed in the cache but have not yet been processed.

[0036] <6> WB (Write Buffer): A buffer structure used to temporarily store data corresponding to write requests.

[0037] Figure 1 This is a schematic block diagram of a 3D stacked high-bandwidth hybrid memory control system 100 oriented towards CXL memory semantics, provided in an embodiment of this application. Figure 1 As shown, the system includes: a logic substrate chip and a NAND Flash stacked structure.

[0038] The hybrid memory system in this embodiment adopts a 3D stacked architecture, comprising a logic substrate chip and a NAND Flash stacked structure disposed on top of the logic substrate chip. The NAND Flash is non-volatile flash memory, and the NAND Flash stacked structure includes multiple NAND Flash chips stacked vertically. The number of NAND Flash chip stacking layers can be configured to 4, 8, 16, or more layers depending on capacity requirements. The logic substrate chip can be manufactured using 5nm or 3nm logic processes and serves as the core for computation and scheduling of the entire hybrid memory system. Each NAND Flash chip is connected to the logic substrate chip through a corresponding TSV channel, penetrating the silicon substrate to achieve vertical signal conduction between chip layers. Microbumps are also provided between chip layers to achieve electrical and mechanical connections between adjacent chip layers. The microbumps and TSVs together constitute a vertical physical interconnect structure, enabling high-density data transmission paths between the logic substrate chip and the multiple NAND Flash chips. Each NAND Flash chip has a corresponding TSV channel, and each NAND Flash chip can receive read and write commands and execute data access through its corresponding TSV channel, allowing different NAND Flash chips to work in parallel.

[0039] The logic substrate chip includes: a CXL interface unit, a front-end controller, a back-end controller, an on-chip cache array, and a channel scheduling unit.

[0040] In one embodiment of this application, the on-chip cache array is an eDRAM cache array, which is an embedded dynamic random access memory directly integrated on the logic substrate chip, used to cache hot data and metadata.

[0041] Compared to the DRAM located externally to the controller, the eDRAM cache array interacts with various functional modules within the controller via on-chip interconnects. Specifically, the channel scheduling unit can be a TSV control unit, used to uniformly manage the TSV channels between the logic substrate chip and each NAND Flash chip, and to perform request scheduling based on the load status of each NAND Flash chip and its corresponding TSV channel.

[0042] The front-end controller is used to process memory access requests from the CXL interface unit in a pipeline manner, and to perform address resolution, logical address to physical address mapping, on-chip cache hit determination, and request scheduling.

[0043] Specifically, the logical address to physical address mapping is used to characterize the current physical storage location of the data corresponding to the CXL address in the NAND Flash stack structure. In one embodiment, the same data to be accessed corresponds to one NAND Flash chip under the current valid address mapping relationship; when the front-end controller determines that the data to be accessed has not hit the on-chip cache array, it can determine the NAND Flash chip currently storing the data to be accessed based on the address mapping information corresponding to the CXL address, and use that NAND Flash chip as the target NAND Flash chip corresponding to the current CXL memory access request.

[0044] After identifying the target NAND Flash chip, the channel scheduling unit schedules the timing of the current CXL memory access request or the processing order between different CXL memory access requests based on the load status of the target NAND Flash chip and its corresponding TSV channel, and in conjunction with the data popularity level of the data to be accessed. The load status and data popularity level are used for scheduling the current access request, not for redirecting the current access request to other NAND Flash chips that do not store the data to be accessed.

[0045] The back-end controller is used to manage the storage operations of the NAND Flash chip, including issuing page read / write commands, block erasure, wear leveling, and garbage collection.

[0046] The CXL interface unit connects to the host-side PCIe / CXL physical layer. The CXL interface unit is used to handle CXL.mem and CXL.cache transactions. CXL.mem is used to execute the corresponding memory access transactions, and CXL.cache is used to handle the corresponding cache transactions. The CXL interface unit serves as the communication entry point between the host and the internal control structure of the logic substrate chip, parsing and responding to CXL memory access requests sent by the host.

[0047] In one embodiment of this application, after the host sends a CXL.mem read request, the CXL interface unit receives and parses the CXL.mem read request, extracts the CXL address and transaction ID from the request message, and the transaction ID is used to identify the corresponding CXL transaction; the parsed request can be temporarily stored in the input buffer to support out-of-order processing of requests.

[0048] After receiving the CXL address, the front-end controller performs byte alignment and cache line mapping on the CXL address to obtain the cache line mapping result, and accesses the cache tag RAM according to the cache line mapping result. The cache tag RAM is used to store cache tag data. The front-end controller determines whether the data to be accessed pointed to by the CXL address hits the on-chip cache array by reading the cache tag information in the cache tag RAM. The cache line is the data unit used by the on-chip cache array for data caching and tag management.

[0049] Furthermore, by byte-aligning and cache line mapping the CXL addresses, the byte-addressable access of CXL is made compatible with the data organization of the on-chip cache. The controller hardware can also directly parse the CXL.cache and CXL.mem instructions and utilize the on-chip eDRAM and multiple vertically stacked NAND Flash chips to form a heterogeneous cache and storage hierarchy;

[0050] In one specific implementation, a three-level physical-logical mapping system can be formed, consisting of an eDRAM hot layer, a near-end NAND warm layer, and a far-end NAND cold layer. The cache line granularity is processed to take into account the byte-addressable characteristics of CXL, achieving unified addressing and management between the cache and the NAND storage medium. This embodiment follows the aforementioned hierarchical setup of the near-end NAND layer and the far-end NAND layer, without further limiting the specific division parameters of the two.

[0051] When the front-end controller determines that the data to be accessed has hit the on-chip cache array based on the cache tag information, it directly reads the data from the on-chip cache array without further accessing the NAND Flash chip. In the specific data return process, the target data read from the eDRAM cache array can enter the read response reordering buffer R³B. The read response reordering buffer is used to reorder the read responses obtained from the out-of-order processing according to the CXL transaction ID, and return the corresponding read responses to the host via the CXL interface unit. Thus, in the case of a cache hit, a data access path is formed from the CXL interface unit, the front-end controller, to the on-chip cache array.

[0052] For a CXL memory access request that misses the cache, the front-end controller creates a trace entry corresponding to the current CXL memory access request in the Missing Status Holding Register (MSHR), which is used to record cache miss requests. The MSHR stores the relevant status and request information of incomplete cache miss requests so that processing of the corresponding request can be resumed after the back-end NAND Flash data is read. Simultaneously, the front-end controller triggers a hardware-based hot / cold evaluation process to determine the data hotness level of the data to be accessed pointed to by the CXL address based on the historical access frequency of the CXL address.

[0053] In one specific implementation, the cache line corresponding to the CXL address after cache line mapping is used as the access frequency statistics object. The number of CXL memory accesses to the cache line is counted within a preset statistical period, and the historical access frequency is determined based on the number of CXL memory accesses. The historical access frequency is compared with a preset first frequency threshold and a preset second frequency threshold, wherein the first frequency threshold is greater than the second frequency threshold. When the historical access frequency is not less than the first frequency threshold, the data to be accessed is determined as hot data; when the historical access frequency is less than the first frequency threshold but not less than the second frequency threshold, the data to be accessed is determined as warm data; and when the historical access frequency is less than the second frequency threshold, the data to be accessed is determined as cold data. Hot data, warm data, and cold data correspond to different data heat levels, which are used for subsequent CXL memory access request scheduling, cache residency, and data tiered management.

[0054] Furthermore, within the preset statistical period, each time a CXL memory access request for the statistical object is received, the access count corresponding to the statistical object is updated; after the preset statistical period ends, the corresponding historical access frequency is determined based on the updated access count, and the data popularity level of the corresponding data is re-determined based on the historical access frequency, so that the data popularity level can be updated with CXL memory access. The same data popularity level determination method is used for different statistical periods. The preset statistical period, the first frequency threshold, and the second frequency threshold can be preset according to the operating configuration of the hybrid memory system.

[0055] In one embodiment of this application, for a CXL memory access request that has a cache miss, the target NAND Flash chip storing the data to be accessed is determined based on the address mapping information corresponding to the CXL address, and the load status of the target NAND Flash chip and its corresponding TSV channel is obtained. The load status may include instantaneous load and queue depth, wherein the instantaneous load may represent the data access load currently undertaken by the corresponding NAND Flash chip or TSV channel, and the queue depth may represent the number of data access requests waiting to be executed through the corresponding NAND Flash chip or TSV channel.

[0056] When multiple concurrent CXL memory access requests exist, the target NAND Flash chip corresponding to each CXL memory access request is determined based on the address mapping information of each CXL memory access request. Then, based on the instantaneous load and queue depth of each target NAND Flash chip and its corresponding TSV channel, combined with the data heat level of each CXL memory access request, the processing order of the multiple CXL memory access requests is scheduled. For example, when multiple requests correspond to the same target NAND Flash chip or compete for the same TSV access resource, the scheduling priority of access requests corresponding to hot data can be increased; when different requests correspond to different target NAND Flash chips, corresponding access requests can be initiated in parallel based on the load status of each target NAND Flash chip and its corresponding TSV channel. Thus, load-aware access scheduling for multiple NAND Flash chips and TSV channels is achieved without changing the current physical storage location of the data to be accessed.

[0057] Furthermore, in addition to performing residency and replacement processing in the on-chip cache array based on data popularity levels, the logic substrate chip also implements hot and cold data migration at the controller hardware level. Specifically, the controller manages the storage location of data according to the determined data popularity levels and monitors the cache hit status of the corresponding CXL addresses in the on-chip cache array to characterize the actual access situation of the corresponding data in the on-chip cache array, and adjusts the storage location of data among on-chip eDRAM and multiple NAND Flash chips accordingly. For frequently accessed CXL memory area data, it is retained in eDRAM or near-end NAND layer; for cold data, it is moved to the far-end NAND layer.

[0058] It should be noted that the cold and hot data migration and the immediate access scheduling corresponding to the current CXL memory access request are different processes. For the current CXL memory access request, the target NAND Flash chip for storing the data to be accessed is determined according to the current valid address mapping information corresponding to the request, and the access request corresponding to the target NAND Flash chip is scheduled; the cold and hot data migration is used to adjust the subsequent physical storage location of the data among multiple NAND Flash chips according to the data popularity level.

[0059] When the data to be accessed is migrated from one NAND Flash chip to another, the logic base chip updates the address mapping information corresponding to the data to be accessed so that subsequent CXL memory access requests can determine the storage location of the data after migration based on the updated address mapping information.

[0060] The aforementioned hot and cold data migration strategy is set at the controller hardware layer, independent of the host software for data migration control, and works in conjunction with CXL cache management and TSV channel scheduling. Through this data migration method, the storage location of data among multiple NAND Flash chips can be adjusted according to data frequency. Simultaneously, the on-chip eDRAM serves as a high-speed cache layer for hot and metadata data, and multiple vertically stacked NAND Flash chips act as a capacity storage layer.

[0061] In one embodiment of this application, the front-end controller uses a ten-stage fully pipelined architecture to process CXL memory access requests.

[0062] In Level 0, the HDM address index field of the input request is compared in parallel with the index fields of other requests in transit in the pipeline to detect the risk of tag conflicts. The HDM address index field is the address field used for index comparison in this pipeline. When an index match is detected, the current request is marked as a dangerous request and the corresponding dangerous request identifier is recorded. At the same time, a tag read request is sent to the CacheTag RAM.

[0063] In Level 1, for a write request, the write request data is written to the write data buffer WB, and the tag forwarding control flow is started, where WB is used to temporarily store the write request data.

[0064] In stages 2 through 4, the tag data returned by the Cache Tag RAM is cached through the cascaded register set to cover the multi-cycle read latency of the Cache Tag RAM and absorb the on-chip signal transmission latency.

[0065] In Level 5, a cache hit determination is performed based on the obtained tag data, and an MSHR hit check is performed simultaneously.

[0066] In level 6, for requests that hit the cache, the request is forwarded to the branch pipeline, and the dirty bit and pseudo-LRU replacement information of the corresponding cache tag are updated.

[0067] In levels 7 and 8, for requests where MSHR is not hit, a new MSHR entry is created, a replacement cache line is selected, and the validity and transmission bits of the corresponding cache tag are updated; for requests where MSHR is hit, the information of the existing MSHR entry is updated.

[0068] In level 9, the updated cache tag data is written back to the Cache Tag RAM, and a backend storage access request corresponding to the current cache miss request is generated.

[0069] In one specific implementation, backend storage access adopts the NVMe command mechanism. The frontend controller generates a corresponding NVMe submission queue entry based on the backend storage access request and sends it to the NVMe controller. The NVMe submission queue, or SQ (Submission Queue), is used to carry storage access commands submitted to the NVMe controller.

[0070] The aforementioned ten-level pipeline is used to sequentially complete address correlation checks, cache tag reading, cache hit determination, MSHR management, cache tag updating, and backend storage access request generation.

[0071] In one embodiment of this application, as Figure 2 As shown, the backend controller uses the BEC-CQ state machine, BEC-Tag state machine, and BEC-MSHR state machine to collaboratively process backend access completion information.

[0072] In one specific implementation, the backend access adopts the NVMe command and completion queue mechanism. After the corresponding backend storage access is completed, the NVMe controller returns a completion queue entry, where CQ stands for Completion Queue, which represents the completion queue used to carry storage access completion information.

[0073] The BEC-CQ state machine receives completion queue entries returned by the NVMe controller and determines the access type corresponding to the completion queue entry. When the completion queue entry corresponds to a write operation, the BEC-CQ state machine generates a corresponding NVMe read command and sends it to the NVMe controller to obtain new cache line data. When the completion queue entry corresponds to a read operation, cache tag update processing and missing state processing are triggered. The BEC-Tag state machine accesses and updates the cache tag RAM, specifically reading the original tag information of the target cache set, updating the tag address field and valid bit of the corresponding cache line according to the read data, and writing the updated tag information back to the cache tag RAM. The BEC-MSHR state machine, after the target NAND Flash chip completes data reading, retrieves request information related to the logical block address in the MSHR based on the logical block address that completed the data reading, forwards the retrieved request to the branch pipeline of the front-end controller to continue executing subsequent cache accesses, and releases the corresponding MSHR entry, thereby completing the recovery processing of cache miss requests and avoiding repeated scheduling.

[0074] Furthermore, in the complete CXL memory read request processing process, the CXL.mem read request sent by the host is first parsed by the CXL interface unit to obtain the CXL address and transaction ID. The front-end controller performs a cache hit judgment based on the CXL address. If the cache hits, the data is directly read from the on-chip eDRAM cache array and a read response is formed. If the cache misses, a corresponding tracking entry is created in MSHR, and the data hotness level of the data to be accessed is determined based on the historical access frequency of the CXL address. The target NAND Flash chip storing the data to be accessed is determined based on the address mapping information corresponding to the CXL address. The channel scheduling unit further obtains the instantaneous load and queue depth of the target NAND Flash chip and the corresponding TSV channel, and schedules the current CXL memory access request in combination with the data hotness level of the data to be accessed. The back-end controller controls the target NAND Flash chip to complete the data reading through the TSV channel corresponding to the target NAND Flash chip.

[0075] After data reading is complete, the back-end controller determines the data's residence location and replacement strategy in the eDRAM cache array based on the actual data heat level, and updates the cache tag. The returned data can be reassembled into the data block format specified by CXL, written to the eDRAM cache, processed sequentially by transaction ID through the read response reordering buffer, and then returned. For write requests, the data processing flow corresponds to the read request processing flow described above. The controller can determine the corresponding target NAND Flash chip based on the address mapping information corresponding to the write request, and write the write data to the target NAND Flash chip through the TSV channel corresponding to the target NAND Flash chip. For multiple concurrent write requests, the parallel access capability of multiple NAND Flash chips and their corresponding TSV channels can be used to perform the write operation. Thus, the on-chip eDRAM cache, multi-layer NAND Flash stacking structure, TSV channels corresponding to each NAND Flash chip, data heat management, and TSV load-aware scheduling work together in the logic base chip to realize data processing between CXL front-end access and back-end NAND storage.

[0076] Figure 3This document illustrates a flowchart of a 3D-stacked high-bandwidth hybrid memory control method oriented towards CXL memory semantics, according to an embodiment of the present invention. The 3D-stacked high-bandwidth hybrid memory control method oriented towards CXL memory semantics in this embodiment is applied to a hybrid memory system. The hybrid memory system includes a logic substrate chip and a NAND Flash stacked structure. The NAND Flash stacked structure includes multiple NAND Flash chips stacked vertically above the logic substrate chip, and each NAND Flash chip is connected to the logic substrate chip through a corresponding TSV channel. The main steps include the following:

[0077] Step S31: Receive and parse the CXL memory access request sent by the host, and obtain the CXL address corresponding to the CXL memory access request;

[0078] Step S32: Perform a cache hit determination on the on-chip cache array set on the logic substrate chip according to the CXL address; when the cache hits, access the on-chip cache array; when the cache misses, determine the data hotness level of the data to be accessed pointed to by the CXL address according to the historical access frequency of the CXL address.

[0079] Step S33: Determine the target NAND Flash chip storing the data to be accessed based on the address mapping information corresponding to the CXL address, obtain the load status of the target NAND Flash chip and the corresponding TSV channel, and schedule the CXL memory access request based on the load status and the data heat level of the data to be accessed.

[0080] Step S34: Schedule the CXL memory access request to the target NAND Flash chip, and perform data access on the target NAND Flash chip corresponding to the CXL memory access request through the TSV channel corresponding to the target NAND Flash chip.

[0081] It should be understood that the specific process of performing the corresponding methods in each step has been described in detail in the above system embodiments, and will not be repeated here for the sake of brevity.

[0082] It should also be understood that the module division in the embodiments of this application is illustrative and only represents a logical functional division; in actual implementation, there may be other division methods. Furthermore, the functional modules in the various embodiments of this application can be integrated into a single processor, exist as separate physical entities, or be integrated into a single module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0083] Figure 4 This is a schematic block diagram of the electronic terminal provided in an embodiment of this application. Figure 4 As shown, the electronic terminal 400 includes at least one processor 401, a memory 402, at least one network interface 403, and a user interface 405. The various components in the electronic terminal 400 are coupled together via a bus system 404. It is understood that the bus system 404 is used to implement communication between these components. In addition to a data bus, the bus system 404 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in… Figure 4 The general will label all buses as bus systems.

[0084] The user interface 405 may include a monitor, keyboard, mouse, trackball, clicker, button, touchpad, or touch screen.

[0085] It is understood that memory 402 can be volatile memory or non-volatile memory, or both. Non-volatile memory can be read-only memory (ROM) or programmable read-only memory (PROM), used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM) and synchronous static random access memory (SSRAM). The memories described in the embodiments of this invention are intended to include, but are not limited to, these and any other suitable categories of memory.

[0086] In this embodiment of the invention, the memory 402 is used to store various types of data to support the operation of the electronic terminal 400. Examples of this data include: any executable program for operation on the electronic terminal 400, such as the operating system 4021 and application programs 4022; the operating system 4021 contains various system programs, such as the framework layer, core library layer, driver layer, etc., for implementing various basic services and handling hardware-based tasks. The application program 4022 may contain various applications, such as a media player, browser, etc., for implementing various application services. The 3D stacked high-bandwidth hybrid memory control method for CXL memory semantics provided in this embodiment of the invention can be included in the application program 4022.

[0087] The methods disclosed in the above embodiments of the present invention can be applied to processor 401, or implemented by processor 401. Processor 401 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in processor 401 or by instructions in the form of software. The processor 401 may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Processor 401 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. General-purpose processor 401 may be a microprocessor or any conventional processor, etc. The steps of the accessory optimization method provided in the embodiments of the present invention can be directly reflected as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software module may be located in a storage medium, which is located in a memory. The processor reads the information in the memory and combines it with its hardware to complete the steps of the aforementioned method.

[0088] In an exemplary embodiment, the electronic terminal 400 may be used by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), or complex programmable logic devices (CPLDs) to execute the aforementioned method.

[0089] According to the method provided in the embodiments of this application, this application also provides a computer program product, which includes: computer program code, which, when run on a computer, causes the computer to perform the method of any of the embodiments described above.

[0090] According to the method provided in the embodiments of this application, this application also provides a computer-readable storage medium storing program code, which, when run on a computer, causes the computer to perform the method of any of the embodiments described above.

[0091] As used in this specification, the terms "component," "module," "system," etc., are used to refer to computer-related entities, hardware, firmware, combinations of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program, and / or a computer. As illustrated, applications running on computing devices and computing devices can both be components. One or more components may reside in a process and / or an execution thread, and components may be located on a single computer and / or distributed among two or more computers. Furthermore, these components can be executed from various computer-readable media on which various data structures are stored. Components can communicate, for example, via local and / or remote processes based on signals having one or more data packets (e.g., data from two components interacting with another component between a local system, a distributed system, and / or a network, such as the Internet interacting with other systems via signals).

[0092] Those skilled in the art will recognize that the various illustrative logical blocks and steps described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

[0093] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0094] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0095] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0096] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0097] In the above embodiments, the functions of each functional unit can be implemented entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. A computer program product includes one or more computer instructions (programs). When the computer program instructions (programs) are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. Available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., high-density digital video discs (DVDs)), or semiconductor media (e.g., solid-state disks (SSDs)).

[0098] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0099] In summary, this application provides a 3D stacked high-bandwidth hybrid memory control system, method, medium, terminal, and program product oriented towards CXL memory semantics. The system includes a logic substrate chip and a NAND Flash stacked structure. Multiple NAND Flash chips are stacked vertically and connected to the logic substrate chip via corresponding TSV channels. The logic substrate chip performs on-chip cache hit determination based on CXL addresses. When a cache miss occurs, it determines the data hotness level of the data to be accessed based on historical access frequency. It then determines the target NAND Flash chip for storing the data to be accessed based on the address mapping information corresponding to the CXL address. Finally, it schedules CXL memory access requests by combining the load status of the target NAND Flash chip and its corresponding TSV channel with the data hotness level to execute data access. This application can improve the utilization of backend storage resources and TSV channels, improve cache utilization efficiency, and alleviate the problem of mismatch between CXL front-end bandwidth and NAND back-end bandwidth. Therefore, this application effectively overcomes the various shortcomings of the prior art and has high industrial application value.

[0100] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this application should still be covered by the claims of this application.

Claims

1. A 3D stacked high-bandwidth hybrid memory control system oriented towards CXL memory semantics, characterized in that, include: The logic substrate chip includes a CXL interface unit, a front-end controller, a back-end controller, an on-chip cache array, and a channel scheduling unit; The NAND Flash stacking structure includes multiple NAND Flash chips stacked vertically above the logic substrate chip, and each NAND Flash chip is connected to the logic substrate chip through a corresponding TSV channel. The CXL interface unit is used to receive and parse the CXL memory access request sent by the host, and obtain the CXL address corresponding to the CXL memory access request; The front-end controller is used to determine the cache hit of the on-chip cache array based on the CXL address, access the on-chip cache array when the cache hit occurs, and determine the data hotness level of the data to be accessed pointed to by the CXL address based on the historical access frequency of the CXL address when the cache miss occurs. The channel scheduling unit is used to determine the target NAND Flash chip storing the data to be accessed based on the address mapping information corresponding to the CXL address, obtain the load status of the target NAND Flash chip and the corresponding TSV channel, and schedule the CXL memory access request according to the load status and the data heat level of the data to be accessed, and schedule the CXL memory access request to the target NAND Flash chip. The back-end controller is used to control the target NAND Flash chip to perform data access corresponding to the CXL memory access request through the TSV channel corresponding to the target NAND Flash chip.

2. The 3D stacked high-bandwidth hybrid memory control system for CXL memory semantics according to claim 1, characterized in that, The step of determining a cache hit in the on-chip cache array based on the CXL address, and accessing the on-chip cache array when a cache hit occurs, specifically includes: Perform byte alignment and cache line mapping on the CXL address to obtain the cache line mapping result. Access the cache tag RAM according to the cache line mapping result to obtain the corresponding cache tag information. Based on the cache tag information, determine whether the data to be accessed pointed to by the CXL address hits the on-chip cache array; When a cache hit occurs, the data to be accessed is read from the on-chip cache array, and the read response corresponding to the data to be accessed is returned to the host through the CXL interface unit.

3. The 3D stacked high-bandwidth hybrid memory control system for CXL memory semantics according to claim 1, characterized in that, The load status includes instantaneous load and queue depth; the channel scheduling unit is also used to determine the corresponding target NAND Flash chip for multiple concurrent CXL memory access requests according to the address mapping information corresponding to each CXL memory access request, and to schedule each CXL memory access request according to the instantaneous load and queue depth of each target NAND Flash chip and the corresponding TSV channel, combined with the data heat level of the data to be accessed corresponding to each CXL memory access request.

4. The 3D stacked high-bandwidth hybrid memory control system for CXL memory semantics according to claim 1, characterized in that, The front-end controller is further configured to select a replacement cache line when a cache miss occurs; the back-end controller is further configured to, after the target NAND Flash chip completes the reading of the data to be accessed, determine the cache residency policy and cache replacement policy of the data to be accessed in the on-chip cache array based on the data popularity level of the data to be accessed, and update the corresponding cache line according to the cache residency policy and cache replacement policy; wherein, for the data to be accessed with a data popularity level of "hot", the data to be accessed is retained in the on-chip cache array; for the data to be accessed with a data popularity level of "warm", the replacement cache line is cache-replaced according to the cache replacement policy; and for the data to be accessed with a data popularity level of "cold", the residency priority of the data to be accessed in the on-chip cache array is reduced.

5. The 3D stacked high-bandwidth hybrid memory control system for CXL memory semantics according to claim 1, characterized in that, The logic substrate chip is also used to control the migration of the data to be accessed among multiple NAND Flash chips according to the data popularity level of the data to be accessed, so as to adjust the storage location of the data to be accessed in the NAND Flash stack structure.

6. The 3D stacked high-bandwidth hybrid memory control system for CXL memory semantics according to claim 1, characterized in that, In the event of a cache miss, the front-end controller is also configured to create a trace entry corresponding to the CXL memory access request in a missing status holding register used to record cache miss requests; The back-end controller is further configured to, after the target NAND Flash chip completes data reading, retrieve a trace entry related to the logical block address from the missing state holding register according to the logical block address where the data reading was completed, forward the CXL memory access request corresponding to the retrieved trace entry to the front-end controller to continue executing subsequent cache access, and release the trace entry.

7. A 3D stacked high-bandwidth hybrid memory control method oriented towards CXL memory semantics, characterized in that, The method is applied to a hybrid memory system, the hybrid memory system including a logic substrate chip and a NAND Flash stacked structure, the NAND Flash stacked structure including multiple NAND Flash chips stacked vertically above the logic substrate chip, each of the NAND Flash chips being connected to the logic substrate chip through a corresponding TSV channel; the method includes: Receive and parse the CXL memory access request sent by the host to obtain the CXL address corresponding to the CXL memory access request; The on-chip cache array configured on the logic substrate chip is used to determine cache hits based on the CXL address. When a cache hit occurs, the on-chip cache array is accessed; When a cache miss occurs, the data hotness level of the data to be accessed pointed to by the CXL address is determined based on the historical access frequency of the CXL address. The target NAND Flash chip storing the data to be accessed is determined based on the address mapping information corresponding to the CXL address. The load status of the target NAND Flash chip and the corresponding TSV channel is obtained. The CXL memory access request is scheduled according to the load status and the data heat level of the data to be accessed, and the CXL memory access request is scheduled to the target NAND Flash chip. The CXL memory access request is scheduled to the target NAND Flash chip, and data access corresponding to the CXL memory access request is performed on the target NAND Flash chip through the TSV channel corresponding to the target NAND Flash chip.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in claim 7.

9. A computer program product, characterized in that, The computer program product includes computer program code that, when run on a computer, causes the computer to implement the method as described in claim 7.

10. An electronic terminal, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the method as described in claim 7.

Images