An adaptive tensor cache scheduling system and method based on distributed near-memory computing

Abstract

The application relates to the technical field of semiconductor storage and near-memory computing fusion, in particular to an adaptive tensor cache scheduling system and method based on distributed near-memory computing, which comprises a three-dimensional stacked storage array, a distributed near-memory computing unit cluster, a hierarchical adaptive tensor cache module and a global cooperative scheduling controller, adopts a mesh point-to-point interconnection of a network on a chip to eliminate a centralized bus bottleneck, a three-level cache structure comprises a micro cache, a distributed cache and a convergence cache, an asymmetric bidirectional data channel is adopted to adapt to tensor calculation characteristics, and the global cooperative scheduling controller realizes adaptive cache scheduling of tensor data through tensor characteristic analysis and classification, cache state monitoring and weighted multi-factor scheduling decision-making. The application solves the problems of a centralized cache bandwidth bottleneck, poor adaptability of cache scheduling to tensor calculation characteristics, low interlayer communication efficiency and multi-unit concurrent conflict, significantly reduces inference delay and data transfer power consumption, and improves system concurrent throughput.

Description

Technical Field

[0001] This invention relates to the field of semiconductor storage and near-memory computing integration technology, and in particular to an adaptive tensor cache scheduling system and method based on distributed near-memory computing. Background Technology

[0002] With the rapid development of artificial intelligence technology, edge AI inference applications are placing increasingly higher demands on the bandwidth, latency, and concurrent processing capabilities of storage systems. In traditional AI inference hardware architectures, storage systems typically employ a centralized bus structure to connect storage arrays, cache modules, and computing units. All data access requests must be scheduled and forwarded through a centralized arbitration node. This centralized architecture suffers from severe bandwidth bottlenecks and access conflicts when facing high-concurrency tensor data access, making it difficult to meet the low-latency data access requirements of multi-path parallel AI inference tasks.

[0003] Existing cache scheduling mechanisms typically employ fixed replacement strategies (such as LRU and LFU), failing to adaptively adjust based on the access characteristics and computational dependencies of tensor data. Tensor data in AI inference exhibits a clear hierarchical structure: weight tensors are accessed frequently and have strong computational dependencies, intermediate results from feature maps are accessed at a moderate frequency, and temporary residual data is accessed infrequently and used only once. Fixed replacement strategies cannot effectively distinguish between different types of tensor data, leading to frequent-access weight data potentially being replaced by infrequent temporary data, resulting in decreased cache hit rates and increased power consumption during data movement.

[0004] Furthermore, in a three-dimensional stacked memory architecture, inter-layer communication employs a traditional single-multiplexing method, where control signals and data signals share the same transmission channel, leading to signal crosstalk and low transmission efficiency. Impedance mismatch and signal attenuation issues in inter-layer through-silicon vias (TSVs) also limit data transmission rate and reliability. Simultaneously, existing near-memory computing schemes lack effective collaborative scheduling mechanisms between computing units and cache modules, making it prone to address conflicts, bandwidth conflicts, and resource conflicts when multiple computing units access the system concurrently, thus impacting overall system throughput.

[0005] Therefore, it is necessary to provide an adaptive tensor cache scheduling system and method based on distributed near-memory computing to solve problems such as bandwidth bottlenecks in centralized cache architectures, poor adaptability between cache scheduling and tensor computing features, low communication efficiency between three-dimensional stacked storage layers, and concurrent conflicts among multiple computing units. Summary of the Invention

[0006] In view of the aforementioned existing problems, the present invention is proposed.

[0007] The purpose of this invention is to provide an adaptive tensor cache scheduling system and method based on distributed near-memory computing. By combining a distributed near-memory computing architecture with a hierarchical adaptive tensor cache scheduling mechanism, the system solves the bandwidth bottleneck problem of centralized cache architecture, improves the adaptability of cache scheduling to tensor computing characteristics, optimizes the communication efficiency between three-dimensional stacked storage layers, and resolves concurrent access conflicts of multiple computing units.

[0008] To solve the above-mentioned technical problems, the present invention provides the following technical solution: In a first aspect, the present invention provides an adaptive tensor cache scheduling system based on distributed near-memory computing, comprising: A three-dimensional stacked memory array includes multi-layer memory wafers and a top-level logic wafer. The multi-layer memory wafers and the top-level logic wafer are interconnected by vertical bonding through silicon vias. Each memory wafer integrates a phase-change memory cell array and is divided into multiple independent memory blocks. The top-level logic wafer integrates a global collaborative scheduling controller and a convergence cache. The distributed near-memory computing unit cluster includes multiple computing sub-clusters. Each computing sub-cluster is physically mapped to a storage block in the three-dimensional stacked storage array. Each computing sub-cluster contains multiple computing units, which are used to perform tensor operations. The hierarchical adaptive tensor cache module includes a three-level cache structure of micro-cache, distributed cache and convergence cache. The micro-cache is coupled inside the computing unit, the distributed cache is bound to the physical layer of the storage wafer, and the convergence cache, as the third-level cache of the hierarchical adaptive tensor cache module, is deployed on the top-level logic wafer. The three-level caches are interconnected by an asymmetric bidirectional data channel. The global collaborative scheduling controller is used to perform feature parsing and classification of tensor data, monitor the status of the three-level cache, and perform cache scheduling of tensor data according to weighted multi-factor scheduling decisions, so as to realize the collaborative work between storage, cache and computation. The three-dimensional stacked storage array, the distributed near-memory computing unit cluster, the hierarchical adaptive tensor cache module, and the global collaborative scheduling controller are interconnected point-to-point through an on-chip network Mesh.

[0009] As a preferred embodiment of the adaptive tensor cache scheduling system based on distributed near-memory computing described in this invention, the micro-cache adopts a static random access memory, which is deeply coupled to each computing unit and supports single-cycle read / write and in-situ logical operations. The distributed cache adopts a hybrid architecture of non-volatile storage and volatile storage. The non-volatile storage is used to store long-term valid tensor weight data, and the volatile storage is used to store temporary intermediate calculation results. The converged cache adopts a cache group structure with multiple groups of dynamically switchable mapping modes, and the cache groups are fully interconnected through a cross-switch matrix.

[0010] As a preferred embodiment of the adaptive tensor cache scheduling system based on distributed near-memory computing described in this invention, the global collaborative scheduling controller includes: a tensor feature parsing unit, a cache status monitoring unit, a scheduling decision unit, an access conflict arbitration unit, and an instruction distribution unit. The tensor feature parsing unit is used to extract the tensor's dimension parameters, access step size, consecutive access length, computational dependencies, and access time interval in real time. It uses an exponentially weighted moving average algorithm to calculate the access popularity value and classifies the tensor into three categories: core tensor, auxiliary tensor, and temporary tensor based on the access popularity value. The cache status monitoring unit is used to perform full-dimensional status sampling on the three-level cache at a preset sampling period, and output the monitoring data after digital filtering. The scheduling decision unit is used to prioritize tensor data and output scheduling instructions based on a weighted multi-factor decision model that considers access popularity, computational dependency, spatial locality, access latency, and power consumption cost.

[0011] As a preferred embodiment of the adaptive tensor cache scheduling system based on distributed near-memory computing described in this invention, the inter-layer communication of the three-dimensional stacked storage array adopts a hybrid transmission mechanism of frequency division multiplexing and time division multiplexing. The control signal is transmitted through multiple subcarriers of frequency division multiplexing, and the data signal is transmitted through time slot scheduling of time division multiplexing. The through-silicon via interface has built-in adjustable matching resistors and signal compensation circuits to dynamically adjust the driving voltage and signal slope according to the transmission distance. Each layer of the communication controller has a built-in data buffer queue and monitors the occupancy rate. When the occupancy rate reaches the congestion threshold, a congestion flag is sent to the controller to switch the transmission path. A leaky bucket flow control algorithm is used to smooth the flow. Inter-layer data transmission is accompanied by a CRC checksum, which supports automatic error correction for single-bit errors and detection of multi-bit errors. Automatic retransmission is triggered when errors cannot be corrected.

[0012] As a preferred embodiment of the adaptive tensor cache scheduling system based on distributed near-memory computing described in this invention, the system further includes a low-power control module. The low-power control module is used to execute a hierarchical power control strategy; when the micro-cache idle time is greater than or equal to a first threshold, it enters a shallow sleep mode; when the distributed cache idle time is greater than or equal to a second threshold, the non-volatile storage partition enters a deep sleep mode; the converged cache performs power control at the group level, and the power is turned off for idle cache groups; the computing unit shuts down the arithmetic array when it is idle; the low-power control module is also used to perform dynamic voltage and frequency adjustment based on system load, cache occupancy rate, and computing throughput.

[0013] As a preferred embodiment of the adaptive tensor cache scheduling system based on distributed near-memory computing described in this invention, wherein: in the distributed near-memory computing unit cluster, each computing sub-cluster includes a basic computing unit and an enhanced computing unit, the basic computing unit integrates a fixed-point multiplication array, an adder, and activation function hard logic, and the enhanced computing unit additionally integrates a high-bit-width multiplier, an accumulator, and batch normalization hard logic; the computing units are interconnected by a ring topology, supporting ring forwarding and sharing of intermediate values.

[0014] As a preferred embodiment of the adaptive tensor cache scheduling system based on distributed near-memory computing described in this invention, the system further includes a data reliability protection module and a heterogeneous interface adaptation unit. The data reliability protection module is used to perform different levels of data verification on the three-level cache, perform bad block management and wear leveling on the storage array, and perform overheat protection and power failure protection. The overheat protection includes temperature monitoring and automatic frequency and voltage reduction when the temperature exceeds the limit, and the power failure protection includes backup power supply startup and writing valid cache data to the non-volatile storage area. The heterogeneous interface adapter unit supports multiple external protocol interfaces and has a built-in protocol conversion module and bit width adaptation module to convert external protocols into internal storage protocols and support dynamic adaptation of multiple bit widths.

[0015] Secondly, the present invention provides an adaptive tensor cache scheduling method based on distributed near-memory computing, comprising: The input tensor data is subjected to feature parsing to extract the tensor's dimension parameters, access step size, consecutive access length, computational dependencies and access time intervals, and access popularity value is calculated. Based on the access popularity value, the tensor is divided into core tensors, auxiliary tensors and temporary tensors. The system performs full-dimensional state sampling on the three-level cache (micro-cache, distributed cache, and convergence cache) at a preset sampling period, and obtains the occupancy rate, hit rate, and dirty data volume of each level of cache after digital filtering. Based on a weighted multi-factor decision model that considers access popularity, computational dependency, spatial locality, access latency, and power consumption, tensor data is prioritized and scheduling instructions are generated. The system performs cache data migration and prefetching according to scheduling instructions, enabling collaborative addressing and data interaction between the storage array, L3 cache, and computing units.

[0016] Furthermore, the step of performing cached data migration and prefetching according to scheduling instructions includes: When the micro-cache occupancy rate is greater than or equal to the first occupancy threshold, the ping-pong buffer non-blocking migration mechanism is activated to migrate temporary tensors and low-priority auxiliary tensors from the micro-cache to the distributed cache. When the distributed cache hit rate is less than or equal to the first hit threshold, tensor space locality prefetching is initiated. Based on the historical access addresses, the tensor address range of the next consecutive access is predicted, and data is prefetched from the aggregate cache or storage array to the distributed cache.

[0017] Furthermore, the step of performing cache data migration and prefetching according to scheduling instructions to achieve coordinated addressing and data interaction between the storage array, L3 cache, and computing units includes: The computing unit sends a read request to the micro-cache; if the micro-cache is hit, the data is output directly. If the micro-cache misses, a read request is sent to the distributed cache; If the distributed cache misses, a read request is sent to the converged cache; If the convergence cache misses, a read command is sent to the storage array; After the storage array reads the data, it sends it to the convergence cache and simultaneously initiates prefetching; After the computing unit completes the operation, it writes the result back to the micro-cache. Dirty data is automatically marked and written back to the distributed cache or the aggregation cache at regular intervals.

[0018] Compared with existing technologies, the beneficial effects of this invention are as follows: By adopting a distributed near-memory computing architecture, this invention physically maps computing units to storage arrays, eliminating the bandwidth bottleneck of centralized bus structures; through a hierarchical adaptive tensor cache three-level structure design, tensor data with different access characteristics reside in appropriate cache levels, improving cache hit rate and reducing data transport power consumption; through a weighted multi-factor scheduling decision algorithm, it comprehensively considers multiple factors such as access popularity, computational dependency, spatial locality, access latency, and power consumption, achieving fine-grained scheduling of tensor data; through a hybrid frequency division multiplexing and time division multiplexing inter-layer communication mechanism, it optimizes the inter-layer data transmission efficiency of three-dimensional stacked storage; and through a computation cache collaborative scheduling mechanism, it resolves concurrent access conflicts of multiple computing units, improving the overall system throughput. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a schematic diagram of the overall structure of an adaptive tensor cache scheduling system based on distributed near-memory computing, according to an embodiment of the present invention. Detailed Implementation

[0021] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0022] Example 1, referring to Figure 1 As one embodiment of the present invention, an adaptive tensor cache scheduling system based on distributed near-memory computing is provided. This system includes a three-dimensional stacked storage array, a distributed near-memory computing unit cluster, a hierarchical adaptive tensor cache module, and a global cooperative scheduling controller. All hardware modules adopt a network-on-a-chip (NoC) to achieve mesh point-to-point interconnection, completely eliminating the traditional centralized bus and single-node arbitration structure, thus eliminating bandwidth bottlenecks and access conflicts at the physical level. This includes: The three-dimensional stacked memory array employs a vertical bonding structure between multilayer memory wafers and a top-level logic wafer. In this embodiment, a structure of 5 memory wafers plus 1 top-level logic wafer is used, with layers interconnected via composite dielectric through-silicon vias (TSVs). The TSV diameter is 2.2μm~3.1μm, the spacing is 8.5μm~11.2μm, the vertical spacing between layers is 15.8μm~20.3μm, and the bonding yield is ≥99.992%. Each memory wafer integrates a phase-change memory cell array, with the memory cell adopting a 1S1R (1 selector transistor, 1 phase-change resistive switch) structure. The phase-change material is a Ge2Sb2Te5 and TiN composite doping system. The crystallization temperature is 165℃~182℃, the set voltage is 1.35V~1.52V, the reset voltage is 2.15V~2.38V, and the data retention time is ≥10 years. Each layer of the memory array is divided into multiple (e.g., 16) equal memory blocks based on physical addresses. Each memory block has built-in pre-decoding circuitry, I / O driver circuitry, resistive sensing circuitry, and bad block marker register, supporting block-level parallel addressing, parallel read / write, and parallel erase operations. The top-level logic wafer integrates a global collaborative scheduling controller, convergence cache, heterogeneous interface adapter unit, and TSV routing matrix. All logic circuits are manufactured using FD-SOI technology, with an operating voltage of 0.9V~1.1V and leakage current controlled between 12.5nA / μm and 18.3nA / μm.

[0023] The three-dimensional stacked storage inter-layer communication optimization mechanism in this application includes: a hybrid multiplexing transmission mechanism, TSV interface impedance matching and signal compensation, inter-layer congestion control and flow management, and inter-layer data verification and error correction. The specific implementation is as follows: The hybrid multiplexing transmission mechanism employs a combination of frequency division multiplexing (FDM) and time division multiplexing (TDM): control signals (address, command, status) are multiplexed using FDM and transmitted through 8 subcarriers spaced 25MHz apart, with no interference between subcarriers; high-frequency data signals are multiplexed using TDM, with a time slot length of 7.8ns ± 0.7ns, and each time slot is allocated one buffer / storage node, with time slot scheduling using a dynamic priority algorithm; inter-layer synchronous clocks are used, with a clock deviation ≤ ± 0.3ns, ensuring synchronization between time slots and carriers, with no crosstalk or misalignment.

[0024] TSV interface impedance matching and signal compensation are as follows: The TSV interface has a built-in adjustable matching resistor, with a resistance value continuously adjustable from 45Ω to 65Ω in 1Ω steps, automatically matching the transmission line impedance, and a return loss ≤-28dB; the drive voltage and signal slope are dynamically adjusted according to the transmission distance. Short distance (same layer / adjacent layer): driving voltage 0.95V±0.05V, slope 1.2V / ns; Long distance (spanning 3 or more layers): Drive voltage 1.25V±0.05V, slope 1.8V / ns; Equalization circuit: Continuous-time linear equalization (CTLE) circuit is used to compensate for high-frequency attenuation, and the equalization gain is adjustable from 0dB to 12dB.

[0025] Inter-layer congestion control and traffic management: Each layer's global collaborative scheduling controller has a built-in data buffer queue with a queue depth of 24 levels, and monitors the queue occupancy rate in real time; when the queue occupancy rate is ≥71%±3%, it automatically sends a congestion flag to the global collaborative scheduling controller; the global collaborative scheduling controller switches data transmission paths in real time to reduce traffic on congested links, with congestion mitigation latency ≤13ns; a leaky bucket traffic control algorithm is adopted, with a bucket capacity of 64B and dynamically adjustable output rate to avoid traffic bursts.

[0026] Interlayer data verification and error correction: Each frame of data is appended with a 16-bit CRC checksum, with a frame length of 256 bits + 16 bits of checksum; supports automatic error correction for 1-bit errors and detection for 2-bit errors, with an error correction latency of ≤4.3ns; when an uncorrectable error is detected, retransmission is automatically triggered, with a maximum of 3 retransmissions and a retransmission success rate of ≥99.999%.

[0027] The distributed near-memory computing unit cluster adopts a heterogeneous multi-subcluster architecture, divided into multiple (e.g., 8) computing subclusters. Each computing subcluster corresponds to multiple (e.g., 16) memory blocks on the storage wafer, with a physical mapping ratio of 1:4.3 to 1:4.8 between computing subclusters and memory blocks, achieving asymmetric coverage. Each computing subcluster contains 12 basic computing units and 2 enhanced computing units: the basic computing units integrate an 8-bit fixed-point multiplication array, a 32-bit carry-lookahead adder, and ReLU / Sigmoid dual activation function hard logic; the enhanced computing units additionally integrate a 16-bit multiplier, a 64-bit accumulator, and batch normalization hard logic. The data path within the computing units adopts a fully parallel pipelined structure, capable of completing 16 8-bit multiply-accumulate operations per cycle with an accuracy error ≤ ±0.023%. The computing units are interconnected using a 2D ring topology, with a data transmission width of 128 bits within the ring and a transmission latency ≤ 6.2ns. It supports ring forwarding and sharing of intermediate values ​​without the need for cache relay. Each computing unit is equipped with an independent computing status register, data enable register, and interrupt flag register, enabling it to independently receive scheduling instructions, execute computing tasks, and write back results, supporting 128-way concurrent computing without conflict.

[0028] Specifically, the computation unit reads tensor data directly from the micro-cache without bus relay; the multiplication array, adder and activation function perform operations in parallel, and intermediate results are temporarily stored in the internal registers of the computation unit; after the operation is completed, the result is written directly back to the micro-cache, and dirty data is marked at the same time; when the micro-cache is full or the computation task ends, dirty data is written back to the distributed cache or the converged cache according to the scheduling instructions.

[0029] It should be noted that the hierarchical adaptive tensor cache is divided into three levels: micro cache (L0), distributed cache (L1), and convergence cache (L2). The three levels are connected by an asymmetric bidirectional data channel. The physical bit width of the write channel is 256 bits, and the physical bit width of the read channel is 192 bits. The bandwidth of the write channel is 1.33 times that of the read channel, which is adapted to the characteristics of tensor computation, which involves writing many intermediate values ​​and reading few feature values.

[0030] The micro-cache (L0) is deeply coupled within the near-memory computing unit. Each computing unit is equipped with an independent micro-cache with a capacity of 3.2KB ± 0.3KB. It uses 6-transistor SRAM memory cells with a cell area of ​​0.024μm² ± 0.002μm², and operates at a frequency of 1.8GHz to 2.2GHz. It supports single-cycle (0.46ns to 0.53ns) read / write, in-situ logic operations, and data latching. The L0 micro-cache consists of 28 memory subarrays, a word line driver module, a bit line sensing module, a data latch module, and a control logic module. Specifically, each subarray in the storage subarray contains 128 rows × 64 columns of storage cells, adopts a differential bit line structure, with bit line load capacitance of 8.7fF~11.3fF and word line delay ≤0.21ns; the word line drive module adopts a three-stage inverter drive structure, with a drive capability of 4 × standard cells, rise / fall time ≤0.18ns, and supports fast row address selection; the bit line sensing module adopts a latching sensitive amplifier, with reference voltage calibration accuracy of ±4.2mV, sensing delay ≤0.27ns, and can distinguish signal differences as small as 12mV; the data latch module adopts a D flip-flop array, supports data hold, clear, and set operations, with a hold time ≥1.2μs, meeting the short-term data dwell requirements of the computing unit; the control logic module is used to receive scheduling instructions, calculate enable signals and address signals, and generate read / write control timing, with timing deviation ≤±0.09ns.

[0031] The distributed cache (L1) is strongly bound to the physical layer of the three-dimensional stacked storage array. Each storage wafer is configured with four L1 cache blocks, each with a capacity of 96KB±7KB. It adopts a hybrid architecture of non-volatile and volatile storage: non-volatile storage (such as eFlash) accounts for 72%±3% of the capacity and is used to store long-term valid tensor weight data, while volatile storage (such as SRAM) accounts for 28%±3% of the capacity and is used to store temporary intermediate computation results. The L1 distributed cache uses a hybrid storage architecture, with independent control of eFlash and SRAM partitions: eFlash partition: uses NOR flash memory cells, operating voltage 1.8V~2.0V, erase block size 4KB, programming latency 85ns~97ns, read latency 12ns~15ns, supports byte-by-byte read / write and block-by-block erase; SRAM partition: uses 6 transistor cells, operating voltage 1.0V, read / write latency ≤6.8ns, supports single-bit / half-word / full-word access; the address mapping module uses a combination of hash mapping and offset addressing, with the mapping algorithm being XOR hash and linear offset, and address translation latency ≤5.3ns; the data verification module uses 16-bit ECC verification, supports 2-bit automatic error correction and 4-bit error detection, and error correction latency ≤4.1ns; the read / write control module has a built-in state machine, supporting six working states: idle, read, write, prefetch, migrate, and reclaim, with state switching latency ≤3.2ns.

[0032] The convergence cache (L2) is deployed on the top-level logic wafer and is a globally shared cache with a total capacity of 2.2MB ± 0.15MB. It uses 8TSRAM storage units and is divided into 20 cache groups. Each group supports 2-way to 4-way dynamic associative mapping. It adopts a cache group structure with multiple groups that can dynamically switch mapping modes, and can switch in real time between three modes: fully associative, directly associative, and group associative. The cache groups are fully interconnected through a 16×16 crossbar switch matrix. The data transmission latency between any two groups is ≤8.7ns, and there is no centralized relay node. The L2 converged cache adopts a group-associative reconfigurable architecture: Storage array: 8 TSRAM cells, holding voltage 0.7V, static power consumption ≤1.2mW / KB, data retention time ≥100ms; Association mapping module: each cache group is equipped with 4 comparators, simultaneously comparing 4 tag addresses, hit judgment latency ≤6.5ns; Crossbar switch matrix: 16 inputs and 16 outputs non-blocking crossbar switches, using a transmission gate selection structure, on-resistance of any port ≤45Ω, transmission latency ≤8.7ns; Cache management module: real-time statistics of each cache group's hit count, access frequency, dirty data ratio, and remaining space, with a statistical period of 8ns~12ns; Signal enhancement module: includes bit line equalization unit, reference voltage calibration unit, and differential signal isolation unit, bit line equalization voltage deviation ≤±3.8mV, inter-layer crosstalk suppression ratio ≤-42dB.

[0033] It should be noted that all three levels of cache are configured with a unified signal integrity protection circuit: Bit line equalization circuit: Automatically equalizes bit line voltage during read / write intervals, with an equalization time of 1.2ns~1.8ns and a bit line voltage deviation of ≤±2.9mV after equalization; Voltage calibration circuit: Real-time calibration of the induction amplifier reference voltage, calibration step size 0.5mV, calibration accuracy ±3.5mV; Noise isolation circuit: It adopts differential signal transmission and grounded shielding structure to suppress power supply noise and crosstalk noise to below -45dB; Timing calibration circuit: compensates for transmission delays along different paths, controlling timing deviations within ±0.13ns to ensure stable read / write timings.

[0034] It should be noted that the global collaborative scheduling controller adopts a dual-core RISC-V 2.5GHz processor architecture and has five core sub-modules: tensor feature parsing unit, cache status monitoring unit, access conflict arbitration unit, scheduling decision unit, and instruction dispatch unit. The controller integrates 256KB instruction cache and 128KB data cache on-chip, with an instruction execution efficiency of ≥1.92DMIPS / MHz, and can handle 64 concurrent scheduling requests simultaneously.

[0035] The tensor feature parsing unit performs real-time parsing and three-level classification of the input AI tensor data. The parsing process is entirely hardware-based, with no software intervention. During the feature extraction stage, the tensor's dimensional parameters (N / C / H / W), access step size, consecutive access length, computational dependencies, access time interval, and hit history are extracted in real time. In the popularity calculation stage, the exponentially weighted moving average (EWMA) algorithm is used to calculate the access popularity value H, with the formula: H(n) = α × H(n-1) + (1-α) × S(n) Where α is the smoothing coefficient, α = 0.72 ± 0.03, and S(n) is the access flag for the current period (1 for access, 0 for no access). During the tensor classification phase, tensors are divided into three categories based on their frequency: core tensors H ≥ 0.76 ± 0.04 represent frequently accessed, computationally dependent weights or features; auxiliary tensors 0.39 ± 0.03 ≤ H < 0.76 ± 0.04 represent intermediate computational data with medium-frequency access; and temporary tensors H < 0.39 ± 0.03 represent data with low-frequency access and one-time computation. During the classification and labeling phase, a 2-bit classification flag is added to each tensor and stored in the cached label bit. The flag is directly read during scheduling to execute the policy.

[0036] The cache status monitoring unit performs full-dimensional status sampling of the three-level cache with a sampling period of 10ns ± 1.5ns. The sampled data is then input into the scheduling decision unit after digital filtering. Micro-cache monitoring items include occupancy rate, hit count, missing count, dirty data quantity, free space, and computation unit waiting status; distributed cache monitoring items include occupancy rate, hit rate, dirty block quantity, prefetch queue length, migration queue length, and inter-layer communication status; aggregation cache monitoring items include global occupancy rate, group hit rate, crossbar switch congestion status, interface data traffic, and storage array access status. Data filtering uses a moving average filter to remove instantaneous noise and interference data, ensuring stable scheduling decisions.

[0037] The scheduling decision-making unit adopts a weighted multi-factor decision-making model, and the decision formula is as follows: S core =W1×H+W2×D+W3×L+W4×T+W5×P Wherein, W1 is the access popularity weight, with a value of 0.34±0.02; W2 is the computation dependency weight, with a value of 0.26±0.02; W3 is the spatial locality weight, with a value of 0.21±0.02; W4 is the latency weight, with a value of 0.12±0.01; W5 is the power consumption weight, with a value of 0.07±0.01; D is the computation dependency, L is the spatial locality value, T is the access latency, and P is the power consumption cost. The scheduling decision units are sorted from high to low score, the decision calculation latency is ≤7.8ns, and it supports outputting 16 scheduling instructions per cycle.

[0038] The instruction dispatch unit is used to execute the computation task scheduling process. Specifically, the computation task scheduling process includes: the global collaborative scheduling controller receives computation tasks from external hosts or internal computing units, parses the task type, tensor address, and computation parameters; allocates the tasks to idle computing units according to the load balancing principle of computing subclusters; the scheduling controller's instruction cache prefetches the required tensor data into the micro-cache of the target computing unit; and sends a computation enable signal to the computing unit, which then starts pipelined computation.

[0039] The access conflict arbitration unit is used to execute the computation conflict resolution mechanism. Specifically, the computation conflict resolution mechanism includes: address conflicts are resolved according to computation urgency and task priority, with higher urgency tasks receiving priority access; bandwidth conflicts are resolved using a time slot allocation method, allocating a fixed transmission time slot to each computation unit, with no contention or preemption; resource conflicts are resolved using a status marking method, with computation units, cache ports, and storage ports automatically waiting when busy, and automatically rescheduling after a timeout.

[0040] The specific implementation of the hierarchical power control strategy executed by the low-power control module is as follows: Micro-cache: When idle time is ≥8ns, it enters shallow sleep, retaining power supply only for the address listening circuit, reducing power consumption by 72%±3%; Distributed cache: When idle time is ≥55μs, the eFlash partition enters deep sleep, while the SRAM partition retains power to keep the data, reducing power consumption by 85%±3%; Convergence cache: Power consumption is controlled at the group level, and the power is directly turned off for idle cache groups, with the shutdown ratio dynamically adjustable from 0% to 65% and a shutdown delay ≤6.2ns; Computing unit: When idle, the arithmetic array is turned off, retaining power supply only for the control circuit, reducing power consumption by 89%±2%.

[0041] Furthermore, Dynamic Voltage and Frequency Scaling (DVFS) enables voltage regulation: the operating voltage is adjustable in three levels, 0.65V, 0.9V, and 1.1V, with an adjustment step of 0.05V and an adjustment response time of ≤6.8ns; frequency regulation: the operating frequency is adjustable in three levels, 200MHz, 800MHz, and 2.0GHz, with no glitches during frequency switching; and adjustment triggering: it automatically adjusts according to system load, cache utilization, and computing throughput, achieving high performance under high load and low power consumption under low load.

[0042] Furthermore, power consumption-aware computing scheduling monitors the total system power consumption in real time with a monitoring accuracy of ±1.2mW and a monitoring period of 5ns. The power consumption threshold is 720mW±30mW. If the threshold is exceeded, the accuracy and frequency of non-core computing units are automatically reduced. When the power consumption falls back below the threshold, the original computing accuracy and operating frequency are automatically restored without performance loss.

[0043] This application embodiment also provides a data reliability and anomaly protection scheme implemented through a data reliability protection module. The micro-cache uses parity checking with a check latency of ≤1.5ns and single-bit error detection; the distributed cache uses 16-bit ECC checking, with 2 bits for error correction and 4 bits for error detection; the aggregation cache uses CRC-32+ECC hybrid checking with an error detection rate of ≥99.9999% and an error correction coverage of ≥99.99%.

[0044] The data reliability protection module performs different levels of data verification on the three-level cache, performs bad block management and wear leveling on the storage array, and implements overheat protection and power-loss protection. Overheat protection includes temperature monitoring and automatic frequency and voltage reduction for overheating. Power-loss protection includes backup power supply startup and writing valid cache data to the non-volatile storage area. Specifically, upon power-up, the storage array is automatically scanned with a detection accuracy of ±1 unit, and bad block markers are written to a dedicated register. Redundant block replacement is used, with a redundancy ratio of 3%, and replacement does not affect addressing or capacity. A dynamic wear leveling algorithm is used to balance the number of erase / write cycles for all storage blocks, with a wear deviation ≤8.3%, extending the lifespan to ≥100,000 erase / write cycles. The temperature sensor accuracy is ±0.5℃, the overheat threshold is 90℃±3℃, and automatic frequency and voltage reduction is implemented for overheating. The backup power supply (on-chip integrated microcapacitor) startup time is ≤1.6μs. During power-loss, valid cache data is automatically written to the non-volatile area of ​​the storage array, with a data loss rate ≤0.001%. After power-on, cache data is automatically recovered from the storage array with a 100% recovery success rate.

[0045] The heterogeneous interface adapter unit supports three protocols: PCIe 5.0, MIPIC-PHY, and a custom high-speed storage interface, with a maximum transmission rate of 32Gbps. It incorporates a protocol conversion module and a bit-width adaptation module, supporting dynamic adaptation to any bit width from 8bit to 512bit. The specific implementation method is as follows: Protocol conversion: The heterogeneous interface adapter unit receives instructions from the external host, synchronizes them via a synchronization trigger, and then inputs them into the protocol parsing module; it parses the instruction type, address, data length, and verification information, with a parsing latency of ≤14ns; The PCIe / MIPI protocol is converted into an internal custom storage protocol, and the instruction format is unified as 36-bit address + 256-bit data + 8-bit control bits. The converted instructions are sent to the global collaborative scheduling controller to perform cache / storage access.

[0046] Data writing: The heterogeneous interface adapter unit receives external data and stores it in the receiving buffer with a buffer depth of 48 levels; it splits / assembles the data according to the internal storage bit width and adds a checksum; the data is sent to the L2 aggregation buffer and distributed to the L1 buffer or storage array according to the scheduling policy; after writing is completed, it returns an ACK response to the host with a response latency of ≤28ns.

[0047] Data readout: The heterogeneous interface adapter unit sends the read command to the scheduling controller, and the controller instruction stores / buffers the read data; the data is transmitted step by step from L2 to L1 to the heterogeneous interface adapter unit, and output after verification; it is converted to the external interface bit width and sent to the external host; when the host's receiving rate is insufficient, the transmission buffer is automatically started to avoid data loss, and the flow control response time is ≤9ns.

[0048] Example 2: This example provides an adaptive tensor cache scheduling method based on distributed near-memory computing. This method is applied to the aforementioned system and includes: S1: Perform feature parsing on the input tensor data, extract the tensor's dimension parameters, access step size, consecutive access length, computational dependencies, and access time interval, calculate the access popularity value, and divide the tensor into core tensors, auxiliary tensors, and temporary tensors based on the access popularity value.

[0049] The tensor feature parsing unit of the global collaborative scheduling controller performs real-time parsing and three-level classification on the input AI tensor data. First, it extracts the tensor's dimension parameters, access step size, consecutive access length, computational dependencies, and access time interval in real time. Then, it uses the EWMA algorithm to calculate the access popularity value H. Next, based on the relationship between the access popularity value and the classification threshold, the tensor is divided into core tensors, auxiliary tensors, and temporary tensors. Finally, a classification label is added to each tensor and stored in the cache label bit.

[0050] S2: Perform full-dimensional state sampling on the three levels of cache (micro-cache, distributed cache, and converged cache) at a preset sampling period, and obtain the occupancy rate, hit rate, and dirty data volume of each cache level after digital filtering. Specifically, the cache state monitoring unit samples the state of the micro-cache, distributed cache, and converged cache respectively, and obtains monitoring data such as occupancy rate, hit rate, dirty data volume, and inter-layer communication congestion of each cache level, which is then output after moving average filtering.

[0051] S3: Based on a weighted multi-factor decision model considering access popularity, computational dependency, spatial locality, access latency, and power consumption, tensor data is prioritized and scheduling instructions are generated. Specifically, tensor data is prioritized and scheduling instructions are generated based on the weighted multi-factor scheduling decision model. The scheduling decision unit comprehensively considers five factors—access popularity, computational dependency, spatial locality, access latency, and power consumption—and sorts them from highest to lowest weighted score, prioritizing the scheduling of high-priority tensor data.

[0052] S4: Executes cache data migration and prefetching according to scheduling instructions, realizing coordinated addressing and data interaction between the storage array, L3 cache, and computing units. Includes: S4.1: When the micro-cache occupancy rate is greater than or equal to the first occupancy threshold, the ping-pong buffer non-blocking migration mechanism is activated to migrate temporary tensors and low-priority auxiliary tensors from the micro-cache to the distributed cache; the specific implementation method is as follows: When the micro-cache (L0) occupancy rate is ≥75%±3%, data migration is automatically initiated, using a ping-pong buffer non-blocking migration mechanism. Based on the classification label and score value, temporary tensors and low-score auxiliary tensors are filtered out and marked as data to be migrated; the standby buffer is enabled, the computing unit switches to the standby buffer to perform operations, and the original cache data is migrated out without interruption; the data to be migrated is written to the SRAM partition of the corresponding L1 distributed cache in address order, with a write bit width of 256 bits and a migration rate of 32GB / s; the original cache space is cleared, the cache status register and address mapping table are updated, and the system switches back to the original cache, with a total migration latency of ≤21ns.

[0053] S4.2: When the distributed cache hit rate is less than or equal to the first hit threshold, tensor space locality prefetching is initiated. Based on historical access addresses, the tensor address range of the next consecutive access is predicted, and data is prefetched from the aggregation cache or storage array to the distributed cache. The specific implementation is as follows: When the distributed cache (L1) hit rate is ≤46%±3%, initiate tensor space locality prefetching: Based on historical access addresses, predict the tensor address range for the next consecutive access segment. The predicted length is 1.5 ± 0.2 times the single calculation requirement. The scheduling controller prioritizes TSV direct connection channels to avoid congested inter-layer links, with a path switching latency of ≤ 5.6 ns. The L2 aggregation cache reads prefetched data from itself or the storage array and transmits it to the target L1 cache via a cross-connect matrix. Prefetched data is marked with a prefetch flag. If a hit occurs, the prefetched data is used first, and if a hit occurs, it is automatically discarded, with a prefetch hit rate of ≥ 68%.

[0054] It should be noted that the computing unit sends a read request to the micro-cache. If the micro-cache hits, the data is output directly; if the micro-cache misses, a read request is sent to the distributed cache; if the distributed cache misses, a read request is sent to the convergence cache; if the convergence cache misses, a read instruction is sent to the storage array; the storage array reads the data and sends it to the convergence cache, while simultaneously initiating prefetching; after completing the computation, the computing unit writes the result back to the micro-cache, and dirty data is automatically marked and periodically written back to the distributed cache or convergence cache. The specific implementation method is as follows: The computing unit initiates a read request: The computing unit sends a read instruction, tensor address, and data length to the L0 microbuffer. The address width is 36 bits, and the instruction transmission latency is ≤0.5ns. L0 hit detection: The L0 address mapping module compares tags in a single cycle. If a hit is found, the data is directly output to the computing unit's operation logic. L0 miss handling: Send a read request to the corresponding L1 cache; if the L1 cache is hit, transmit the data to L0 through the inter-layer dedicated channel, with a transmission latency of ≤11.5ns. L1 miss handling: Send a read request to the L2 cache; if L2 cache hits, transmit the data via the crossbar switch matrix; if L2 cache misses, send a read command to the storage array. Memory array read: The pre-decoding module locates the target memory block, the resistive sensing circuit reads the data, and sends it to the L2 cache via the I / O drive circuit, while simultaneously initiating prefetch; Computation and Write-back: After the computation unit completes the operation, it writes the result directly back to the L0 micro-cache. Dirty data is automatically marked and written back to the L1 / L2 cache periodically.

[0055] Example 3, the third embodiment of the present invention, provides an adaptive tensor cache scheduling system and method based on distributed near-memory computing. It is applied to a three-dimensional vertically stacked phase-change memory (3DPCM) architecture with 5 layers of storage wafers and 1 layer of top-level logic wafers. Designed for high-concurrency AI image inference scenarios at the edge, it achieves low-latency access to tensor data, in-situ near-memory computing, and adaptive cache scheduling. It can adapt to 4-way concurrent AI object detection inference tasks. The specific implementation is as follows: I. System Hardware Architecture and Physical Deployment: The hardware in this embodiment adopts a fusion architecture of 3DPCM, distributed near-memory computing (PNC), and hierarchical tensor caching. All hardware modules are interconnected point-to-point through Mesh topology on-chip network (NoC), abandoning the traditional centralized bus structure and eliminating bandwidth bottlenecks.

[0056] 3D stacked memory array deployment: Employing a 5-layer vertical stacking of memory wafers and bonding of the top-layer logic wafer, the layers are interconnected via Cu-SiO2 composite dielectric TSVs. The TSV diameter is 2.6 μm, the spacing is 9.8 μm, the vertical spacing between layers is 18.1 μm, and the bonding yield is 99.994%. Each memory wafer integrates a 304MB phase-change memory cell array. The memory cells have a 1S1R structure, and the phase-change material uses a Ge2Sb2Te5-TiN composite doping system with a crystallization temperature of 172℃, a set voltage of 1.43V, a reset voltage of 2.26V, and a data retention time of 12 years at 85℃.

[0057] Each layer of the memory array is divided into 16 independent 19MB memory blocks. Each memory block is configured with independent pre-decoding, I / O drivers, resistive sensing circuits, and bad block registers, supporting block-level parallel read, write, and erase operations. The top-level logic wafer uses a 22nm FD-SOI process and integrates a global collaborative scheduling controller, a 2.2MB L2 convergence cache, heterogeneous interface adapter units, and a TSV routing matrix. The operating voltage is 1.0V, and the leakage current is 15.6nA / μm.

[0058] Distributed near-memory computing unit cluster configuration: Eight heterogeneous computing subclusters are configured, asymmetrically mapped to five layers of storage wafer memory blocks at a ratio of 1:4.5. Each computing subcluster contains 12 basic computing units and 2 enhanced computing units. The basic units integrate an 8-bit fixed-point multiply-accumulate array and ReLU activation hard logic, while the enhanced units are equipped with a 16-bit multiplier, a 64-bit accumulator, and batch normalization logic, capable of performing 16 8-bit multiply-accumulate operations per cycle with an accuracy error of ±0.02%. The computing units adopt a 2D ring interconnect with 128-bit width transmission within the ring, a latency of 6.0ns, and support for 128-way concurrent conflict-free computing.

[0059] Three-level hierarchical adaptive tensor cache construction: Construct a three-level asymmetric bidirectional cache architecture consisting of micro-caches, distributed caches, and convergence caches, with a 256-bit write channel and a 192-bit read channel, matching the write-heavy and read-light characteristics of AI inference tensors.

[0060] Micro-cache: Each computing unit has a built-in 3.2KB independent 6-tube SRAM micro-cache with a cell area of ​​0.024μm², an operating frequency of 2.0GHz, and a single-cycle read / write operation of 0.5ns, supporting in-situ logic operations; Distributed caching: Each storage wafer is configured with four 96KB caches, with 72% of the capacity in NOR eFlash for storing target detection weight tensors and 28% in SRAM for storing intermediate results of feature maps. Independent address mapping, ECC verification and scheduling control circuitry are also configured. Convergence Cache: Top-level global shared cache, divided into 20 cache groups with dynamically switchable mapping modes, fully interconnected through 16×16 non-blocking cross switches, with a data transmission latency of 8.5ns between groups.

[0061] Global scheduling and heterogeneous interface adaptation unit configuration: The global collaborative scheduling controller adopts a dual-core RISC-V architecture with a main frequency of 2.5GHz. It integrates tensor feature parsing, cache monitoring, conflict arbitration, power consumption control, and instruction distribution modules, and integrates 256KB instruction cache and 128KB data cache, which can handle 64 scheduling requests simultaneously. The heterogeneous interface adaptation unit is equipped with a PCIe 5.0 protocol interface with a maximum speed of 32Gbps, supports dynamic adaptation of 8~512bit bit width, and has a protocol conversion latency of 15ns.

[0062] II. Hardware Circuit Implementation: The three-level cache circuit is implemented according to the design scheme of the embodiment of this application: L0 micro-cache uses 28 groups of 128 rows × 64 columns of storage sub-arrays, three-stage inverter word line drive, latch-type sensitive amplifier bit line sensing, and timing deviation ±0.08ns; L1 cache eFlash partition 4KB erase block, SRAM partition 6 transistor units, XOR hash and linear offset addressing, 16bit ECC to achieve 2bit error correction; L2 cache 8 TSRAM units, cross switch on-resistance 42Ω, and configured with signal enhancement module; the whole system is equipped with bit line equalization, voltage calibration, noise isolation, and timing calibration circuits to suppress power supply crosstalk noise to -46dB and ensure signal integrity.

[0063] III. Adaptive Tensor Cache Scheduling Execution Flow: This embodiment of the application executes the complete scheduling flow for a 4-way parallel YOLO-v5 object detection inference task: Tensor Feature Parsing and Classification: The tensor feature parsing unit is hardware-based for real-time parsing and inference of tensor dimensions N / C / H / W, access stride, and computational dependencies. The EWMA algorithm (α=0.72) is used to calculate the access heat H. The YOLO-v5 convolution weight tensor is labeled as the core tensor (H=0.81), the intermediate tensor of the feature map is labeled as the auxiliary tensor (H=0.58), and the temporary residual tensor is labeled as the temporary tensor (H=0.31). A 2-bit classification label is added to each tensor.

[0064] Real-time cache status monitoring: The status of the three-level cache is sampled at a period of 10ns, and the monitoring data is output after 8-cycle moving average filtering to obtain the cache occupancy rate, hit rate, dirty data volume, and inter-layer communication congestion in real time.

[0065] Weighted Multi-Factor Scheduling Decision: The scheduling decision module uses a weighted formula: S core =0.34×H+0.26×D+0.21×L+0.12×T+0.07×P, prioritize the tensor data, output 16 scheduling instructions per cycle, and prioritize scheduling core weight tensors to L0 / L1 cache.

[0066] Cache data migration and prefetching: When the micro-cache occupancy rate reaches 76%, ping-pong buffer non-blocking migration is initiated to move temporary tensors out to the L1 cache SRAM partition, with a total migration latency of 20ns; when the L1 cache hit rate drops to 45%, tensor prefetching is initiated, predicting the address range according to 1.5 times the single inference demand, and prefetching tensors via the TSV direct connection channel, with a prefetch hit rate of 70%.

[0067] Storage-Cache-Computation Cooperative Addressing: When a computation unit initiates a tensor read request, it first accesses the local L0 micro-cache; if L0 is not hit, it accesses the corresponding L1 cache with a latency of 11ns; if L1 is not hit, it is relayed through the L2 cache and finally reads the data from the three-dimensional stacked storage array while performing prefetching; after computation, the inference feature results are directly written back to the L0 cache, and dirty data is periodically flushed back to the L1 cache.

[0068] IV. Optimization of Inter-Layer Communication in 3D Stacked Storage: Inter-layer communication adopts a hybrid multiplexing mechanism of FDM and TDM. Control signals are transmitted through frequency division multiplexing of 8 subcarriers, and data signals are scheduled using time-division multiplexing of 7.8ns time slots. The TSV interface is equipped with a 55Ω adjustable matching resistor, which dynamically increases the driving voltage to 1.25V during cross-layer transmission and compensates for high-frequency attenuation through the CTLE circuit. The inter-layer queue occupancy rate is monitored in real time, and the transmission path is switched when the occupancy rate reaches 72%. The flow is smoothed through the leaky bucket algorithm. Each frame of data is appended with a 16-bit CRC check code to achieve 1-bit error correction and 2-bit error detection. Data that cannot be corrected triggers 3 automatic retransmissions, with a retransmission success rate of 99.999%.

[0069] V. Distributed Near-Memory Computing – Cache Collaboration: The global collaborative scheduling controller receives 4 YOLO-v5 inference tasks and allocates computing units according to the load balancing principle of 8 computing subclusters, with a load deviation of 7.5%; the core weight tensors are prefetched into the L0 cache of the corresponding computing unit; the computing unit directly reads data from L0 to complete convolution, activation, and batch normalization operations, and the intermediate results are temporarily stored in the L0 cache; multi-unit access conflicts are resolved according to task urgency and time slot division, ensuring that the 4 inference tasks can run in parallel without lag.

[0070] VI. Low-Power Control Implementation: L0 cache enters light sleep after 9ns of idle time, reducing power consumption by 72%; L1 cache eFlash partition enters deep sleep after 55μs of idle time, reducing power consumption by 85%; L2 cache idle groups are dynamically shut down, with a maximum shutdown of 60%; idle compute units shut down the arithmetic array, reducing power consumption by 89%. The system adopts DVFS three-level voltage and frequency regulation, operating at 1.1V and 2.0GHz during peak load periods, switching to 0.65V and 200MHz during low load periods; power consumption is monitored in real time, and when the system power consumption reaches 730mW, the operating frequency of non-core units is reduced, and performance is automatically restored as power consumption falls back.

[0071] VII. Data Reliability and External Interaction: The three-level cache employs parity check, 16-bit ECC, CRC-32, and a hybrid ECC checksum. Upon power-up, the storage array scans for bad blocks, replacing them with 3% redundant blocks. Dynamic wear leveling reduces the write / erase deviation of storage blocks to 8%, supporting 100,000 write / erase cycles. Automatic frequency reduction occurs when the temperature exceeds 90°C. During power loss, on-chip microcapacitors provide power, writing valid cached data to the non-volatile storage area within 1.5μs, achieving a data loss rate of 0.0009%. Externally, inference image data is received via the PCIe 5.0 interface, converted to a different protocol, and adapted to the bit width before being written to the system cache. After inference, results are read out sequentially, flow-controlled, and output to the external host, achieving end-to-end low-latency data interaction.

[0072] This embodiment is applied to a 4-way parallel AI object detection inference scenario at the edge. Compared with the traditional centralized caching architecture, the inference latency is reduced by 41%, the data transfer power consumption is reduced by 36%, and the system concurrent throughput is increased by 58%. It effectively solves the problems of centralized bandwidth bottleneck, poor tensor scheduling adaptation, uneven inter-layer communication latency, and multi-unit concurrent conflicts, and realizes distributed near-memory computing and adaptive tensor cache scheduling under high-density three-dimensional stacked storage.

[0073] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. An adaptive tensor cache scheduling system based on distributed near-memory computing, characterized in that, include: A three-dimensional stacked memory array includes a multi-layer memory wafer and a top-level logic wafer. The multi-layer memory wafer and the top-level logic wafer are interconnected by vertical bonding through silicon vias. Each memory wafer integrates a phase-change memory cell array and is divided into multiple independent memory blocks. The top-level logic wafer integrates a global collaborative scheduling controller and is deployed as a converged cache as a third-level cache. The distributed near-memory computing unit cluster includes multiple computing sub-clusters. Each computing sub-cluster is physically mapped to a storage block in the three-dimensional stacked storage array. Each computing sub-cluster contains multiple computing units, which are used to perform tensor operations. The hierarchical adaptive tensor cache module includes a three-level cache structure of micro-cache, distributed cache and convergence cache. The micro-cache is coupled inside the computing unit, the distributed cache is bound to the physical layer of the storage wafer, and the convergence cache, as the third-level cache of the hierarchical adaptive tensor cache module, is deployed on the top-level logic wafer. The three-level caches are interconnected by an asymmetric bidirectional data channel. The global collaborative scheduling controller is used to perform feature parsing and classification of tensor data, monitor the status of the three-level cache, and perform cache scheduling of tensor data according to weighted multi-factor scheduling decisions, so as to realize the collaborative work between storage, cache and computation. The three-dimensional stacked storage array, the distributed near-memory computing unit cluster, the hierarchical adaptive tensor cache module, and the global collaborative scheduling controller are interconnected point-to-point through an on-chip network Mesh.

2. The adaptive tensor cache scheduling system based on distributed near-memory computing as described in claim 1, characterized in that, The micro-cache uses static random access memory, which is deeply coupled to each computing unit and supports single-cycle read / write and in-situ logic operations. The distributed cache adopts a hybrid architecture of non-volatile storage and volatile storage. The non-volatile storage is used to store long-term valid tensor weight data, and the volatile storage is used to store temporary intermediate calculation results. The converged cache adopts a cache group structure with multiple groups of dynamically switchable mapping modes, and the cache groups are fully interconnected through a cross-switch matrix.

3. The adaptive tensor cache scheduling system based on distributed near-memory computing as described in claim 1, characterized in that, The global collaborative scheduling controller includes: a tensor feature parsing unit, a cache status monitoring unit, a scheduling decision unit, an access conflict arbitration unit, and an instruction distribution unit; The tensor feature parsing unit is used to extract the tensor's dimension parameters, access step size, consecutive access length, computational dependencies, and access time interval in real time. It uses an exponentially weighted moving average algorithm to calculate the access popularity value and classifies the tensor into three categories: core tensor, auxiliary tensor, and temporary tensor based on the access popularity value. The cache status monitoring unit is used to perform full-dimensional status sampling on the three-level cache at a preset sampling period, and output the monitoring data after digital filtering. The scheduling decision unit is used to prioritize tensor data and output scheduling instructions based on a weighted multi-factor decision model. The access conflict arbitration unit is used to perform access arbitration based on task priority, computation urgency and preset time slots when multiple computing units concurrently access the same cache address, cache port or storage port. The instruction distribution unit is used to distribute the scheduling instructions output by the scheduling decision unit.

4. The adaptive tensor cache scheduling system based on distributed near-memory computing as described in claim 1, characterized in that, The inter-layer communication of the three-dimensional stacked storage array adopts a hybrid transmission mechanism of frequency division multiplexing and time division multiplexing. The control signal is transmitted through multiple subcarriers of frequency division multiplexing, and the data signal is transmitted through time slot scheduling of time division multiplexing. The through-silicon via interface has built-in adjustable matching resistors and signal compensation circuits to dynamically adjust the driving voltage and signal slope according to the transmission distance. Each layer communication controller has a built-in data buffer queue and monitors the occupancy rate. When the occupancy rate reaches the congestion threshold, it sends a congestion flag to the global collaborative scheduling controller, which then switches the inter-layer data transmission path. A leaky bucket flow control algorithm is used to smooth the flow. Inter-layer data transmission is accompanied by a CRC checksum, which supports automatic error correction for single-bit errors and detection of multi-bit errors. Automatic retransmission is triggered when errors cannot be corrected.

5. The adaptive tensor cache scheduling system based on distributed near-memory computing as described in claim 1, characterized in that, The system also includes a low-power control module; A low-power control module is used to execute a hierarchical power control strategy; When the micro-cache idle time is greater than or equal to the first threshold, it enters a shallow sleep mode; When the idle time of the distributed cache is greater than or equal to the second threshold, the non-volatile storage partition enters a deep sleep mode; the pooled cache is controlled by group level, and the power is turned off when the idle cache group is idle; the computing unit shuts down the arithmetic array when it is idle. The low-power control module is also used to perform dynamic voltage and frequency adjustment based on system load, cache utilization, and computing throughput.

6. The adaptive tensor cache scheduling system based on distributed near-memory computing as described in claim 1, characterized in that, In the distributed near-memory computing unit cluster, each computing sub-cluster contains a basic computing unit and an enhanced computing unit. The basic computing unit integrates a fixed-point multiplication array, an adder, and activation function hard logic, while the enhanced computing unit additionally integrates a high-bit-width multiplier, an accumulator, and batch normalization hard logic. The computing units are interconnected in a ring topology, supporting ring forwarding and sharing of intermediate values.

7. The adaptive tensor cache scheduling system based on distributed near-memory computing as described in claim 1, characterized in that, The system also includes a data reliability protection module and a heterogeneous interface adaptation unit; The data reliability protection module is used to perform different levels of data verification on the three-level cache, perform bad block management and wear leveling on the storage array, and perform overheat protection and power failure protection. The overheat protection includes temperature monitoring and automatic frequency and voltage reduction when the temperature exceeds the limit, and the power failure protection includes backup power supply startup and writing valid cache data to the non-volatile storage area. The heterogeneous interface adapter unit supports multiple external protocol interfaces and has a built-in protocol conversion module and bit width adaptation module to convert external protocols into internal storage protocols and support dynamic adaptation of multiple bit widths.

8. An adaptive tensor cache scheduling method based on distributed near-memory computing, applied to the adaptive tensor cache scheduling system based on distributed near-memory computing as described in any one of claims 1-7, characterized in that, include: The input tensor data is subjected to feature parsing to extract the tensor's dimension parameters, access step size, consecutive access length, computational dependencies and access time intervals, and access popularity value is calculated. Based on the access popularity value, the tensor is divided into core tensors, auxiliary tensors and temporary tensors. The system performs full-dimensional state sampling on the three-level cache (micro-cache, distributed cache, and convergence cache) at a preset sampling period, and obtains the occupancy rate, hit rate, and dirty data volume of each level of cache after digital filtering. Based on a weighted multi-factor decision model that considers access popularity, computational dependency, spatial locality, access latency, and power consumption, tensor data is prioritized and scheduling instructions are generated. The system performs cache data migration and prefetching according to scheduling instructions, enabling collaborative addressing and data interaction between the storage array, L3 cache, and computing units.

9. The adaptive tensor cache scheduling method based on distributed near-memory computing as described in claim 8, characterized in that, The step of performing cached data migration and prefetching according to scheduling instructions includes: When the micro-cache occupancy rate is greater than or equal to the first occupancy threshold, the ping-pong buffer non-blocking migration mechanism is activated to migrate temporary tensors and low-priority auxiliary tensors from the micro-cache to the distributed cache. When the distributed cache hit rate is less than or equal to the first hit threshold, tensor space locality prefetching is initiated. Based on the historical access addresses, the tensor address range of the next consecutive access is predicted, and data is prefetched from the aggregate cache or storage array to the distributed cache.

10. The adaptive tensor cache scheduling method based on distributed near-memory computing as described in claim 8, characterized in that, The step of performing cache data migration and prefetching according to scheduling instructions to achieve collaborative addressing and data interaction between the storage array, L3 cache, and computing units includes: The computing unit sends a read request to the micro-cache; if the micro-cache is hit, the data is output directly. If the micro-cache misses, a read request is sent to the distributed cache; If the distributed cache misses, a read request is sent to the converged cache; If the convergence cache misses, a read command is sent to the storage array; After the storage array reads the data, it sends it to the convergence cache and simultaneously initiates prefetching; After the computing unit completes the operation, it writes the result back to the micro-cache. Dirty data is automatically marked and written back to the distributed cache or the aggregation cache at regular intervals.

Images