A server intelligent integrated layered storage system and method based on a U.2 interface
By redefining the U.2 interface parallel channel and using a temporal convolutional neural network to predict data popularity, the problems of low resource utilization and low data migration efficiency in hierarchical storage systems are solved, achieving efficient resource management and data migration optimization.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-10-14
- Publication Date
- 2026-04-10
AI Technical Summary
Existing tiered storage systems face challenges in terms of resource utilization and intelligent data scheduling, including issues such as fragmented hardware resources, inaccurate data migration decisions, and resource waste, especially under high and low load scenarios.
By redefining the reserved pins of the U.2 interface to establish a multi-protocol parallel channel, combining signal integrity parameters and device I/O access characteristics to generate physical layer status information, and using a temporal convolutional neural network to predict data access frequency, dynamically adjust the concurrency of migration tasks and data block granularity, thereby achieving zero-copy migration of data blocks and resource coordination.
It improves the resource utilization and economy of the storage system, reduces the total cost of ownership, ensures the accuracy of data migration and the system's adaptive optimization capabilities, and avoids resource waste and migration delays.
Smart Images

Figure CN121300702B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of computer server storage, in particular to a server intelligent integrated hierarchical storage system and method based on a U.2 interface. BACKGROUND
[0002] With the rapid development of cloud computing, big data and artificial intelligence applications, servers are responsible for massive data storage and high-speed processing in enterprise-level data centers and high-performance computing environments. In the face of diversified business needs, it is necessary to meet the requirements of high performance for critical tasks and also to consider the economic storage of large-capacity data. Therefore, hierarchical storage systems have gradually become mainstream solutions, which achieve the balance between performance and cost by classifying and migrating data between different types of storage media. However, with the diversification of SSD interface protocols and the complexity of application access patterns, traditional storage architectures face challenges in resource utilization and data scheduling intelligence.
[0003] The existing hierarchical storage method has some substantial defects in practical application: on the one hand, different interfaces and protocols often rely on independent controllers and channels, resulting in hardware resource fragmentation, and the system needs additional bridge devices to support multiple protocol concurrency, causing architecture redundancy and increased energy consumption; on the other hand, traditional data heat recognition mainly relies on file system or logical layer historical access statistics, which is difficult to reflect the dynamic changes of underlying physical link state and I / O characteristics, and is prone to cause heat data determination lag or even error, thereby causing inaccurate migration decision; in addition, the existing migration scheduling is usually executed with fixed task concurrency and data block granularity, which cannot be adjusted in real time according to system load, resulting in link congestion and reduced overall throughput in high load scenarios, and causing resource idling in low load scenarios, which seriously affects the running efficiency and reliability of the hierarchical storage system. SUMMARY
[0004] In view of the deficiencies of the prior art, the present application provides a server intelligent integrated hierarchical storage system and method based on a U.2 interface, which solves the problems of the above background technology.
[0005] To achieve the above object, the application is implemented by the following technical solutions: a server intelligent integrated layered storage system based on a U.2 interface, comprising the following modules: a state sensing module, configured to establish multiple protocol parallel channels by redefining reserved pins of the U.2 interface, monitor signal integrity parameters of each channel in real time, fuse and process the signal integrity parameters and device I / O access features, and generate an I / O feature sequence of physical layer state information; a heat prediction module, configured to receive the I / O feature sequence of the physical layer state information, extract multi-scale features of an access mode through a time sequence convolutional neural network, calculate access heat probability of each data block, and determine a target storage level of each data block according to the access heat probability, thereby generating a data migration scheduling instruction containing a source physical address, a target level identifier and a migration priority; a migration execution module, configured to receive the data migration scheduling instruction, read a data block from a corresponding SSD flash memory according to the source physical address in the instruction, allocate an isolated buffer of a corresponding level in a shared DRAM pool according to the migration priority in the instruction, write the data block into a corresponding SSD flash memory according to the target level identifier in the instruction, and realize zero-copy migration of the data block through a DMA engine; and a resource coordination module, configured to monitor PCIe link utilization and SSD controller queue depth of the migration execution module in real time, dynamically adjust concurrency and data block granularity of a migration task according to a system load threshold and a migration priority, and feed back to the state sensing module to adjust sampling frequency and feature dimension of the I / O feature sequence.
[0006] Further, the specific process of establishing multiple protocol parallel channels by redefining reserved pins of the U.2 interface is as follows: identifying a reserved signal pin group in the U.2 interface preset for a SAS protocol, redefining a first group of SAS signal pins as a PCIe x1 link to establish an independent data transmission channel with an M.2 NVMe SSD, redefining a second group of SAS signal pins as a SATA x1 link to establish an independent data transmission channel with an M.2 SATA SSD, and reserving a native PCIe x4 link in the U.2 interface as a high-speed data channel with a U.2 NVMe SSD; configuring independent device presence detection signals for each redefined link, and identifying insertion states of each SSD through level state combination of detection pins.
[0007] Further, the specific process of fusing the signal integrity parameters of each channel with the I / O access characteristics of the device to generate the I / O characteristic sequence of the physical layer state information is as follows: through the signal quality monitoring unit built in the interface controller, the signal integrity parameters of each data channel are periodically collected, including the signal error rate, timing jitter amplitude and eye opening degree, a mapping model of signal integrity parameters to link health degree is established, the original signal parameters are converted into standardized link quality scores; the I / O requests passing through each channel are captured in parallel, the logical block address, operation type, data size and timestamp information in the request are extracted, the link quality scores are timestamped and aligned with the real-time I / O characteristics, and a hybrid feature vector is formed; the hybrid feature vector is serialized and normalized through a sliding time window to generate the I / O characteristic sequence of the physical layer state information.
[0008] Further, the specific process of receiving the I / O characteristic sequence of the physical layer state information, extracting the multi-scale features of the access pattern through the time convolution neural network, and calculating the access heat probability of each data block is as follows: the input I / O characteristic sequence is preprocessed in time dimension, the access pattern features of different time scales are extracted through the time convolution neural network with causal expansion convolution; the multi-head self-attention mechanism is introduced at each level of the convolution network, and the key time steps and feature channels in the feature map are adaptively weighted; the feature maps of different levels are connected and fused across layers, the high-dimensional features after fusion are mapped to the access heat probability values of each data block in the current time window through the full connection layer, the heat probability sequence is exponentially weighted and smoothed based on the sliding time window mechanism, and the influence of instantaneous fluctuations on the prediction stability is reduced.
[0009] Further, the specific process of determining the target storage level of each data block according to the access heat probability high-low mapping, thereby generating the data migration scheduling instruction containing the source physical address, target level identification and migration priority, is as follows: a dynamic adjustment heat probability interval division mechanism is established, the access heat probability of the data block is mapped to the corresponding storage level, wherein the high heat interval corresponds to the U.2 NVMe SSD, the medium heat interval corresponds to the M.2 NVMe SSD, and the low heat interval corresponds to the M.2 SATA SSD; the residual capacity distribution of each storage level is monitored in real time; when the residual capacity of the target level is lower than the set threshold, the boundary value of the heat probability interval is dynamically adjusted, and part of the data block is redistributed to the available space of the adjacent level; according to the determined target storage level, a structured migration scheduling instruction is constructed, the instruction contains the source physical address of the data block, the target level identification, and the migration priority based on the heat probability change rate and the data block size; the generated scheduling instruction is verified for resource availability to ensure that the target level has sufficient receiving capacity and performance margin; at the same time, the conflict detection of the executing migration task is performed to avoid address overlap and resource contention.
[0010] Further, the migration execution module includes the following steps: according to the migration priority, the isolated buffer area with different access permissions is divided in the shared DRAM pool, wherein the high-priority migration task is allocated an exclusive buffer area, and the low-priority migration task uses a shared buffer area; a direct reading channel from the source SSD flash memory to the shared DRAM pool buffer area is established through the DMA engine, and the data block is transmitted from the source SSD flash memory to the allocated isolated buffer area; after the data transmission is completed, a direct writing channel from the shared DRAM pool buffer area to the target SSD flash memory is established through the DMA engine, and the data block is written from the buffer area to the specified physical address of the target SSD; in the data transmission process, the transmission data is checked in real time through the cyclic redundancy check mechanism, the check code is generated in the source SSD reading stage, and the check code is verified in the target SSD writing stage.
[0011] Further, the PCIe link utilization rate and the SSD controller queue depth of the migration execution module are monitored in real time, and the specific process of dynamically adjusting the concurrency and data block granularity of the migration task according to the system load threshold and the migration priority is as follows: the bandwidth utilization rate of each PCIe link and the command queue depth of each SSD controller are collected in real time through the performance counter, and the current comprehensive load index of the system is calculated; a load evaluation function based on a sliding window is established, the collected load data is smoothed and trended, when the system load exceeds the threshold, the migration task is scheduled according to the migration priority, the number of concurrent executions of the low-priority migration task is reduced, and the data block granularity is reduced to reduce the impact of single migration on system performance.
[0012] Further, the specific process of adjusting the sampling frequency and feature dimension of the I / O feature sequence is as follows: a mapping relationship between the system load state and the sampling parameter is established, the sampling frequency of the I / O feature sequence is dynamically adjusted according to the load level, the sampling frequency is reduced in high load, and the sampling frequency is increased in low load; the feature dimension combination of the I / O feature sequence is configured based on the system load, the core feature dimension is retained in high load, the extended feature dimension is enabled in low load, and the configuration parameter is real-time issued to the state sensing module.
[0013] A server intelligent integrated layered storage method based on a U.2 interface, comprising the following steps: S1. A plurality of protocol parallel channels are established by redefining the reserved pins of the U.2 interface, the signal integrity parameters of each channel are monitored in real time, the signal integrity parameters are fused with the device I / O access features, and the I / O feature sequence of the physical layer state information is generated; S2. The I / O feature sequence of the physical layer state information is received, the multi-scale features of the access mode are extracted through a timing convolutional neural network, the access heat probability of each data block is calculated, and the target storage level of each data block is determined according to the access heat probability, so as to generate a data migration scheduling instruction containing a source physical address, a target level identifier and a migration priority; S3. The data migration scheduling instruction is received, the data block is read from the corresponding SSD flash memory according to the source physical address in the instruction, the corresponding level of the isolated buffer area is allocated in the shared DRAM pool according to the migration priority in the instruction, the data block is written into the corresponding SSD flash memory according to the target level identifier in the instruction, and the zero-copy migration of the data block is realized through the DMA engine; S4. The PCIe link utilization rate and the SSD controller queue depth of the migration execution module are monitored in real time, the concurrency degree and the data block granularity of the migration task are dynamically adjusted according to the system load threshold and the migration priority, and the state sensing module is fed back to adjust the sampling frequency and the feature dimension of the I / O feature sequence.
[0014] The application has the following beneficial effects:
[0015] (1) An intelligent integrated layered storage system of a server based on a U.2 interface, the reserved pins of the U.2 interface are redefined by a state sensing module, the mutual exclusion limitation of the existing U.2 interface in multi-protocol support is broken, the concurrent access and identification of multiple different protocol SSDs by a single physical port are realized, and the storage density and economy of the server are effectively improved. Meanwhile, the I / O feature sequence generated by combining the real-time signal integrity parameter and the device I / O access feature, the multi-scale feature extraction of the timing convolutional neural network by the heat prediction module can more accurately predict the data access heat, automatically complete the mapping of the data block and the target storage level, avoid the resource waste caused by the traditional layered storage relying on fixed rules or artificial strategies, and thus reduce the total cost of ownership while ensuring the access performance.
[0016] (2) A server intelligent integrated hierarchical storage method based on a U.2 interface, which realizes zero-copy migration of data blocks through a shared DRAM pool and a DMA engine, avoids the occupation of host CPU and memory resources by the traditional software scheduling-based migration mode, reduces migration delay and system overhead, and ensures the rapid flow of data between different levels of storage devices. At the same time, the utilization rate of the PCIe link and the queue depth of the SSD controller are sensed in real time, the migration concurrency and data block granularity are dynamically adjusted, and feedback is fed back to the state sensing module to form a closed loop optimization, so that the system can still maintain the data integrity and consistency of the migration process under high load scenarios, and the adaptive optimization allocation of storage system resources is realized.
[0017] Of course, implementing any product of the present application does not necessarily require all the advantages described above to be achieved at the same time. BRIEF DESCRIPTION OF DRAWINGS
[0018] Figure 1 A flowchart of a server intelligent integrated hierarchical storage system based on a U.2 interface of the present application.
[0019] Figure 2 A flowchart of a server intelligent integrated hierarchical storage method based on a U.2 interface of the present application. DETAILED DESCRIPTION
[0020] The embodiments of the present application solve the problems of poor interface compatibility, rigid level division, and low data migration efficiency in existing hierarchical storage through a server intelligent integrated hierarchical storage system and method based on a U.2 interface.
[0021] The scheme in the embodiments of the present application has the following overall idea:
[0022] First, the state sensing module redefines the U.2 interface reserved pins, establishes a multi-protocol parallel channel, realizes concurrent access of different protocol SSDs, and generates physical layer state information in combination with signal integrity parameters and I / O characteristics; second, the heat prediction module performs deep learning analysis on the I / O characteristic sequence, obtains data block access heat probability, and automatically completes level mapping; third, the migration execution module realizes zero-copy migration of data blocks based on a shared DRAM pool and a DMA engine, reduces resource consumption during migration; and finally, the resource coordination module dynamically adjusts the migration concurrency and data block granularity according to the system load, and forms a feedback closed loop to the state sensing module to realize adaptive optimization of the overall system.
[0023] Please refer to Figure 1The embodiment of the present application provides a technical scheme: a server intelligent integrated layered storage system based on a U.2 interface, comprising the following modules: a state sensing module, configured to establish a multi-protocol parallel channel by redefining a reserved pin of the U.2 interface, monitor signal integrity parameters of each channel in real time, fuse and process the signal integrity parameters and device I / O access characteristics, and generate an I / O characteristic sequence of physical layer state information; a heat prediction module, configured to receive the I / O characteristic sequence of the physical layer state information, extract multi-scale features of an access mode through a time sequence convolutional neural network, calculate access heat probability of each data block, and determine a target storage level of each data block according to the access heat probability, so as to generate a data migration scheduling instruction containing a source physical address, a target level identifier and a migration priority; a migration execution module, configured to receive the data migration scheduling instruction, read a data block from a corresponding SSD flash memory according to the source physical address in the instruction, allocate an isolated buffer area of a corresponding level in a shared DRAM pool according to the migration priority in the instruction, write the data block into the corresponding SSD flash memory according to the target level identifier in the instruction, and realize zero-copy migration of the data block through a DMA engine; and a resource coordination module, configured to monitor a PCIe link utilization rate and an SSD controller queue depth of the migration execution module in real time, dynamically adjust a concurrency degree and a data block granularity of a migration task according to a system load threshold and a migration priority, and feed back to the state sensing module to adjust a sampling frequency and a feature dimension of the I / O characteristic sequence.
[0024] In this embodiment, the system consists of four modules that work together: the data flow of each module is as follows: the state-aware module collects physical signals and I / O information and outputs the physical layer state I / O feature sequence → the hotness prediction module predicts the data block hotness based on the sequence and outputs the migration scheduling instruction → the migration execution module performs zero-copy migration between the shared DRAM and each SSD according to the instruction → the resource coordination module monitors the migration running state and adjusts the migration parameters, and then feeds back the adjustment results to the state-aware module to adjust the sampling / feature strategy, forming a closed-loop optimization. The state-aware module: combines the running state of the physical link layer and the I / O access behavior of the upper layer into a feature sequence that can be used for machine learning, providing input with physical awareness for subsequent hotness prediction. Main inputs: physical signals on the U.2 interface channel (such as differential pair waveform, clock / synchronization information, error count, etc.) and I / O request stream from the host (including logical block address, operation type, data size, timestamp, etc.). On the redefined U.2 channel, the signal integrity indicators are sampled and recorded in real time (see the term description), the device presence detection signal is periodically read and verified to confirm the online state of each SSD; the metadata of the I / O request (LBA, read / write, length, timestamp) is captured in parallel and preprocessed (denoising, standardization); the physical signal indicators are mapped to link health score (e.g. bit error rate, jitter, eye diagram, etc. are converted to a unified quality score), and are aligned with the I / O metadata by timestamp, and are spliced into a mixed feature vector at each time step; the continuous mixed feature vector is serialized using a sliding time window (generating a time series), and is normalized / denoised to output the physical layer state I / O feature sequence (which is the input of the subsequent ML model). Output: physical layer state I / O feature sequence (time series format, each time step contains link quality score and corresponding I / O metadata). U.2 interface: a standard 2.5 inch SSD physical / electrical interface that can carry PCIe / NVMe or SATA / SAS protocols, etc. Signal integrity parameters: such as bit error rate, timing jitter, eye opening, etc., reflecting the quality of high-speed differential signals. LBA (logical block address): address identifier in block storage. Sliding time window: a common method for segmenting time series for time series analysis. Hotness prediction module: based on the time series output by the state-aware module, identifies the "hotness" (probability / intensity of access) of each data block, and decides which storage level (hot / warm / cold) the data block should be migrated to, while generating executable migration scheduling instructions. Main input: physical layer state I / O feature sequence (time series, each step is a mixed feature vector).The input sequence is time-domain standardized and missing value processed to ensure the consistency of the input of the time series model. A time series convolutional neural network (TCN) is used for multi-scale feature extraction. The TCN captures short-term / medium-term / long-term access patterns at different time scales through causal and dilated convolution. A multi-head self-attention mechanism is introduced on the feature map extracted by convolution to weight the key signals from different time steps or channels, enhancing the model's response to sudden hotspots or important signals. The fused high-dimensional features are mapped to the access heat probability (classification or regression form) of each data block, and the heat probability sequence is smoothed (e.g., exponentially weighted average) to reduce the misjudgment of transient fluctuations. According to the heat probability and the current available capacity / performance status of each level, the target level is determined, and the data migration scheduling instruction is generated. The instruction structure includes: source physical address, target level identifier, preliminary migration priority, and instruction generation timestamp, etc. Before the instruction is issued, resource availability check (to ensure that the target level space / IOPS has a margin) and conflict detection (to avoid address repeated migration) can be performed. Output: structured data migration scheduling instruction set (which can be directly parsed and executed by the migration execution module). Time series convolutional neural network (TCN): a neural network structure for sequence modeling, with causal convolution and dilated convolution to capture long-term dependencies. Self-attention mechanism: a mechanism for assigning learnable weights to different positions in a sequence, highlighting important time steps or channels. Exponential smoothing: a smoothing method that gives higher weights to recent observations, reducing the impact of random fluctuations. Migration execution module: according to the scheduling instruction issued by the heat prediction module, it efficiently and safely completes the data block cross-level migration on the physical device, minimizing host intervention and additional copying. Main input: migration scheduling instruction (including source physical address, target level, migration priority, etc.). According to the source physical address in the instruction, locate the source SSD (U.2 or some M.2 within the module) and request to read; allocate isolated buffer in shared DRAM pool according to migration priority: high priority tasks can be allocated exclusive area or higher QoS buffer, low priority tasks share buffer to save resources; configure DMA (Direct Memory Access) engine to establish a direct read path from source SSD to shared DRAM buffer, complete zero-copy reading of data blocks (zero-copy: data is directly transmitted between devices and DRAM, avoiding intermediate copying by CPU); after the buffer is ready, configure the DMA write path to write data from the buffer directly to the physical location specified by the target SSD; perform data integrity check (e.g., CRC check) during transmission, generate check code in the read stage and verify in the write stage; complete the write and atomically update the related metadata (such as FTL mapping table), and return the migration completion / failure status to the upper management. Output: migration completion / failure status, migration time consumption / occupied bandwidth, etc. runtime metrics (for resource coordination module to collect).DMA (Direct Memory Access): a mechanism that allows peripherals to exchange data with memory directly without CPU involvement. Shared DRAM pool: a piece of physical DRAM within the module, logically divided into multiple virtually isolated regions by the controller for different purposes (FTL mapping, cache, migration channel). FTL (Flash Translation Layer): a management layer used by SSD controllers to map logical addresses to physical flash pages. Zero-copy: a way of transferring data without multiple copies through CPU buffers, reducing latency and CPU burden. Resource coordination module (runtime monitoring and adaptive adjustment): during migration execution, continuously monitor system internal resource utilization (e.g. PCIe link utilization, SSD controller queue depth), and based on these runtime indicators and migration priority, adjust migration parameters (concurrency, single migration block size) in real time to dynamically balance migration efficiency and business I / O response; at the same time, feedback the adjustment results to the state perception module to optimize subsequent sampling / feature strategies. Main inputs: real-time running indicators from the migration execution module (PCIe bandwidth utilization, SSD controller command queue depth, migration task number, current block size, etc.); also receives system preset load threshold and priority strategy. Periodically sample PCIe link bandwidth utilization and SSD queue depth through performance counters or controller registers, and calculate short-term moving average and trend indicators; compare the sampling results with the preset threshold (and use hysteresis / interval to avoid frequent jitter), and decide whether to adjust based on the priority of each migration task: if the link is close to saturation or the queue is significantly increased → reduce migration concurrency or reduce single migration block size to reduce instantaneous load; if the link is idle and the queue is shallow → appropriately increase concurrency or block size to increase migration throughput; for high-priority tasks, you can use relaxed restrictions or priority tokens to ensure their execution. Adjust the migration parameters (e.g. concurrency task number C, block size B) and record the observed values after adjustment; feedback the above records or parameter summaries to the state perception module, which can adaptively adjust the sampling frequency (reduce sampling frequency to reduce additional monitoring overhead under high load, increase sampling to capture details under low load) and feature dimension (only retain core features under high load, enable extended features under low load) when generating the next round of I / O feature sequence. Output: adjusted migration parameters and feedback vectors / summaries for state perception module. Performance counter: a hardware or firmware provided counter / measurement register used to count bandwidth, error frames, queue depth, etc. Queue depth: the number of commands queued for execution in the SSD controller, reflecting instantaneous load and response pressure. Concurrency: the number of concurrent migration tasks; block granularity refers to the size of data transferred in each migration (affects the number of I / O requests and single bandwidth occupancy).
[0025] Specifically, the specific process of establishing a multi-protocol parallel channel by redefining the reserved pins of the U.2 interface is as follows: identifying a group of reserved signal pins in the U.2 interface originally preset for the SAS protocol, redefining a first group of SAS signal pins as a PCIex1 link to establish an independent data transmission channel with the M.2 NVMe SSD, redefining a second group of SAS signal pins as a SATAx1 link to establish an independent data transmission channel with the M.2 SATA SSD, and reserving the original PCIe x4 link of the U.2 interface as a high-speed data channel with the U.2 NVMe SSD; configuring independent device presence detection signals for each redefined link, and identifying the insertion state of each SSD by detecting the level state combination of the pins.
[0026] In the present embodiment, first, a group of signal pins in the U.2 interface originally preset for the SAS (Serial Attached SCSI, a serial connection SCSI storage protocol) is identified. In the present embodiment, these pins are not actually used for the SAS protocol, and thus can be redefined as reserved resources. A first group of SAS signal pins is redefined as a PCIe (Peripheral Component Interconnect Express) x1 link, i.e., a single-channel PCIe high-speed data transmission channel, to establish an independent data path with the M.2 NVMe SSD (a solid-state disk supporting the PCIe-based Non-Volatile Memory Standard). A second group of SAS signal pins is redefined as a SATA (Serial ATA, Serial Advanced Technology Attachment) x1 link to establish an independent data path with the M.2 SATA SSD (a solid-state disk based on the SATA protocol). At the same time, the original PCIe x4 link of the U.2 interface itself is reserved as a high-speed data transmission channel between the U.2 NVMe SSD. To ensure the manageability and device recognition capability of each redefined link, the present embodiment further configures an independent device presence detection signal for each link (i.e., confirming whether a device is inserted by detecting the level state of a specific pin), and distinguishes the insertion state of different types of SSDs by level combination mode. In this way, the system can achieve plug-and-play management of different protocol SSDs without an additional external controller.
[0027] Specifically, the specific process of monitoring the signal integrity parameters of each channel in real time, fusing the signal integrity parameters with the device I / O access features, and generating the I / O feature sequence of the physical layer state information is as follows: through the signal quality monitoring unit built in the interface controller, the signal integrity parameters of each data channel are periodically collected, including the signal error rate, the timing jitter amplitude and the eye opening degree, a mapping model of the signal integrity parameters to the link health degree is established, the original signal parameters are converted into standardized link quality scores; the I / O requests transmitted through each channel are captured in parallel, the logical block address, the operation type, the data size and the timestamp information in the request are extracted, the link quality scores are timestamped and aligned with the real-time I / O features, and a hybrid feature vector is formed; through a sliding time window, the hybrid feature vector is serialized, organized and normalized to generate the I / O feature sequence of the physical layer state information.
[0028] In the embodiment, first, through the signal quality monitoring unit built in the interface controller, the signal integrity parameters of each data channel are periodically collected, including: the error rate parameter (denoted as ), indicating the ratio of the number of error bits to the total number of bits in the transmitted data at the kth sampling; the jitter amplitude parameter (denoted as ), indicating the offset range of the clock edge position at the kth sampling; the eye opening degree parameter (denoted as ), indicating the stable voltage range of the signal waveform at the kth sampling. Subsequently, a mapping model of the signal integrity parameters to the link health degree is established. Specifically, the link quality score at the kth sampling is calculated , and the formula is as follows: ; wherein: : the link quality score at the kth sampling; : the error rate parameter at the kth sampling; : the jitter amplitude parameter at the kth sampling; : the eye opening degree parameter at the kth sampling; : the reference maximum eye opening degree; : the weighting coefficient, used to balance the contribution of different indicators to the link quality. The determination method of the weighting coefficient is as follows: in the system factory calibration stage, based on multiple groups of actual measurement data, the least squares regression is used to calculate the weighting coefficient by minimizing the error between the predicted score and the actual link failure probability. While collecting the link quality parameters, the I / O requests transmitted through each channel are captured in parallel, and the feature information thereof is extracted, including: the logical block address (denoted as ), indicating the logical storage position corresponding to the I / O request; the operation type (denoted as ), using a 0 / 1 variable to distinguish read / write operations; the data size (denoted as ), represents the amount of data transmitted in this request; the timestamp (denoted as ), records the time point when the request arrives. Subsequently, the link quality score is time-aligned and feature-spliced with the corresponding I / O features to obtain the kth mixed feature vector: ; then the mixed feature vector sequence is organized through a sliding time window. Let the window length be L, the feature sequence in the mth window is denoted as: ; finally, each dimension feature in the sequence is normalized to fall into a unified numerical interval, forming the standardized physical layer state information I / O feature sequence as the input of the subsequent hotness prediction module.
[0029] Specifically, the specific process of receiving the I / O feature sequence of the physical layer state information, extracting the multi-scale features of the access pattern through the time convolutional neural network, and calculating the access hotness probability of each data block is as follows: the input I / O feature sequence is subjected to time dimension standardization preprocessing, and different time scale access pattern features are extracted through the time convolutional neural network with causal dilated convolution; the multi-head self-attention mechanism is introduced at each level of the convolutional network to adaptively allocate weights to the key time steps and feature channels in the feature map; the feature maps at different levels are connected and fused across layers, and the high-dimensional fused features are mapped to the access hotness probability value of each data block in the current time window through the full connection layer, and the hotness probability sequence is subjected to exponential weighted smoothing processing based on the sliding time window mechanism to reduce the influence of instantaneous fluctuations on the prediction stability.
[0030] In the embodiment, specifically, the process of receiving the I / O feature sequence of the physical layer state information, extracting the multi-scale features of the access pattern through the time convolutional neural network, and calculating the access hotness probability of each data block is as follows: the input I / O feature sequence is subjected to time dimension standardization processing, each feature dimension is subtracted by the mean value and divided by the standard deviation to reduce the influence of dimensional difference on network training. Standardization can ensure that the response of the convolutional network on different feature dimensions is balanced, and improve the training stability. Time convolutional neural network feature extraction, multi-scale access pattern features are extracted using the time convolutional neural network (TCN) with causal dilated convolution. For the layer convolutional network, the th output feature channel is denoted as: ; wherein: : the output feature of the th channel of the th layer at the time step ; : the input sequence at the time step eigenvalues; : Kernel length, i.e., the number of historical time steps considered in each convolution; Dilation factor: controls the expansion of the receptive field; Convolution weights; Convolution bias; Activation functions, such as ReLU, are used to introduce non-linearity. Weight determination methods: Network weights. The algorithm is optimized using training data, employing backpropagation and a loss function that minimizes prediction error. A multi-head self-attention mechanism is introduced at each layer of the convolutional network, adaptively weighting the feature maps at different time steps and feature channels. For each time step... and feature channels Calculate the attention-weighted features: ;in: Attention-weighted features; : Convolutional layer Channel at time step The output characteristics; : No. Channel 1 Time step corresponds to the first The attention weights at each time step are calculated using the softmax function: ; Attention scoring function, typically calculated using dot product scoring or a small feedforward network; weights are determined automatically through network training, with the training objective being to minimize the error in predicting popularity probability. Cross-layer feature fusion involves connecting and fusing feature maps from different layers to obtain fused high-dimensional features. : ;in: Total number of convolutional layers; Feature channel splicing operation; Time step The fused feature vector. The high-dimensional feature mapping is converted into heat probability, which will affect the fused features. The input fully connected layer is mapped to the access frequency probability of each data block. : ;in: Time step The probability of data block access popularity; : Weight matrix of fully connected layers; : Bias vector; : Activation function, mapping the output to the range [0,1]; weights With bias Optimized using training data. Exponentially weighted smoothing is applied to the heat probability sequence. Perform exponentially weighted smoothing using a sliding window to reduce the impact of instantaneous fluctuations: ;in: : Smoothed heat probability; : Smoothing coefficient, with a value range of [0,1]; Smoothing coefficient determination: set according to system experience, usually adjusted according to I / O access volatility, so that short-term fluctuations do not significantly affect the stability of prediction.
[0031] Specifically, the process of determining the target storage level for each data block based on the access popularity probability, and generating data migration scheduling instructions containing the source physical address, target level identifier, and migration priority, is as follows: A dynamically adjusted popularity probability interval division mechanism is established to map the access popularity probability of data blocks to the corresponding storage level. The high popularity interval corresponds to U.2 NVMe SSD, the medium popularity interval to M.2 NVMe SSD, and the low popularity interval to M.2 SATA SSD. The remaining capacity distribution of each storage level is monitored in real time. When the remaining capacity of the target level is lower than a set threshold, the boundary values of the popularity probability interval are dynamically adjusted, and some data blocks are reallocated to the available space of adjacent levels. Based on the determined target storage level, a structured migration scheduling instruction is constructed. The instruction includes the source physical address of the data block, the target level identifier, and the migration priority based on the popularity probability change rate and data block size. The generated scheduling instruction undergoes resource availability verification to ensure that the target level has sufficient receiving capacity and performance margin. Simultaneously, conflict detection is performed on the ongoing migration tasks to avoid address overlap and resource contention.
[0032] In this implementation plan, a dynamically adjustable heat probability interval is established. , data block Access popularity probability Mapped to the corresponding storage level: ;in: : data block Target storage tier identifier; : data block The probability of being accessed based on popularity; High-intensity zone, used for mapping to U.2 NVMe SSD; Medium heat range, used for mapping to M.2 NVMe SSD; Low-heat zone, used for mapping to M.2 SATA SSDs; zone dynamic adjustment: by monitoring the remaining capacity of each storage tier in real time, when the remaining capacity falls below a threshold... Adjust as appropriate The boundary ensures a balanced distribution of heat across the available space. Threshold determination: according to the storage layer capacity statistics, generally take 10%-15% of the remaining capacity as the trigger threshold. Migration priority calculation for each data block , according to the heat probability change rate And the size of the data block Calculate the migration priority : ; Where: : the migration priority of the data block , the larger the value, the higher the priority of migration; : heat probability change rate, , the last predicted value; : the size of the data block ; : weight coefficient, used to adjust the influence of heat change and data block size on migration priority; weight coefficient determination: can be set according to historical access fluctuation and system I / O performance statistical experience value, for example, according to the average heat change value and the average data block size ratio for normalization. Generate structured migration scheduling instructions according to the determined target storage hierarchy And migration priority , generate migration scheduling instructions : ; Where: : the migration scheduling instruction of the data block ; : the source physical address of the data block ; : target storage hierarchy identifier; : migration priority. Before generating the instruction, the system will verify the availability of the target level capacity and performance margin to ensure that the instruction execution will not exceed the storage capacity. For the migration tasks being executed, address conflict detection is also required to avoid two migration tasks pointing to the same physical space or competing for the same buffer resources. Dynamic adjustment and conflict control through real-time monitoring of the remaining capacity of each level and the depth of the migration task queue, dynamically adjusting the heat interval boundary and migration priority calculation strategy, to ensure the balanced distribution of migration tasks and prevent performance bottlenecks or resource contention. Adjustment of the heat interval boundary , can be as follows: ; ; : U.2 NVMe SSD and M.2 SATA SSD current remaining capacity; : adjustment coefficient, public determination: according to the total remaining capacity ratio, such as 0.05~0.1; when the remaining capacity of a layer is lower than the threshold, the corresponding interval boundary is contracted or expanded, and the data block is dynamically migrated to the available space.
[0033] Specifically, the migration execution module includes the following steps: dividing isolated buffers with different access permissions in the shared DRAM pool according to migration priorities, wherein high-priority migration tasks are allocated exclusive buffers, and low-priority migration tasks use shared buffers; establishing a direct read channel from the source SSD flash memory to the shared DRAM pool buffer through the DMA engine, and transferring data blocks from the source SSD flash memory to the allocated isolated buffer; after the data transmission is completed, establishing a direct write channel from the shared DRAM pool buffer to the target SSD flash memory through the DMA engine, and writing data blocks from the buffer to the specified physical address of the target SSD; during the data transmission process, real-time verification of the transmitted data is performed through a cyclic redundancy check mechanism, a check code is generated in the source SSD reading stage, and the check code is verified in the target SSD writing stage.
[0034] In the embodiment, the isolated buffer is divided: the system divides the isolated buffer in the shared DRAM pool according to the priority of the migration task. High-priority migration tasks are allocated exclusive buffers to ensure that their bandwidth and response time during data transmission are not disturbed; low-priority migration tasks use shared buffers to allow multiple tasks to access concurrently while ensuring overall resource utilization. In this way, dynamic resource isolation and priority scheduling of migration tasks are achieved, and the timeliness of high-heat data block migration and the overall efficiency of the system are improved. Data is read from the source SSD to the buffer: a direct memory access channel is established using the DMA engine to directly transfer data blocks from the source SSD flash memory to the allocated DRAM isolated buffer. The DMA mechanism can bypass the CPU for high-speed transmission, reducing CPU load and data duplication overhead, and achieving zero-copy efficient migration. This can improve the data block migration speed while ensuring data integrity. Data is written from the buffer to the target SSD: after the data blocks are ready in the buffer, the DMA engine establishes a direct write channel from the DRAM buffer to the target SSD to write data to the specified physical address. In this way, data blocks can be accurately migrated according to the target level and physical location specified by the scheduling instructions, achieving intelligent hierarchical storage management. Data transmission verification: during the data transmission process, real-time verification of the data is performed through a cyclic redundancy check (CRC) mechanism. A check code is generated in the source SSD reading stage, and the check code is verified in the target SSD writing stage to ensure that no errors or damage occur to the data during the migration process. This step ensures the reliability of the migration, prevents data loss or damage during transmission, and thus ensures the data integrity and security of the storage system.
[0035] Specifically, the process of dynamically adjusting the concurrency and data block granularity of migration tasks based on system load thresholds and migration priorities by real-time monitoring of PCIe link utilization and SSD controller queue depth of the migration execution module is as follows: The bandwidth utilization of each PCIe link and the command queue depth of each SSD controller are collected in real time using performance counters to calculate the current comprehensive load index of the system; a load evaluation function based on a sliding window is established to smooth and predict the trend of the collected load data; when the system load exceeds the threshold, migration tasks are scheduled in stages according to migration priorities to reduce the number of concurrent executions of low-priority migration tasks and decrease the data block granularity to reduce the impact of a single migration on system performance.
[0036] In this implementation scheme, the performance acquisition and load metric calculation system collects the bandwidth utilization of each PCIe link in real time through a built-in performance counter. and the command queue depth of each SSD controller ,in Indicates the link number. This indicates the controller number. Based on this collected data, the current overall load index of the system is calculated. : ;in: The overall system load metric measures the overall pressure exerted on the system by the current migration task. : No. Bandwidth utilization of each PCIe link; : No. Command queue depth of each SSD controller; PCIe link weight coefficient, used to reflect the importance of different links. The determination method can be based on the link bandwidth ratio initialization and dynamically adjusted according to the operating status. : SSD controller weight coefficient, used to reflect the degree of impact of the controller on system performance, can be set according to the historical distribution of controller queue length; Total number of PCIe links; Total number of SSD controllers. Explanation: This is a comprehensive load metric obtained by combining link bandwidth utilization and controller queue depth using a weighted average method. This can reflect the real-time stress status of the entire migration system. Load smoothing and trend prediction, to reduce the impact of instantaneous fluctuations on decision-making, employ a sliding window mechanism to adjust load indicators. Perform smoothing and trend forecasting: ;in: : Smoothed system load metrics; : The real-time load metrics currently being collected; : Smoothed load metrics for the previous time window; The smoothing coefficient controls the impact of historical data on current load forecasting. It is determined empirically by initializing it to a range of 0.3 to 0.5 and can be dynamically adjusted based on system response time. Explanation: Smoothed load metrics reflect short-term system trends, preventing instability in migration tasks caused by instantaneous fluctuations. Dynamically adjusting migration task concurrency and data block granularity based on smoothed load metrics... Exceeding the set system load threshold At that time, the system will determine the priority of the migration task. Migration tasks are scheduled in a tiered manner: high-priority tasks maintain high concurrency and larger data block granularity; low-priority tasks reduce concurrent execution and divide data blocks into smaller granularities to minimize the instantaneous impact on system performance. Explanation: This dynamic adjustment strategy enables adaptive control of system load, ensuring the timeliness of high-priority data migration while preventing low-priority migration tasks from causing PCIe link congestion or SSD controller queue accumulation. Resource feedback and loop optimization adjustment results are fed back to the state-aware module to adjust the sampling frequency and feature dimension of the / / O feature sequence, thus forming a closed-loop control mechanism to achieve dynamic matching between migration tasks and system resources.
[0037] Specifically, the process of adjusting the sampling frequency and feature dimensions of the I / O feature sequence is as follows: establish a mapping relationship between system load status and sampling parameters, dynamically adjust the sampling frequency of the I / O feature sequence according to the load level, reduce the sampling frequency under high load and increase the sampling frequency under low load; configure the feature dimension combination of the I / O feature sequence based on system load characteristics, retain the core feature dimensions under high load and enable the extended feature dimensions under low load, and send the configuration parameters to the status awareness module in real time.
[0038] In this embodiment, the sampling frequency is dynamically adjusted: the system first establishes a mapping relationship between the system load state and the I / O sampling parameters, that is, according to the current system load level to determine the sampling frequency. When the system load is high, in order to reduce the performance overhead and the interference to the migration task, the sampling frequency of the I / O characteristic sequence is reduced, thereby reducing the real-time monitoring pressure of the interface controller and the storage device; when the system load is low, the sampling frequency is increased to obtain more detailed I / O behavior information, which facilitates the heat prediction module and the migration scheduling module to make more accurate decisions. The feature dimension is adaptively configured: the system selects the feature dimension combination to be retained or enabled in the I / O characteristic sequence according to the load characteristics. In the high load state, only the core feature dimensions (such as link quality score, key I / O operation type, hot spot logical block address, etc.) are retained, reducing the data processing amount and transmission overhead; in the low load state, the extended feature dimensions (such as time-related features, multi-level cache hit rate, queue waiting time, etc.) are enabled to obtain more rich I / O state information, thereby improving the accuracy of heat prediction and migration decision. The parameters are real-time issued and closed-loop controlled: the adjusted sampling frequency and feature dimension configuration parameters are real-time issued to the state perception module, so that the state perception module can collect I / O features according to the new sampling strategy to form a closed-loop control. This mechanism ensures the dynamic balance between I / O monitoring and system resource usage, realizing the adaptive optimization of system stability under high load and data monitoring accuracy under low load.
[0039] Referring to Figure 2 A server intelligent integrated layered storage method based on a U.2 interface, comprising the following steps: S1. A plurality of protocol parallel channels are established by redefining the reserved pins of the U.2 interface, the signal integrity parameters of each channel are monitored in real time, the signal integrity parameters are fused with the device I / O access characteristics, and an I / O characteristic sequence of physical layer state information is generated; S2. The I / O characteristic sequence of physical layer state information is received, multi-scale features of the access mode are extracted through a time sequence convolutional neural network, the access heat probability of each data block is calculated, and the target storage level of each data block is determined according to the access heat probability, so as to generate a data migration scheduling instruction containing a source physical address, a target level identifier and a migration priority; S3. The data migration scheduling instruction is received, the data block is read from the corresponding SSD flash memory according to the source physical address in the instruction, the corresponding level of the isolated buffer area is allocated in the shared DRAM pool according to the migration priority in the instruction, the data block is written into the corresponding SSD flash memory according to the target level identifier in the instruction, and the zero-copy migration of the data block is realized through the DMA engine; S4. The PCIe link utilization rate and the SSD controller queue depth of the migration execution module are monitored in real time, the concurrency and the data block granularity of the migration task are dynamically adjusted according to the system load threshold and the migration priority, and the sampling frequency and the feature dimension of the I / O characteristic sequence are adjusted and fed back to the state perception module.
[0040] In this embodiment, step S1: by redefining the reserved pins of U.2 interface, the signal pins originally used for SAS protocol are re-assigned as PCIe x1 link or SATA x1 link to establish independent data transmission channels with M.2 NVMe SSD and M.2 SATA SSD respectively, while the original PCIe x4 link of U.2 interface is reserved for high-speed NVMe SSD communication. Through independent device detection signals, the insertion state of each SSD can be identified, thereby supporting multiple types of SSDs to work simultaneously. By using the signal quality monitoring unit built-in the interface controller, the signal integrity parameters (such as bit error rate, timing jitter amplitude and eye opening) of each channel are periodically collected and standardized as link health score. At the same time, the I / O requests passing through each channel are captured in parallel, and information such as logical block address, operation type, data size and timestamp is extracted. The link health score is aligned and spliced with the I / O features to form a mixed feature vector, and through sliding window serialization and normalization processing, an I / O feature sequence of physical layer state information is generated. A unified description of the real-time physical layer state and I / O behavior of the system is provided, which provides basic data for subsequent heat prediction and migration decision. Step S2: After time standardization processing of the I / O feature sequence, the time series convolutional neural network with causal dilated convolution is used to extract access pattern features of different time scales, which can capture short-term and long-term access trends. The multi-head self-attention mechanism is introduced at each layer of the convolutional network to adaptively weight the key time steps and feature channels, enhancing the expression ability of important information. The feature maps of different levels are connected and fused across layers, mapped to the access heat probability of each data block through the fully connected layer, and the probability sequence is exponentially weighted and smoothed through the sliding window to reduce the influence of instantaneous fluctuations. According to the heat probability, the storage hierarchy is dynamically divided, with high-heat data allocated to U.2 NVMe SSD, medium-heat data allocated to M.2 NVMe SSD, and low-heat data allocated to M.2 SATA SSD. When the target level capacity is insufficient, the boundary value is dynamically adjusted to ensure available space; at the same time, structured migration scheduling instructions are generated, including source physical address, target level identifier and migration priority. The access heat of each data block is intelligently predicted, and executable migration instructions are generated to dynamically optimize the storage hierarchy. Step S3: According to the migration priority, different access permissions of the buffer area in the shared DRAM pool are allocated, with high-priority tasks using exclusive buffer area and low-priority tasks using shared buffer area. Through the DMA engine, a direct read channel from the source SSD to the buffer area is established, and after the transmission is completed, a direct write channel from the buffer area to the target SSD is established, realizing zero-copy migration and reducing CPU load. In the transmission process, the data is checked in real time through the cyclic redundancy check mechanism to ensure data integrity. The data block is efficiently and safely migrated from the source SSD to the target SSD, supporting dynamic scheduling of data between different storage hierarchies.Step S4: Real-time collection of PCIe link bandwidth utilization and SSD controller command queue depth, establishment of comprehensive load index, sliding window smoothing processing and prediction of load trend. When the system load exceeds the threshold, reduce the concurrency and data block granularity of low-priority migration tasks to reduce the impact on system performance. Feedback the load information to the state-aware module, adjust the sampling frequency and feature dimension of the I / O feature sequence according to the load, reduce the sampling frequency and retain the core feature dimension under high load, and increase the sampling frequency and enable the extended feature dimension under low load. Achieve dynamic balance between migration tasks and system load, ensure the stability of the data migration process, and maintain high-precision I / O monitoring.
[0041] In summary, the present application has at least the following effects:
[0042] The present application provides a server intelligent integrated layered storage system based on U.2 interface, which redefines the reserved pins of U.2 interface to establish multiple protocol parallel channels, and monitors the signal integrity parameters of each channel in real time. The signal integrity parameters and device I / O access features are fused and processed to generate an I / O feature sequence of physical layer state information, which realizes the state awareness of different storage devices and the accurate description of I / O behavior, and provides a reliable basis for subsequent access heat prediction and data migration. By receiving the I / O feature sequence of physical layer state information, the multi-scale features of access mode are extracted using a time series convolutional neural network, the access heat probability of each data block is calculated, and the target storage level of the data block is dynamically mapped according to the access heat probability, to generate data migration scheduling instructions containing source physical address, target level identifier and migration priority, realize intelligent scheduling of data blocks among different storage levels, and optimize storage resource utilization and access performance. Further, through the migration execution module, zero-copy migration of data blocks between shared DRAM pool buffer and SSD is realized, and combined with the isolation buffer strategy and the cyclic redundancy check mechanism, the efficiency and data integrity of the migration task are ensured. Through the resource coordination module, the system load and the migration task state are monitored in real time, the migration task concurrency and data block granularity are dynamically adjusted, and the sampling frequency and feature dimension of the I / O feature sequence are adaptively adjusted, to realize dynamic balance between the migration process and the system load, improve the system stability and the storage scheduling accuracy.
[0043] Those skilled in the art will understand that embodiments of the present application can be provided as methods, systems or computer program products. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0044] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks Figure 1 means for functionally implementing the steps listed in the flowchart block or blocks.
[0045] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks Figure 1 means for functionally implementing the steps listed in the flowchart block or blocks.
[0046] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks Figure 1 means for functionally implementing the steps listed in the flowchart block or blocks.
[0047] While the preferred embodiments of the application have been described, additional variations and modifications can be employed by those skilled in the art. Therefore, the appended claims intend to cover all such modifications and variations as fall within the true spirit and scope of the present application.
[0048] It is apparent that a number of modifications and variations of the present application are possible in light of the above teachings. It is therefore to be understood that within the scope of the appended claims and their equivalents, the application can be practiced otherwise than as specifically described.
Claims
1. A server-integrated intelligent hierarchical storage system based on a U.2 interface, characterized in that, Includes the following modules: The state awareness module is used to establish a multi-protocol parallel channel by redefining the reserved pins of the U.2 interface, monitor the signal integrity parameters of each channel in real time, and fuse the signal integrity parameters with the device I / O access characteristics to generate an I / O feature sequence of physical layer state information. The heat prediction module is used to receive the I / O feature sequence of physical layer status information, extract multi-scale features of access patterns through a temporal convolutional neural network, calculate the access heat probability of each data block, and determine the target storage level of each data block based on the high and low access heat probabilities, thereby generating a data migration scheduling instruction containing the source physical address, target level identifier and migration priority. The migration execution module is used to receive data migration scheduling instructions, read data blocks from the corresponding SSD flash memory according to the source physical address in the instructions, allocate the corresponding level of isolation buffer in the shared DRAM pool according to the migration priority in the instructions, write the data blocks to the corresponding SSD flash memory according to the target level identifier in the instructions, and realize zero-copy migration of data blocks through the DMA engine. The resource coordination module is used to monitor the PCIe link utilization and SSD controller queue depth of the migration execution module in real time. It dynamically adjusts the concurrency and data block granularity of migration tasks according to the system load threshold and migration priority, and feeds back to the status awareness module to adjust the sampling frequency and feature dimension of I / O feature sequences.
2. The server intelligent integrated hierarchical storage system based on the U.2 interface according to claim 1, characterized in that: The specific process of establishing a multi-protocol parallel channel by redefining the reserved pins of the U.2 interface is as follows: Identify the reserved signal pin groups in the U.2 interface that are preset for the SAS protocol, redefine the first group of SAS signal pins as PCIe x1 links to establish an independent data transmission channel with the M.2 NVMe SSD, redefine the second group of SAS signal pins as SATA x1 links to establish an independent data transmission channel with the M.2 SATA SSD, and retain the native PCIe x4 link in the U.2 interface as a high-speed data channel with the U.2 NVMe SSD; Each redefined link is configured with an independent device presence detection signal, and the insertion status of each SSD is identified by the combination of the level states of the detection pins.
3. The server intelligent integrated hierarchical storage system based on the U.2 interface according to claim 2, characterized in that: The specific process of real-time monitoring of signal integrity parameters for each channel, fusing these parameters with device I / O access characteristics, and generating an I / O feature sequence of physical layer status information is as follows: By using the signal quality monitoring unit built into the interface controller, the signal integrity parameters of each data channel are periodically collected, including the signal bit error rate, timing jitter amplitude and eye diagram opening. A mapping model from signal integrity parameters to link health is established, and the original signal parameters are converted into standardized link quality scores. Parallel capture of I / O requests through each channel, extract logical block address, operation type, data size and timestamp information from the requests, and align link quality score with real-time I / O features by timestamp and feature concatenation to form a hybrid feature vector; By using a sliding time window, the hybrid feature vectors are serialized and normalized to generate a physical layer state information I / O feature sequence.
4. The server intelligent integrated hierarchical storage system based on the U.2 interface according to claim 1, characterized in that: The specific process of receiving I / O feature sequences of physical layer state information, extracting multi-scale features of access patterns through a temporal convolutional neural network, and calculating the access frequency probability of each data block is as follows: The input I / O feature sequence is preprocessed with time-dimensional standardization, and access pattern features at different time scales are extracted by a temporal convolutional neural network with causal dilated convolution. A multi-head self-attention mechanism is introduced at each layer of the convolutional network to adaptively allocate weights to key time steps and feature channels in the feature map. The feature maps of different levels are connected across layers and feature fusion is performed. The fused high-dimensional features are mapped to the access popularity probability value of each data block in the current time window through a fully connected layer. The popularity probability sequence is exponentially weighted and smoothed based on the sliding time window mechanism to reduce the impact of instantaneous fluctuations on prediction stability.
5. The server intelligent integrated hierarchical storage system based on the U.2 interface according to claim 4, characterized in that: The specific process of determining the target storage level of each data block based on the probability of access popularity, and thus generating a data migration scheduling instruction containing the source physical address, target level identifier, and migration priority, is as follows: A dynamic hot probability interval division mechanism is established to map the access hot probability of data blocks to the corresponding storage level. The high hot interval corresponds to U.2 NVMe SSD, the medium hot interval corresponds to M.2 NVMe SSD, and the low hot interval corresponds to M.2 SATA SSD. The remaining capacity distribution of each storage level is monitored in real time. When the remaining capacity of the target level is lower than the set threshold, the boundary value of the hot probability interval is dynamically adjusted to reallocate some data blocks to the available space of adjacent levels. Based on the determined target storage level, a structured migration scheduling instruction is constructed. The instruction includes the source physical address of the data block, the target level identifier, and the migration priority based on the probability change rate of heat and the data block size. The generated scheduling instruction is verified for resource availability to ensure that the target level has sufficient receiving capacity and performance margin. At the same time, conflict detection is performed on the migration task being executed to avoid address overlap and resource contention.
6. The server intelligent integrated hierarchical storage system based on the U.2 interface according to claim 1, characterized in that: The migration execution module includes the following steps: Based on migration priority, isolated buffers with different access permissions are divided in the shared DRAM pool, where high-priority migration tasks are allocated exclusive buffers and low-priority migration tasks use shared buffers. A direct read channel from the source SSD flash memory to the shared DRAM pool buffer is established through the DMA engine, transferring data blocks from the source SSD flash memory to the allocated isolation buffer; After the data transfer is completed, a direct write channel from the shared DRAM pool buffer to the target SSD flash memory is established through the DMA engine, and the data block is written from the buffer to the specified physical address of the target SSD. During data transmission, a cyclic redundancy check mechanism is used to verify the transmitted data in real time. A checksum is generated during the source SSD read phase and verified during the target SSD write phase.
7. The server intelligent integrated hierarchical storage system based on the U.2 interface according to claim 1, characterized in that: The specific process of dynamically adjusting the concurrency and data block granularity of migration tasks based on system load thresholds and migration priorities, while monitoring the PCIe link utilization and SSD controller queue depth of the migration execution module in real time, is as follows: The bandwidth utilization of each PCIe link and the command queue depth of each SSD controller are collected in real time by performance counters to calculate the current comprehensive load index of the system. A sliding window-based load assessment function is established to smooth and predict the trend of the collected load data. When the system load exceeds the threshold, the migration tasks are scheduled in a hierarchical manner according to the migration priority to reduce the number of concurrent executions of low-priority migration tasks and reduce the data block granularity to reduce the impact of a single migration on system performance.
8. The server intelligent integrated hierarchical storage system based on the U.2 interface according to claim 7, characterized in that: The specific process for adjusting the sampling frequency and feature dimension of the I / O feature sequence is as follows: Establish a mapping relationship between system load status and sampling parameters, and dynamically adjust the sampling frequency of I / O characteristic sequences according to the load level, reducing the sampling frequency under high load and increasing the sampling frequency under low load. The configuration of I / O feature sequence feature dimension combination is based on system load characteristics. Core feature dimensions are retained under high load, and extended feature dimensions are enabled under low load. The configuration parameters are sent to the status awareness module in real time.
9. A server intelligent integrated hierarchical storage method based on a U.2 interface, applied to the server intelligent integrated hierarchical storage system based on a U.2 interface as described in any one of claims 1-8, characterized in that, Includes the following steps: S1. By redefining the reserved pins of the U.2 interface, a multi-protocol parallel channel is established, the signal integrity parameters of each channel are monitored in real time, and the signal integrity parameters are fused with the device I / O access characteristics to generate the I / O feature sequence of physical layer status information; S2. Receive the I / O feature sequence of physical layer status information, extract multi-scale features of access patterns through a temporal convolutional neural network, calculate the access hotness probability of each data block, and determine the target storage level of each data block based on the high and low access hotness probability, thereby generating a data migration scheduling instruction containing the source physical address, target level identifier and migration priority. S3. Receive data migration scheduling instructions, read data blocks from the corresponding SSD flash memory according to the source physical address in the instructions, allocate isolation buffers of the corresponding level in the shared DRAM pool according to the migration priority in the instructions, write data blocks to the corresponding SSD flash memory according to the target level identifier in the instructions, and realize zero-copy migration of data blocks through the DMA engine. S4. Monitor the PCIe link utilization and SSD controller queue depth of the migration execution module in real time, dynamically adjust the concurrency and data block granularity of the migration task according to the system load threshold and migration priority, and feed back to the status awareness module to adjust the sampling frequency and feature dimension of the I / O feature sequence.
Citation Information
Patent Citations
A quantitative model for evaluating indexes of service support capability of command and control system
CN109472494A
High-speed data unloading and storage system and method based on DPU
CN119166075A