Storage Resource Pooling System and Dynamic Adjustment Method

Abstract

The present invention discloses a storage resource pooling system, including a storage subsystem and a server motherboard subsystem, wherein: the storage subsystem includes a switching module and an Ethernet solid-state disk, the switching module uses an Ethernet switching chip to complete the data exchange function, and the Ethernet solid-state disk is a solid-state disk based on the Ethernet interface; the server motherboard subsystem includes a processor system module, an intelligent network card module and an artificial intelligence module, the server motherboard subsystem is connected to the storage subsystem through the intelligent network card module to complete the exchange of business data, and the server motherboard subsystem realizes the management of the storage subsystem through PCIE. The access path of the storage architecture of the present invention can directly access the SSD array without passing through the CPU, and will not cause a CPU processing bottleneck; based on Ethernet switching technology, the external interface is 200G and can be expanded, and the bandwidth for each SSD is 100G and can be expanded, and there is no bandwidth bottleneck in the entire transmission path. The present invention also provides a corresponding dynamic adjustment method for the storage resource pooling system.

Description

Technical Field

[0001] The present invention belongs to the field of storage technology, and more specifically, relates to a storage resource pooling system and a dynamic adjustment method. Background Art

[0002] Storage systems are the core infrastructure of data center IT environments and the ultimate carrier of data access. Storage has undergone significant changes with the advent of cloud computing, virtualization, big data, edge computing, and other related technologies. Block storage, file storage, and object storage now support access to a variety of data types. Centralized storage is no longer the mainstream storage architecture in data centers. Accessing massive amounts of data and intelligent applications at the edge requires a distributed storage architecture with high bandwidth, low latency, and strong scalability.

[0003] Traditional servers supporting distributed storage in data centers rely on an underlying resource pooling architecture based on a PCIe (Peripheral Component Interconnect Express) or SAS (Serial Attached SCSI) bus switching architecture. The storage access path passes through the network interface card (NIC), CPU, PCIe or SAS switch, and sometimes even port expanders, before reaching the NVMe (Non-Volatile Memory Express) hard drive. This architecture presents the following issues: 1. Data is processed at the software layer, resulting in high and unpredictable access latency. 2. The CPU and PCIe card are interconnected separately, creating a bottleneck in interface bandwidth. 3. The CPU's inherent processing power is limited, creating a bottleneck in processing stored data. Compared to the increased speed of PCIe4 / 5, it's difficult to improve distributed storage I / O performance. 4. The total cost of physical implementation of this architecture is high. Summary of the Invention

[0004] To reduce latency, expand bandwidth, and optimize the overall cost of storage resource pools, the present invention introduces an Ethernet solid-state drive based on an Ethernet data exchange mechanism and NVMe over ROCEV2 (Remote Direct Memory Access over Converged EtherNet V2). To achieve zero packet loss and high throughput in Ethernet switching, the ECN (Explicit Congestion Notification) threshold is intelligently adjusted, and a feasible reinforcement learning model is developed. To ensure the reliability and efficiency of the model, four feature items are selected, and monitoring time slots and a series of related preset parameters are determined. Ultimately, the solution is implemented in software using the DDQN (Deep DoubleQ Network) algorithm.

[0005] To achieve the above objectives, according to one aspect of the present invention, a storage resource pooling system is provided, comprising a storage subsystem and a server motherboard subsystem, wherein:

[0006] The storage subsystem includes a switching module and an Ethernet solid-state disk, wherein the switching module uses an Ethernet switching chip to complete the data exchange function, and the Ethernet solid-state disk is a solid-state disk based on an Ethernet interface;

[0007] The server motherboard subsystem includes a processor system module, an intelligent network card module and an artificial intelligence module. The server motherboard subsystem is connected to the storage subsystem through the intelligent network card module to complete the exchange of business data. The server motherboard subsystem manages the storage subsystem through PCIE.

[0008] In one embodiment of the present invention, the switching module provides an Ethernet port externally and is expandable, and provides an Ethernet port internally, and is connected to the processor system module through an intelligent network card that supports NVMe Over ROCEV2.

[0009] In one embodiment of the present invention, the Ethernet solid-state drive is embedded with an NVMe Over ROCEV2 acceleration module, a 100G MAC layer module, a main control module, an NVMe read / write controller and a Nand module, wherein the main control module is responsible for completing the flash translation layer FTL function of the traditional solid-state drive SSD; the NVMe read / write controller is responsible for completing direct memory access DMA support, multi-queue access and Nand access and management; the NVMe Over ROCEV2 acceleration module realizes hardware acceleration of NVMe read and write based on the ROCEV2 protocol; and the 100G MAC layer module is responsible for completing the media access sublayer function of 100G Ethernet.

[0010] In one embodiment of the present invention, the processor system module is embedded in the system motherboard and includes a single CPU or a dual CPU cascade, as well as a memory module; the intelligent network card module is inserted into the system motherboard in the form of a plug-in card and works, supports the NVMeOver ROCEV2 protocol, and completes PCIe and Ethernet channel data conversion and transmission; the artificial intelligence module uses an AI component embedded in the CPU or an artificial intelligence plug-in card configured on the PCIe slot of the motherboard, or runs directly on the processor system module of the server motherboard subsystem in the form of a software module.

[0011] In one embodiment of the present invention, ECN intelligent tuning software is run on the processor system module of the server motherboard subsystem. The ECN intelligent tuning software consists of a main control module, an embedded AI component, a data collection and status monitor, a threshold configurator, and a chip driver module. The main control module instructs the data collection and status monitor to collect communication status data and monitor the status of the switching chip, reads and writes the ECN threshold of the switching chip through the threshold configurator, and performs DDQN calculation through the embedded AI component, and is overall responsible for the implementation of ECN intelligent tuning.

[0012] According to another aspect of the present invention, a method for dynamically adjusting a storage resource pooling system is provided, comprising:

[0013] (1) Start the ECN intelligent tuning software, randomly initialize the values Q corresponding to all states and actions, randomly initialize all parameters θ′ of the neural network PreNet, initialize the parameters θ=θ′ of the target network Target Net, and clear the experience resource pool;

[0014] (2) The ECN intelligent tuning software uses the PreNet network to update and accumulate data in the experience resource pool and iteratively optimize its own network parameters;

[0015] (3) uniformly sample m samples {s j , a j , r j , s′ j}, j = 1, 2,,, m, input Target Net network to calculate Q value y j :y j =R j +γQ'(S' j ,arg max a' Q(S j ,a,θ'),θ); use the following mean square error loss function after iteration j to update all parameters of the Pre Net network through the gradient back propagation of the neural network: Where θ is the parameter of the target network and θ′ is the parameter of the estimated network;

[0016] (4) The Pre Net network performs cyclic iterative training and copies its parameters to the Target Net network at preset time intervals until the ECN intelligent tuning software exits.

[0017] In one embodiment of the present invention, step (2) includes:

[0018] (2.1) The data collector of the ECN intelligent tuning software collects the environment, i.e., the switching chip network data, and sends it to the environment database;

[0019] (2.2) Preprocess the data s of the environment database and input it into the neural network PreNet to obtain the Q value output corresponding to all actions;

[0020] (2.3) Use the improved greedy algorithm to select action a from the current Q value;

[0021] (2.4) After receiving this action a, the environment executes it, determines the environmental improvement status, gives a reward r based on the result, and then moves to the next environmental state s′;

[0022] (2.5) Change the sequence<s,a,r,s′> Store it in the experience resource pool, then repeat steps (2.1) to (2.5) for a preset number of times, and continue to store the data in the experience resource pool.

[0023] In one embodiment of the present invention, the ECN intelligent tuning software is implemented using a dual deep Q network (DDQN). The dual deep Q network (DDQN) has two neural networks with exactly the same structure but different parameters: a predictor network (Pre Net) for action selection and a target network (TargetNet) for action evaluation.

[0024] In one embodiment of the present invention, the improved greedy algorithm randomly extracts actions with a probability of 1-∈, and extracts the action with the largest Q value with a probability of ∈, that is, the optimal action a t =maxQ(s, a, θ′), θ′ is the parameter of the prediction network PreNet.

[0025] In one embodiment of the present invention, the environmental improvement condition is an improvement condition of the communication state of the switch.

[0026] In general, the above technical solutions conceived by the present invention have the following beneficial effects compared with the prior art:

[0027] (1) Avoiding CPU bottlenecks: The access path of this storage architecture can directly access the SSD array without going through the CPU, which will not cause CPU processing bottlenecks;

[0028] (2) Expanded bandwidth: This storage architecture is based on Ethernet switching technology, with an external interface of 200G and scalable. Each SSD has a 100G bandwidth and is scalable. There is no bandwidth bottleneck in the entire transmission path.

[0029] (3) Maintain compatibility: The local main system of this storage architecture accesses the SSD array and external network communication through the smart network card, which is compatible with the distributed storage infrastructure of the traditional architecture main system;

[0030] (4) Low and predictable latency: This storage architecture is based on Ethernet switching technology and ASIC-level data forwarding, which can process data without CPU software. Compared with traditional architectures, it has predictable and low latency.

[0031] (5) Cost optimization: This resource pooling architecture uses a universal Ethernet switching architecture and an ASIC-based Ethernet solid-state drive. Compared with the traditional PCIe / SAS Switch + bus extender + NVMe solid-state drive, the overall cost has greater room for optimization. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 is a schematic structural diagram of a storage resource pooling system according to an embodiment of the present invention;

[0033] Figure 2 This is a schematic diagram of the structure of an Ethernet solid-state drive in an embodiment of the present invention;

[0034] Figure 3 This is a schematic diagram of the structure of the ECN intelligent tuning system in an embodiment of the present invention;

[0035] Figure 4 Schematic diagram of the DDQN algorithm principle in an embodiment of the present invention. DETAILED DESCRIPTION

[0036] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely for the purpose of explaining the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

[0037] like Figure 1 As shown, the present invention proposes a storage resource pooling system based on Ethernet data exchange and Ethernet solid-state disk. The architecture consists of a storage subsystem and a server motherboard subsystem, wherein the storage subsystem consists of the following modules:

[0038] 1. Switching Module: This module uses an Ethernet switching chip (e.g., a common data center switch chip) to perform data exchange functions. It provides 200G Ethernet ports externally and is scalable. It also provides 30 100GBase-KR4 Ethernet ports internally, 29 of which are allocated to Ethernet solid-state drives (SSDs) and one to the processor system module. This port is connected to the processor system module via a SmartNIC that supports NVMe over ROCEV2. The switching chip can be a common data center L3 Ethernet switching chip, and the switching capacity can be selected as needed. For example, if a 32-port 100Gbps bandwidth chip is required, a 3.2T switching capacity chip can be selected; if a 32-port 200Gbps bandwidth chip is required, a 6.4T switching capacity chip can be selected.

[0039] 2. Ethernet solid state drive: a solid state drive based on Ethernet interface, such as Figure 2 As shown, it has embedded NVMe Over ROCEV2 acceleration module, 100G MAC layer module, main control module, NVMe read / write controller and Nand module. The main control module is responsible for completing the FTL (Flash Translation Layer) function of the traditional SSD (Solid State Disk); the NVMe read / write controller is responsible for completing DMA (Direct Memory Access) support, multi-queue access and Nand access and management; the NVMe Over ROCEV2 acceleration module implements hardware acceleration of NVMe read and write based on the ROCEV2 protocol; the 100G MAC layer module is responsible for completing the media access sublayer function of 100G Ethernet.

[0040] The server motherboard subsystem consists of a processor system module, an intelligent network card module, an artificial intelligence module, etc.

[0041] 1. Processor system module: embedded on the system motherboard, including single CPU or dual CPU cascade, memory module, etc.

[0042] 2. Smart network card module: It can be inserted into the system motherboard in the form of a card, mainly supports NVMe Over ROCEV2 protocol, and completes PCIe and Ethernet channel data conversion and transmission.

[0043] 3. Artificial intelligence module: You can use the AI component embedded in the CPU or configure an artificial intelligence card on the PCIe slot of the motherboard. The software module runs on the processor system module of the server motherboard subsystem.

[0044] The server motherboard subsystem connects to the storage subsystem via the PCIe bus. The server motherboard subsystem accesses the Ethernet switch chip via the PCIe bus and can indirectly access the Ethernet solid-state drive (SSD) connected to the switch chip via the SMBus bus. The server motherboard subsystem runs control plane software to manage the Ethernet switch chip and SSD. The server motherboard subsystem connects to the storage subsystem's switch chip via a SmartNIC to exchange storage service data.

[0045] The ECN intelligent tuning software runs on the processor system module of the server motherboard subsystem, such as Figure 3 As shown, it consists of a main control module, embedded AI components, data collection and status monitor, threshold configurator, and chip driver module. The main control module directs the data collection and status monitor to collect data and monitor the communication status of the switching chip, and reads and writes the ECN threshold of the switching chip through the threshold configurator. It performs DDQN calculations through the embedded AI components and is responsible for the overall implementation of ECN intelligent tuning. Among them:

[0046] Main control module: responsible for the control and scheduling of ECN intelligent tuning work;

[0047] Embedded AI component: responsible for the parallel computing and result feedback of the DDQN algorithm;

[0048] Data collection and status monitor: Responsible for collecting data related to the switch chip and Ethernet SSD and the switch communication status data through the chip driver module, and responding to commands or dispatches from the main control module;

[0049] Threshold Configurator: Responsible for threshold statistics and sorting, completes ECN threshold configuration reading and writing through the chip driver module, and responds to commands or scheduling from the main control module;

[0050] Chip driver module: responsible for the software system's configuration and data read and write access to the chip through buses such as PCIe, and receives initialization from the main control module.

[0051] The entire storage resource pool network exchange runs based on the ROCEV2 protocol. The NVMe protocol is embedded in the ROCEV2 payload as the protocol logic upper layer to implement storage access instruction set operations.

[0052] High-performance distributed storage applications in data centers are characterized by a "many-to-one" incast traffic model. In Ethernet switching architectures, incast traffic can easily cause instantaneous congestion in the switch's internal queue buffers, even leading to packet loss. This can increase application latency and reduce throughput, thereby impairing distributed application performance. The RoCE V2 protocol operates on top of the User Datagram Protocol (UDP), which is inherently unreliable. Storage and applications require zero packet loss and high throughput, so RoCEV2 can only achieve this by relying on flow control mechanisms. The flow control technology discussed in this article primarily refers to that based on RoCEV2. RoCEV2's flow control utilizes the back-off N-frame retransmission strategy of the DCQCN (Data Center Quantized Congestion Notification) flow control mechanism. When retransmission is invoked, performance can be severely impacted. Therefore, RoCEV2 requires priority-based flow control to ensure packet loss-free data transmission. Even a loss of just one in a thousand packets can reduce RoCE (Remote Direct Memory Access over Converged EtherNet) service performance by approximately 30%. The storage resource pool here uses the NVMe over ROCEV2 protocol, which requires lossless switching with zero packet loss. This means the switching chip cannot lose packets due to buffer overflows, for example. Furthermore, packet loss cannot cause congestion, which would reduce network communication throughput and increase latency.

[0053] Traffic congestion is addressed using the switch's PFC (Priority Flow Control) and ECN mechanisms. When congestion occurs, the sender is notified to update the injection rate. The ECN threshold is the outbound queue buffer threshold. However, for lossless transmission, the switch chip cannot adapt the ECN threshold to the changing buffer space in the queue. To balance latency-sensitive small flows with throughput-sensitive large flows, and to address the fluctuating proportions of these two, the ECN threshold needs to be adjusted dynamically. Furthermore, the ECN threshold setting must minimize triggering PFC flow control, as this can easily lead to PFC deadlock. The ECN threshold is crucial for maintaining low-latency network queues and significantly impacts the latency and throughput performance of running applications. If the ECN threshold is too low, the link will suffer from insufficient throughput; if it is too high, latency will increase and the benefits of ECN's early congestion feedback will be lost. Therefore, continuous dynamic adjustment and optimization of the ECN threshold is necessary.

[0054] The ECN threshold optimization problem conforms to the Markov decision process of reinforcement learning. ECN optimization can be formulated as a deep reinforcement learning problem. In DRL, there's a dual-network architecture called the DQN (Deep Q Network) algorithm, also known as the DDQN (Deep Double Q-Network) algorithm. This algorithm combines deep learning and reinforcement learning. The DDQN algorithm uses experience replay, which uses a memory block to store previously experienced data. Each time a parameter is updated, a portion of the data is extracted from the memory for a loop to update. This algorithm model is used here.

[0055] The DDQN algorithm is a Q learning optimal strategy with experience replay, which can effectively solve the over-estimation problem of the DQN algorithm, such as Figure 4 As shown in the figure, the algorithm framework contains two neural networks with identical structures but different parameters: the Prediction Network (Pre Net), used for action selection, and the Target Network (Target Net), used for action evaluation. The final code of the DDQN algorithm is implemented and compiled, and runs on the artificial intelligence module of the mainboard subsystem. The specific implementation is as follows:

[0056] (1) Start the ECN intelligent tuning software, randomly initialize the values Q corresponding to all states and actions, randomly initialize all parameters θ′ of the neural network PreNet, and initialize the parameters θ=θ′ of the target network Target Net. Clear the experience resource pool.

[0057] (2) The ECN intelligent tuning software uses the PreNet network to update and accumulate data in the experience resource pool and iteratively optimize its own network parameters;

[0058] (2.1) The data collector of the ECN intelligent tuning software collects the environment, i.e., the switching chip network data, and sends it to the environment database;

[0059] (2.2) Preprocess the data s of the environment database and input it into the neural network PreNet to obtain the Q value output corresponding to all actions;

[0060] (2.3) Use the ∈-greedy algorithm (improved greedy algorithm) to select action a from the current Q value. The ∈-greedy algorithm has a strategy that balances development and exploration. It randomly extracts actions with a probability of 1-∈ and extracts the action with the largest Q value with a probability of ∈, that is, the optimal action a. t =maxQ(s,a,θ′), θ′ is the parameter of the prediction network PreNet;

[0061] (2.4) After receiving this action a, the environment executes it, determines the improvement of the environment (in our application, it determines the improvement of the communication status of the switch), and gives a reward r based on the result (for example, a reward of +1 is given for a positive transformation and a reward of -1 is given for a negative transformation), and then moves to the next environmental state s′.

[0062] (2.5) Change the sequence<s,a,r,s′> Store it in the experience resource pool, then repeat steps (2.1) to (2.5) for a preset number of times, and continue to store the data in the experience resource pool.

[0063] (3) uniformly sample m samples {s j , a j , r j , s′ j}, j = 1, 2,,, m, input Target Net network to calculate Q value y j :

[0064] y j =R j +γQ'(S' j ,arg max a' Q(S j ,a,θ'),θ)

[0065] Use the following mean square error loss function to update all parameters of the Pre Net network through gradient backpropagation of the neural network after iteration j:

[0066]

[0067] Where θ is the parameter of the target network and θ′ is the parameter of the estimated network.

[0068] (4) The PreNet network is trained iteratively in steps (2) and (3), and its parameters are copied to the Target Net network at preset time intervals until the ECN intelligent tuning software exits.

[0069] The above is the complete learning process of the DDQN algorithm. The ECN threshold is key to achieving low-latency, high-throughput communication in congestion control schemes. The DDQN algorithm is adopted and engineered here to automatically adjust ECN parameters at each switch chip. This significantly improves the completion time of small data flows while maintaining high throughput for large data flows, thus stably supporting distributed storage services.

[0070] The technical solution of the present invention is described below with reference to a specific embodiment:

[0071] Storage resource pooling architecture implementation:

[0072] Hardware implementation:

[0073] This primarily involves selecting a switch chip. In theory, any mainstream commercial switch chip should be suitable. For example, for a 3.2Tbps switching capacity, Broadcom's BCM56870 is an option. The CPU for the server motherboard subsystem module can be a general-purpose X86 processor from Intel or AMD with built-in AI components, such as Intel's Skylake Gold 63XX processor.

[0074] The server motherboard subsystem connects to the storage subsystem's switch chip via PCIe 3 / 4. The main system software manages the switch chip through this channel. The switch chip connects to each Ethernet SSD via the SMBus bus. The main system software manages the Ethernet SSDs through the switch chip's internal controller. The server motherboard subsystem connects to the switch chip via the SmartNIC in the PCIe slot, enabling data access to the SSD cluster and external network interconnection.

[0075] Software implementation:

[0076] The server motherboard subsystem runs ECN intelligent tuning software, Figure 1 The PCIe and SMBus buses shown in the figure provide access to the switch chip and Ethernet SSD, and use telemetry to obtain real-time network information from the Ethernet switch chip. The DDQN algorithm is implemented based on the embedded AI component.

[0077] Implementation of dynamic intelligent tuning of ECN switching chips

[0078] 1) Implementation of ECN intelligent tuning software system:

[0079] Main control module: responsible for overall coordination control and data scheduling of the software system;

[0080] Embedded AI component: responsible for implementing the DDQN algorithm, responding to the input status and actions of the main control module, and outputting the results of parallel calculations to the main control module;

[0081] Data Collection and Status Monitor: This module collects network exchange data from the switch chip via the PCIe bus and performs statistical processing. It also monitors the switch status and responds to and provides feedback on commands from the main control module.

[0082] Threshold Configurator: Access and statistical management of complex ECN thresholds, and complete the configuration update of the ECN threshold of the switching chip according to the instructions and data of the main control module.

[0083] Chip driver module: provides software system with chip configuration and data read and write access through buses such as PCIe.

[0084] The above components run systematically under the operation of the main system module, dynamically optimizing the switch ECN threshold through the DDQN algorithm, and ultimately achieving lossless and efficient data transmission on the switching chip.

[0085] 2) Algorithm implementation:

[0086] The basic framework of reinforcement learning generally consists of five components: an agent, an environment, an action A, a state S, and a reward r. For this application, the state represents the network environment information of the switching module. Here, we represent the state as collectible statistics that can be dynamically measured from each switch chip via telemetry via the PCIe bus. To accelerate the convergence of the neural network, we empirically select four switch features: the current queue length (Qlen), the output data rate of each link (Orate), the output rate of ECN-marked packets on each link (Orate-m), and the current ECN setting (ECN-c). We use the queue state over the past k monitoring time slots (Δt) as the state information for each tuning inference. Based on repeated experimental results, k is set to 3.

[0087] The action at time t is defined by the ECN threshold settings, namely the high marking threshold (Kmax), the low marking threshold (Kmin), and the marking probability (Pmax). For the marking probability Pmax, when the Pmax adjustment interval is greater than 5%, the network throughput or latency changes by more than 1%. Therefore, uniform discretization is recommended:

[0088] Action: a j ={K max , K min , P max} j

[0089] Status: S j ={Qlen,Orate,Orate-m,ECN-c} j

[0090] Because the network and action space of a switch chip is large, which affects the convergence speed of the neural network, we discretize the ECN tuning action space and select the discretized action value E(n) as the ECN marking threshold. Testing several fine-grained settings shows that when throughput is greater than 1MB, throughput is insensitive to high marking thresholds. Therefore, we choose a coarse-grained setting here to minimize the action space, such as {1MB, 2MB, 5MB, 10MB}, because the maximum buffer size per queue in commercial switch chips is typically less than 10MB. For low marking thresholds, setting multiple intervals within a short range helps to fine-tune the marking of packets during congestion. To digitize this characteristic, we introduce an exponential function to determine the discrete action value E(n).

[0091] K=E(n)=β×2 n KB, n = 0, ..., 9; where β is used as a step gradient control and is set to 20 in this system.

[0092] For distributed storage design, we can only obtain local information from the switch. To ensure high link utilization and low queue accumulation, we define the following reward function r: The average throughput of an egress queue is represented by ORate, which is the amount of data that has been transmitted to the link within the time interval △t. We normalize ORate by the link bandwidth BW to represent the link utilization T. The delay is represented by the average queue length L to indicate the impact of queuing delay. The larger L is, the greater the delay will be. Here we choose the average value instead of the instantaneous queue length because the instantaneous queue length varies over a large range, which makes the reward unstable. Therefore, we define the reward function as a trade-off between latency and throughput, that is, a trade-off between high link utilization and low queue length for each switch chip:

[0093] r=w1×T(ORate)+w2×D(L)

[0094] Where T(ORate) = ORate / BW, which represents the link utilization. Considering that applications are more sensitive to delay, we design D(L) = 1-n / 10, where L is the average queue length. Here, D(L) is a step-mapping function. w1 and w2 are weights representing the utility-latency tradeoff, where w1 + w2 = 1. Users can easily configure the reward parameters based on the requirements of their application. For example, in our storage resource pool system, we recommend using w1 = 0.7 and w2 = 0.3.

[0095] The specific settings of the DDQN algorithm preset parameters are as follows:

[0096] 1. Preprocessing is based on the ∈-greedy algorithm. Considering the speed of search improvement and the convergence speed of the neural network, ∈=0.9 is taken here.

[0097] 2. After several generations of training, PreNet copies its parameters to Target Net at preset intervals. If this interval is on the same scale as the reaction time of congestion control, it will seriously affect the performance of existing congestion control schemes. DCQCN uses several RTTs to respond to congestion signals in the control loop. To avoid interfering with DCQCN rate control, we choose an interval that is an order of magnitude higher than the RTT, set here to 10ms.

[0098] 3. According to theoretical derivation, the neural network Q value update formula of the DDQN algorithm is:

[0099] Q*(s,a)←Q(s,a)+α[r+γ×max a' Q(s',a')-Q(s,a)]

[0100] Where α is the learning rate and γ is the reward discount factor. If α is too small, it is difficult to find the optimal solution, while if it is too large, it will increase the training cost. Here, α = 1 is selected. The closer γ is to 0, the more it only considers the current interests, and the closer it is to 1, the more foresighted it is and the more it values the value of subsequent states. Here, γ = 0.8 is selected as a compromise.

[0101] Neural network parameter update formula:

[0102]

[0103] Loss function update formula:

[0104] Substitute the previous r function, save the evaluation network every n steps, update the target network parameters every 10ms, and obtain the target network.

[0105] 4. For the DDQN model, offline training and empirical judgment can be used to obtain a pre-installed neural network model. Online training can be used to optimize the model and achieve generalization and application capabilities. Data acquisition for offline training can be performed using auxiliary tools such as Perf-Test.

[0106] This iterative strategy dual neural network model based on the experience replay resource pool can be used to develop AI programs with the help of open source AI frameworks such as the PaddlePaddle platform. It can be used in C or Python, and the amount of code in C is more than 4,000 lines.

[0107] It will be easily understood by those skilled in the art that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for dynamic adjustment of a storage resource pooling system, characterized in that: include: (1) Start the ECN intelligent tuning software, randomly initialize the values Q corresponding to all states and actions, randomly initialize all parameters θ′ of the neural network Pre Net, initialize the parameters θ=θ′ of the target network Target Net, and clear the experience resource pool; (2) The ECN intelligent tuning software realizes the update and accumulation of the experience resource pool data and the iterative optimization of its own network parameters through the Pre Net network; the step (2) includes: (2.1) the data collector of the ECN intelligent tuning software collects the environment, i.e., the switching chip network data s, and sends it to the environment database; (2.2) Preprocess the data s from the environment database and input it into the neural network Pre Net to obtain the Q value output corresponding to all actions; (2.3) Use the improved greedy algorithm to select action a from the current Q value; (2.4) After receiving this action a, the environment executes it, judges the improvement of the environment, gives a reward r based on the result, and then moves to the next environment state s′; (2.5) Sequence <s ， a r ，s′> Store the data in the experience resource pool, then repeat steps (2.1) to (2.5) for a preset number of times, and continue to store the data in the experience resource pool;< / s> <s ， a r ，s′> (3) Uniformly sample m samples from the experience resource pool { }, j=1, 2, ..., m, input TargetNet network to calculate Q value : ; Use the following mean square error loss function to update all parameters of the Pre Net network through the gradient back propagation of the neural network after iteration j: ; where θ is the parameter of the target network and θ′ is the parameter of the estimated network; (4) The PreNet network performs iterative training and copies its parameters to the TargetNet network at preset time intervals until the ECN intelligent tuning software exits. The storage resource pooling system includes a storage subsystem and a server motherboard subsystem, wherein: The storage subsystem includes a switching module and an Ethernet solid-state disk, wherein the switching module uses an Ethernet switching chip to complete the data exchange function, and the Ethernet solid-state disk is a solid-state disk based on an Ethernet interface; The server motherboard subsystem includes a processor system module, an intelligent network card module and an artificial intelligence module. The server motherboard subsystem is connected to the storage subsystem through the intelligent network card module to complete the exchange of business data. The server motherboard subsystem manages the storage subsystem through PCIE.

2. The dynamic adjustment method of the storage resource pooling system according to claim 1, characterized in that: The ECN intelligent tuning software is implemented using a dual deep Q network (DDQN). The dual deep Q network (DDQN) has two neural networks with exactly the same structure but different parameters: a predictor network (Pre Net) for action selection and a target network (Target Net) for action evaluation.

3. The dynamic adjustment method of the storage resource pooling system according to claim 1, characterized in that: Improve the greedy algorithm to The probability of randomly extracting actions is The probability of extracting the action with the largest Q value, that is, the optimal action = maxQ(s, a, θ′), θ′ is the parameter of the prediction network Pre Net.

4. The dynamic adjustment method of the storage resource pooling system according to claim 1, wherein: The environmental improvement status is an improvement status of the communication state of the switch.

5. The dynamic adjustment method of the storage resource pooling system according to claim 1, wherein: The switching module provides an Ethernet port externally and is expandable, and provides an Ethernet port internally, and is connected to the processor system module through an intelligent network card that supports NVMe Over ROCEV2.

6. The dynamic adjustment method of the storage resource pooling system according to claim 1, wherein: The Ethernet solid-state drive is embedded with an NVMe Over ROCEV2 acceleration module, a 100G MAC layer module, a main control module, an NVMe read / write controller, and a Nand module. The main control module is responsible for completing the flash translation layer FTL function of the traditional solid-state drive (SSD); the NVMe read / write controller is responsible for completing direct memory access DMA support, multi-queue access, and Nand access and management; the NVMe Over ROCEV2 acceleration module implements hardware acceleration of NVMe read and write based on the ROCEV2 protocol; and the 100G MAC layer module is responsible for completing the media access sublayer function of 100G Ethernet.

7. The dynamic adjustment method of the storage resource pooling system according to claim 1, wherein: The processor system module is embedded in the system motherboard and includes a single CPU or dual CPU cascade, as well as a memory module. The intelligent network card module is inserted into the system motherboard in the form of a plug-in card and supports the NVMe Over ROCEV2 protocol to complete PCIe and Ethernet channel data conversion and transmission. The artificial intelligence module uses the AI component embedded in the CPU or an artificial intelligence plug-in card configured on the PCIe slot of the motherboard, or runs directly on the processor system module of the server motherboard subsystem in the form of a software module.

8. The method for dynamic adjustment of a storage resource pooling system according to claim 1, wherein: ECN intelligent tuning software runs on the processor system module of the server motherboard subsystem. The ECN intelligent tuning software consists of a main control module, an embedded AI component, a data collection and status monitor, a threshold configurator, and a chip driver module. The main control module directs the data collection and status monitor to collect data on the communication status of the switching chip and monitor the status, reads and writes the ECN threshold of the switching chip through the threshold configurator, and performs DDQN calculations through the embedded AI component. It is overall responsible for the implementation of ECN intelligent tuning. < / s>

Images