Storage controller, read-write method and server

Abstract

The invention provides a storage controller, a read-write method and a server, which can be applied to the technical field of storage. The storage controller comprises at least one multi-core processor, and each multi-core processor comprises a plurality of processing units which are connected with one another; the at least one multi-core processor is configured to sequentially control a plurality of processing units on a storage access path, receive a data transmission instruction and perform data reading and writing on a disk according to the data transmission instruction; wherein the plurality of processing units on the storage access path are configured to realize ordered communication through the host interaction layer, the cache layer, the redundant array of disks and the disk management layer according to the processing sequence of the data transmission instruction.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of storage, in particular to a storage controller, a data read-write method and a server. BACKGROUND

[0002] Unified Storage (US) is a storage solution integrating Network-Attached Storage (NAS) and Storage Area Network (SAN) architectures, supporting multiple protocols such as Fibre Channel (FC), Internet Small Computer System Interface (iSCSI), Network File System (NFS), Common Internet File System (CIFS), etc., to realize unified management of block-level, file-level and object-level data access.

[0003] The performance of storage is usually closely related to the Central Processing Unit (CPU). With the continuous development of CPU technology, multi-CPU multi-die architecture is gradually applied, for example, data reading (Output) / writing (Input) can be realized through multi-CPU and its internal multi-die. However, under the multi-CPU multi-die architecture, the software implementation of data I / O needs to be realized by the multi-die in the CPU, which may cause performance problems due to cache synchronization between dies. SUMMARY

[0004] In view of the above problems, the present application provides a storage controller, a read-write method and a server.

[0005] According to a first aspect of the present application, a storage controller is provided, comprising at least one multi-core processor, each of the at least one multi-core processor comprising a plurality of processing units connected to each other; the at least one multi-core processor is configured to: sequentially control a plurality of processing units on a storage access path, receive a data transmission instruction and perform data read-write on a disk according to the data transmission instruction; wherein the plurality of processing units on the storage access path are configured to: realize orderly communication in a host interaction layer, a cache layer, a redundant array of independent disks layer and a disk management layer, respectively, according to the processing order of the data transmission instruction.

[0006] The second aspect of the present application provides a data read-write method applied to a server, wherein the server comprises a storage controller as described above, and the read-write method comprises: sequentially controlling, by at least one multi-core processor in the storage controller, a plurality of processing units on a storage access path, receiving a data transmission instruction, and performing data read-write on the disk according to the data transmission instruction; wherein the plurality of processing units on the storage access path are configured to: respectively implement orderly communication in a host interaction layer, a cache layer, a redundant array of independent disks layer, and a disk management layer according to a processing order of the data transmission instruction.

[0007] The third aspect of the present application provides a server, comprising: a disk; a storage controller comprising at least one multi-core processor, and each of the at least one multi-core processor comprising a plurality of processing units connected to each other; the at least one multi-core processor sequentially controls a plurality of processing units on a storage access path, receives a data transmission instruction, and performs data read-write on the disk according to the data transmission instruction; wherein the plurality of processing units on the storage access path are configured to: respectively implement orderly communication in a host interaction layer, a cache layer, a redundant array of independent disks layer, and a disk management layer according to a processing order of the data transmission instruction.

[0008] In the embodiments of the present application, the plurality of dies are configured to respectively implement the host interaction layer, the cache layer, the redundant array of independent disks layer, and the disk management layer, and the plurality of dies orderly communicate according to the fixed storage access path, so that the storage controller can implement the entire I / O function by the plurality of dies in a pipelining manner. On the one hand, the storage controller of the present application does not need to be configured with a distribution module, thereby avoiding communication between the die to which the distribution module belongs and other dies, and further avoiding cache synchronization overhead caused by obtaining and storing hardware resource information, and distributing appropriate I / O for each die according to the hardware resource information. On the other hand, the entire I / O function is fixedly implemented by part of the dies, and through strict isolation of the functions between the dies, the cache synchronization problem caused by the same function being implemented by multiple dies can be avoided, thereby reducing the overhead caused by data cache synchronization of mutual communication of the same function modules. BRIEF DESCRIPTION OF DRAWINGS

[0009] The above and other objects, features and advantages of the present application will become more apparent from the following description of the embodiments of the present application taken with reference to the accompanying drawings, in which:

[0010] Figure 1A An example of cross-CPU communication based on CPU implementation of IO is shown.

[0011] Figure 1B An example of IO implementation by a virtual node architecture is shown.

[0012] Figure 1C A first example of IO implementation by a virtual node architecture through a distribution module is shown.

[0013] Figure 1D The second example of IO implementation by the virtual node architecture through the distribution module is shown.

[0014] Figure 1E The example of IO implementation by the virtual node architecture through the multi-instance mode is shown.

[0015] Figure 2 The architecture schematic diagram of the storage controller according to the embodiment of the present application is shown.

[0016] Figure 3 The architecture schematic diagram of a multi-core processor according to the embodiment of the present application is shown.

[0017] Figure 4 The corresponding schematic diagram of the relationship between the three write types and the stripe according to the embodiment of the present application is shown.

[0018] Figure 5 The operation flow chart of the small write type according to the embodiment of the present application is shown.

[0019] Figure 6 The operation flow chart of the small write type according to one specific embodiment of the present application is shown.

[0020] Figure 7 The operation flow chart of the full write type according to the embodiment of the present application is shown.

[0021] Figure 8 The scenario schematic diagram of the large write type according to the embodiment of the present application is shown.

[0022] Figure 9 The flow schematic diagram of the read-write method according to the embodiment of the present application is shown. DETAILED DESCRIPTION

[0023] Hereinafter, the embodiments of the present application will be described with reference to the accompanying drawings. It is to be understood, however, that the description is merely exemplary of the present application, and is not intended to limit the scope of the present application. In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the present application. However, it will be apparent to one skilled in the art that the present application can be practiced without these specific details. In other instances, well-known structures and techniques have not been shown in detail in order not to obscure the understanding of this application.

[0024] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the present application. As used herein, the term "including" as well as other forms such as "include", "includes", and "including", are intended to mean the presence of stated features, steps, operations, and / or components, but do not preclude the presence or addition of one or more other features, steps, operations, or components.

[0025] All terms used herein, including technical and scientific terms, have the meanings as commonly understood by one of ordinary skill in the art unless otherwise defined. It should be noted that the use of terms herein, such as should be construed to have a meaning consistent with the context of this specification and relevant art, and should not be construed ideally or overly formally unless expressly so defined herein.

[0026] In the case of using expressions such as "at least one of A, B, and C", etc., it should be generally construed that the meaning is interpreted as including one or more of the relevant items (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, a system having B alone, a system having C alone, a system having both A and B, a system having both A and C, a system having both B and C, and / or a system having A, B, and C, etc.).

[0027] Figure 1A An example of cross-CPU communication based on CPU implementation of IO is shown. Figure 1B An example of virtual node architecture implementation of IO is shown. Figure 1C A first example of virtual node architecture implementation of IO through a distribution module is shown. Figure 1D A second example of virtual node architecture implementation of IO through a distribution module is shown. Figure 1E An example of virtual node architecture implementation of IO through a multi-instance approach is shown.

[0028] The development of CPUs presents two clear main lines: one is the exponential growth of core numbers, and the other is the fundamental change in architecture design from single-chip to multi-die integration. This evolution not only reflects the progress of semiconductor technology, but also embodies the architectural innovation made to break through the performance bottleneck. The explosive growth of core numbers is the most distinctive feature of CPU development. However, in the early 21st century, when single-core performance stagnated due to power consumption and frequency walls, multi-core became an option. From the initial exploration of dual-core and quad-core, to today's mainstream server CPUs equipped with 64 cores or even 96 cores, the number of cores has almost doubled every eighteen months. This growth directly drives the improvement of parallel computing capabilities, enabling data centers to handle thousands of virtual machine and container tasks simultaneously.

[0029] As Figure 1AAs shown, the central processors 0-2 can each perform read / write operations on the a0-a3 addresses, such as read a0, read a1, read a2, read a3, write a0, write a1, write a2, and write a3. The caches 0-2 corresponding to the central processors 0-2 are used to store data read / written from the a0-a3 addresses and the a0-a3 addresses. The caches 0-2 and the memory can share the data 0 read / written from the a0-a3 addresses and the a0-a3 addresses through a data bus and an address bus. The shared bus can share control signals among the central processors, the memory, and the caches. However, the simple core stacking also brings new challenges: memory access delay, cache consistency overhead, and inter-core communication bottleneck are increasingly prominent, and the parallel limitation revealed by Amdahl's law gradually emerges.

[0030] Thus, a multi-die architecture emerges and becomes the mainstream direction of current CPU design. In a related example, a traditional CPU is decomposed into multiple functional dies: a core operation-focused die, such as a Core Chiplet Die (CCD), and an I / O die that uniformly manages interconnection and external interfaces. This division of labor mode can greatly improve yield and manufacturing flexibility, enabling manufacturers to use more advanced process technology to produce computing units, while the I / O part can use mature technology to reduce costs.

[0031] When a storage system for implementing storage data I / O runs on a multi-die environment, the performance decays by about 20% with each added die (the specific value is related to the CPU manufacturer). This is because after the storage software is deployed in a multi-die environment, the I / O storage protocol stack that undertakes the software I / O part significantly increases the cross-die cache synchronization overhead.

[0032] To solve the cross-die cache overhead problem, in an example, a storage system implemented by multiple CPUs and multiple dies can be divided into multiple virtual nodes (vNodes), each vNode corresponding to a group of CPU and memory resources, that is, the resource pool under the unified storage is uniformly allocated. The association of vNodes and CPUs can reduce CPU scheduling, achieve true "resource pooling" and "load balancing", and improve CPU utilization.

[0033] Under the vNode architecture, all hardware resources of the entire storage system (CPU, memory, cache, and ports of all controllers) are integrated into a unified resource pool. The storage system dynamically and on-demand allocates resources for each vNode from the pool through an intelligent distribution algorithm. Multiple vNodes can run simultaneously on one physical controller, and the resources of one vNode can also come from multiple physical controllers. When a logical unit (LUN) is created, the storage system allocates a master vNode and a slave vNode (master-slave design for high availability) for it. All I / O operations of the LUN are entirely responsible by its master vNode, and the mapping relationship is completely transparent to the host, which only sees a unified storage target.

[0034] As shown in Figure 1B Each host I / O data (such as multiple data with a size of 64 MB) is allocated to multiple vNode nodes (such as vNode0~vNode31) via an intelligent distribution algorithm, each vNode node corresponds to one or more LUNs in the resource pool, achieving global cache intelligent balanced allocation of vNode nodes and intelligent balanced allocation of the underlying global storage pool. Thus, storage is realized from the "controller level" to the "vNode level". Under this architecture, if a certain physical controller fails, only those vNodes running on it need to be migrated, and the storage system will automatically and evenly migrate these vNodes to other healthy physical controllers, rather than migrating all LUNs in the failed physical controller A to a healthy physical controller B under the traditional architecture. The advantages of the vNode architecture include: 1. The load caused by the failure switching is borne by all remaining controllers, there is no performance bottleneck of a single controller, and the business is not aware and the performance is smooth; 2. The migration of vNode is completed in seconds, much faster than the traditional controller-level switching; 3. The failure impact range is reduced from the entire controller to several vNodes, achieving finer failure isolation. In addition, when the storage system finds that a certain physical controller is too high in load, it can automatically migrate some vNodes running on it to a lower-load controller to achieve global load balancing; after adding a new physical controller, the storage system can automatically migrate some vNodes to the new controller to fully utilize the new resources and achieve linear smooth expansion.

[0035] Each vNode of the storage system under the vNode architecture can implement a complete I / O storage protocol stack. A complete I / O storage protocol stack can include: a host layer (Host layer, HL) as a host layer of the I / O storage protocol stack, responsible for interacting with the host; a cache module (CACHE, CA) as a cache layer of the I / O storage protocol stack, responsible for storing hot data of the I / O process; a redundant array of inexpensive disks (Redundant Array of Inexpensive Disks, RAID) module as a RAID layer of the I / O storage protocol stack, responsible for data blocking and data verification; and a volume layer (Volume layer, VL) as a disk management layer, responsible for interacting with the back-end disk, which is the back-end of the storage.

[0036] Under the vNode architecture, each physical controller is composed of multiple vNodes. In addition to the vNode itself, there is a distribution module that allocates resources to each vNode. The vNode can be bound according to the socket, that is, bound to the entire CPU of each socket; or in the future multi-die environment, it is predicted that the vNode can also be bound according to the die, realizing smaller vNodes. To realize the vNode in the multi-die environment, the die can be isolated according to the vNode to reduce the cache synchronization between the dies, avoid the synchronization overhead between the dies, and thus improve the performance.

[0037] However, the core of the vNode scheme is a "distribution module" that can distribute services to realize the allocation of I / O to each vNode. The hardware implementation of this distribution module can be, for example, a shared large card that completes the "forwarding" of host-side I / O to the vNode. As shown in Figure 1C The distribution module (shared large card) receives I / O data from hosts 1-3 and distributes it to vNodes 1-3, and each vNode bears the functions of the vertically divided I / O storage protocol stack, such as including HL, CA, RAID, and VL.

[0038] When implementing vNode in a multi-CPU multi-die architecture, the development cost of the hardware distribution module is too high. Without the hardware condition of a shared large card, a multi-instance scheme can be used to realize a similar vNode scheme. The multi-instance scheme is similar to the vNode, which also vertically divides a complete I / O storage protocol stack into several subsystems to realize similar vNode service division. The multi-instance scheme is a pure software scheme, and system management, distribution, and other transactions are implemented by separate modules or placed in a certain instance. However, in a multi-die environment, as long as it is a software implementation, the CPU is involved, so die problems will inevitably occur.

[0039] As shown in Figure 1D , a complete I / O storage protocol stack is deployed for each die, that is, instance 1 and instance 2 each include HL, CA, RAID, and VL of the I / O protocol stack, and instance 1 and instance 2 are respectively deployed in die 1 and die 2. In addition, a "distribution module" for distributing services is implemented by software and is deployed in die 1. As shown in Figure 1D , die 1 where the distribution module is located has remote access with other dies, and thus there is performance loss caused by inter-die cache synchronization.

[0040] As shown in Figure 1E , in order to avoid the disadvantages of the software-implemented distribution module, multiple instances of each module of the I / O storage protocol stack can be implemented, and instance 1 and instance 2 respectively deployed in die 1 and die 2 each include HL1~VL1 and HL2~VL2. For example, HL is divided into HL1 and HL2, CA is divided into CA1 and CA2; RAID is divided into RAID1 and RAID2, and RAID1 and RAID2 respectively manage different stripes, such as stripe 1-100 and stripe 101-200; and VL is also divided into VL1 and VL2. Among the multiple instances of each module, there is a master instance (such as HL1~VL1) responsible for responding of the entire module. For example, HL1 distributes CA1 or CA2 according to service needs, the master instance CA1 is responsible for managing part of the public data, and the slave instances CA2 and possibly CA3 and CA4 are slave instances; in addition, CA1 also distributes to RAID1, RAID2, and the like according to its own service needs. However, the master-slave instance design on the one hand makes the master instance and the slave instance exist in each other's perception, and there is data interaction between the two, which weakens the effect of die isolation; on the other hand, there is a large development workload between the master and slave instances, and each module on the I / O storage protocol stack has to develop multiple instances, which is a huge workload. In addition, the distribution work of the slave instances is borne by HL1, CA1, and RAID1 in the master instance, resulting in frequent communication between die 1 where the master instance is located and die 2 where the slave instance is located, and there is a large inter-die performance loss.

[0041] Therefore, the present application provides a new I / O storage protocol stack deployment method under a multi-CPU multi-die architecture, which reduces the performance overhead caused by inter-die communication and inter-die cache synchronization while performing read and write functions on the disk based on I / O operations.

[0042] Figure 2 An architecture schematic diagram of a storage controller according to an embodiment of the present application is shown. As shown in Figure 2 , the storage controller includes one CPU, and the CPU includes 4 dies, such as die0~die3.

[0043] The data transmission instruction can be an instruction obtained by the storage controller from the host for implementing data read / write. The storage controller is configured to receive the data transmission instruction and perform data read / write on the disk according to the data transmission instruction; and feed back read / write data obtained from the disk to the host.

[0044] One physical storage controller can include at least one multi-core processor, such as a multi-core CPU. Each multi-core processor includes a plurality of processing units (dies), and the plurality of dies can be connected to each other within the multi-core processor through an interconnection bus and communicate with each other based on a protocol supported by the interconnection bus. Each die within the multi-core processor can include a plurality of processor cores, such as C1, C2, C3, and the like. The plurality of processor cores can share resources bound on the die, such as a shared cache (e.g., a three-level cache) within the die, a bus, and the like. Each die with independent memory control capability can be identified by the storage system as a non-uniform memory access (NUMA) node, so that the memory access efficiency of the die as a whole is realized based on the one-to-one relationship between the die and the NUMA.

[0045] For example, taking a 24-core CPU as an example, the CPU can include 4 dies, each die including 6 processor cores, and the 4 dies can be identified by the operating system as 4 NUMA nodes. The storage controller can be the above-mentioned 24-core single CPU, or a “multi-CPU multi-die architecture” including a plurality of CPUs. It can be understood that the number of processor cores and the number of dies of the CPU can be determined according to actual conditions, and is not limited to the above-mentioned number.

[0046] The plurality of dies within the storage controller (belonging to one or more multi-core CPUs) can be regarded as a whole to jointly implement receiving the data transmission instruction from the host and performing data I / O on the disk based on the I / O operation indicated by the data transmission instruction. In the multi-die architecture of the storage controller, each die only undertakes part of the functions in the entire data I / O process, that is, a single die cannot implement a complete data I / O.

[0047] For example, at least one multi-core processor within the storage controller is configured to sequentially control a plurality of dies on a storage access path, receive the data transmission instruction, and perform data read / write on the disk according to the data transmission instruction. The plurality of processing units on the storage access path are configured to implement ordered communication in the host interaction layer, the cache layer, the disk redundant array layer, and the disk management layer, respectively, according to the processing order of the data transmission instruction.

[0048] The storage access path is pre-configured by the storage controller according to the functions assumed by each die. The processing sequence of the data transmission instruction is as follows: the data transmission instruction sequentially passes through the host interaction layer, the cache layer, the disk redundant array layer, and the disk management layer for processing, and accesses the disk (read or write access). The multiple processing units in the storage access path communicate according to the above processing sequence.

[0049] The host interaction layer is the interaction entrance of the storage system and the host (server), and is used for interacting with the host to receive the data transmission instruction. The cache layer is arranged below the host interaction layer in the processing sequence of the data transmission instruction, and can obtain the data transmission instruction that has been preliminarily processed from the host interaction layer and perform data caching. The cache layer can cache the data read from the disk, and can also cache the data to be written into the disk. Further, the disk redundant array layer is arranged below the cache layer, and realizes data redundancy protection and I / O parallel read / write by combining the disks into a logical array. The disk redundant array layer can perform parallel read / write on the cache data in the cache layer through the logical array. The disk management layer is arranged below the disk redundant array layer, directly communicates with the bottom disk, and performs full life cycle management on the disk, and is a bridge between the disk redundant array layer and the disk. The host interaction layer, the cache layer, the disk redundant array layer, and the disk management layer are respectively referred to as HL, CA, RAID, and VL.

[0050] For each data transmission instruction received from the host, the data transmission instruction is sequentially transmitted to the corresponding die according to the storage access path to perform the corresponding function. Since each die sequentially communicates according to the fixed storage access path, each die only assumes part of the function in the entire data I / O process, and therefore, the storage controller does not need to be configured with a distribution module.

[0051] For example, in the process of receiving the data transmission instruction and performing the data I / O process on the disk based on the I / O operation indicated by the data transmission instruction, the HL, CA, RAID, and VL can be implemented by two or more or multiple dies, as long as the entire I / O process can be horizontally divided into multiple dies and a single die cannot implement the entire I / O process.

[0052] In one specific embodiment, if the storage controller includes 2 dies, one die implements one of the above four modules, and the other die implements the remaining modules of the above four modules; or one die implements two of the above four modules, and the other die implements the remaining modules of the above four modules; if the storage controller includes 3 dies, one die implements two of the above four modules, and the remaining two dies respectively implement the remaining two modules of the above four modules, and so on.

[0053] In the embodiments of this application, multiple dies are configured to respectively implement the host interaction layer, cache layer, redundant array of disks layer, and disk management layer. These dies communicate in an ordered manner according to a fixed storage access path, enabling the storage controller to implement the entire I / O function through a pipelined approach using multiple dies. On one hand, the storage controller in this embodiment does not require a distribution module, avoiding communication between the dies to which the distribution module belongs and other dies. This avoids the cache synchronization overhead caused by acquiring and storing hardware resource information and allocating appropriate I / O to each die based on that information. On the other hand, since the entire I / O function is fixed to be implemented by only a subset of dies, strict isolation between the functions of different dies avoids the cache synchronization problem caused by multiple dies jointly implementing the same function, thereby reducing the overhead of data cache synchronization caused by communication between modules with the same function.

[0054] According to an embodiment of this application, a die includes multiple processor cores and a shared cache of the multiple processor cores, such as an L3 cache, which is used to store data received or processed by each die. Figure 3 A schematic diagram of the architecture of a multi-core processor according to an embodiment of this application is shown. Figure 3 As shown, this CPU consists of four dies, each corresponding to one NUMA, such as NUMA0~3 for the four dies. In this CPU architecture, the shared cache of each die is divided with finer granularity; that is, the three processor cores (C1~C3) within each die share an 8MB L3 cache. This architecture of three processor cores + L3 can be understood as a CPU core complex or core cluster (CCX), which is the most basic core module unit. It should be noted that the number of processor cores within a CCX can vary depending on the architecture.

[0055] According to embodiments of this application, the multiple dies of the storage controller can be divided into a first die, a second die, a third die, and a fourth die according to their functions, respectively implementing the host interaction layer, the cache layer, the redundant array of disks layer, and the disk management layer. It is understood that the first die, second die, third die, and fourth die can each be one or more, determined based on the resources consumed by each die in implementing its function. For example, functions that consume more resources can be implemented by multiple dies, such as multiple second dies. It should be clarified that if any of the above four functions is implemented by multiple dies, then the multiple dies implementing that function can be further divided into different sub-functions implementing that function.

[0056] The first die is configured to receive the data transmission instruction and perform protocol conversion on the data transmission instruction, and distribute the write instruction or the read instruction obtained after the protocol conversion. The second die is configured to, in the case of receiving the write instruction from the first processing unit, forward first data corresponding to the write instruction to the third processing unit, and / or, in the case of receiving the read instruction from the first processing unit, read second data corresponding to the read instruction from a shared cache of the second processing unit. The third die is configured to, in the case of receiving the first data, perform processing on the first data based on a Redundant Array of Independent Disks level to obtain a plurality of first blocks, and send the plurality of first blocks to the fourth processing unit. The fourth die is configured to write the plurality of first blocks to a disk.

[0057] In one embodiment, taking the HL, CA, RAID, and VL included in the I / O storage protocol stack as an example, the first die is configured to implement the function of the HL, the second die is configured to implement the function of the CA, the third die is configured to implement the function of the RAID, and the fourth die is configured to implement the function of the VL.

[0058] For example, the first die mainly undertakes the function of the HL, can receive the data transmission instruction sent by the host, and parse the data transmission instruction into a read instruction or a write instruction according to the corresponding transmission protocol (such as FC, iSCSI, etc.) configured. The read instruction or the write instruction is in the form of a protocol supported in the storage controller, and can be determined according to the actual CPU form. It can be understood that the read instruction or the write instruction is obtained according to the specific command in the data transmission instruction received from the host, that is, when the host has a read / write requirement, the first die obtains the corresponding read instruction / write instruction.

[0059] The second die mainly undertakes the cache function of the CA, and for the read instruction obtained by the first die through protocol conversion, the second data corresponding to the read instruction can be read from the L3 cache of the second die; conversely, for the write instruction obtained by the first die through protocol conversion, the write instruction and the first data corresponding thereto can be forwarded to the third die in the lower layer. It can be understood that if the first data corresponding to the write instruction is I / O hot data, the second die can also store the first data in the corresponding L3 cache, so as to directly read and return from the second die when the next read instruction reads the first data (the currently stored first data is regarded as the second data of the next read instruction).

[0060] The third die mainly undertakes the function of RAID. For the first data received from the second die and needing to be written to the disk, the third die can process the first data according to the processing logic of the disk redundancy array level to which the third die is configured, to obtain a plurality of first blocks. The disk redundancy array level is also referred to as the RAID level. The RAID level to which the third die is configured can be RAID 0, RAID 1, RAID 5, RAID 6, etc. The plurality of first blocks obtained after processing by the third die can be determined according to the RAID level. For example, the plurality of first blocks obtained based on RAID 5 processing can include data blocks obtained after data division and one parity block (referred to as a check block); and the plurality of first blocks obtained based on RAID 6 processing can include data blocks obtained after data division and two different parity blocks.

[0061] The fourth die is configured to undertake the VL function, and write the plurality of first blocks received from the third die to corresponding positions of the disk according to the corresponding physical addresses of the plurality of first blocks. The third die undertakes the RAID function and manages the disk through striping. There is a fixed mapping relationship between the strips and the physical addresses managed by the third die and the fourth die respectively. The third die carries the strip information of each of the plurality of first blocks when transmitting the plurality of first blocks, so that the fourth die determines the corresponding physical addresses of the plurality of first blocks based on the mapping relationship and performs a write operation.

[0062] In an embodiment of the present application, for the multi-die architecture of the storage controller, the I / O storage protocol stack is horizontally split, and the modules that undertake part of the functions after horizontal splitting are deployed in different dies. The entire storage controller is regarded as one instance implemented by software, and there are no multiple instances in the storage controller. There is no problem of insufficient isolation of master-slave instances, and there is no cross-die cache synchronization problem of the same module in the vertical direction of the vNode architecture and the multi-instance solution. Therefore, the deployment mode of the storage controller multi-die in the embodiment of the present application can at least partially overcome the cross-die cache synchronization problem.

[0063] In one specific embodiment, the multi-core processor is two, the first multi-core processor includes the first die, the second die and the fourth die; and the second multi-core processor includes the third die.

[0064] The CA and RAID modules implemented by the second die and the third die respectively are key modules in the I / O storage protocol stack, and the amount of tasks executed by the modules is large. For example, the RAID module accounts for about 40%, the CA module accounts for about 25%-30%, and the HL\VL accounts for about 10%. In the multi-CPU multi-die architecture, in order to avoid the problem of processing efficiency caused by improper resource allocation, the first die to the fourth die can be respectively deployed in different CPUs according to the amount of tasks executed by each die to implement functions.

[0065] For example, the third die which implements the most tasks can be deployed in one CPU alone, and the first die, the second die and the fourth die which respectively implement the remaining three functions can be deployed in another CPU. If there is a small module which implements other functions (with a small amount of tasks) in the I / O storage protocol stack, the fifth die which implements the small module can be deployed in the same CPU as the third die to fully utilize the CPU resources.

[0066] In the embodiments of the present application, the dies are deployed according to the amount of tasks occupied by the functions implemented by the dies in the multi-core multi-CPU architecture, so that the CPU resources can be reused. In addition, compared with the case that the CA and the RAID in the vNode and the multi-instance scheme communicate across sockets (across dies), the second die and the third die which implement the CA and the RAID are deployed separately in different CPUs, so that the cache synchronization overhead of the CA and the RAID across sockets is reduced, and the overall I / O performance of the storage controller is improved.

[0067] According to the embodiments of the present application, the multiple dies included in each multi-core processor perform cross-die communication based on a first protocol; the second die and the third die and the fourth die which are respectively arranged between two multi-core processors perform cross-processor cross-die communication based on a second protocol, and the first protocol and the second protocol are different. The first protocol can be a global memory interconnection (GMI) protocol, and the second protocol can be a socket-chip global memory interconnect (xGMI) protocol.

[0068] For example, in the multi-CPU multi-die architecture of the present application, the first die transmits a read / write instruction to the second die in the same CPU through the GMI protocol; after the second data is read in the cache of the second die, the second data is returned to the first die through the GMI protocol. Alternatively, the second die can transmit the first data corresponding to the write instruction to the third die through the xGMI protocol across the CPU and the dies, and then the third die sends multiple first blocks processed based on the first data to the fourth die. It can be understood that multiple dies in the same CPU can be interconnected through an on-chip bus. In the process of implementing I / O, if communication is required between some dies, the GMI protocol can be used for communication, and communication can not be performed between the remaining dies which do not need to communicate.

[0069] In one specific embodiment, the two CPUs can only include the first to fourth dies.

[0070] Table 1

[0071]

[0072] In another specific embodiment, in a multi-die architecture of two CPUs, the multi-die can be deployed as shown in Table 1 above. As shown in Table 1, the storage controller can include two 24-core multi-core processors, CPU0 and CPU1, each of which includes 4 dies, each of which includes 6 processor cores, and each of which can be identified as a NUMA, such as CPU0 including dies 0-3 and CPU1 including dies 4-7. Among them, die2 in CPU0 is the first die that undertakes the HL function, die0 and die1 are both the second die that undertakes the CA function, and die3 is the fourth die that undertakes the VL function; dies 4, 5, and 6 in CPU1 are the third die that undertakes the RAID function. In addition, die7 in CPU1 is also allocated to undertake other functions in the I / O storage protocol stack.

[0073] In the above embodiment, the processor core allocation scheme of the multi-CPU multi-die architecture is limited by the dual-CPU, and the four main function modules of the I / O storage protocol stack are deployed in different CPUs, which identifies the problem of the dual-CPU and reduces the cache synchronization overhead between the same function modules.

[0074] However, in the above implementation scheme, CA occupies 12 processor cores across dies 0 and 1, and RAID occupies 18 processor cores across dies 4-6. The CA and RAID still inevitably have cross-die cache synchronization overhead, so that the performance of the storage controller is still not excellent enough.

[0075] Therefore, the embodiments of the present application further functionally divide the multiple dies for implementing the same function, so that the multiple dies combine to implement the function, but the multiple dies are only used to implement part of the function under the function, and the operations performed by the multiple dies can be considered as pipeline operations / parallel operations under the function.

[0076] According to the embodiments of the present application, the second dies are two; the first second die is configured to, in a case where a read instruction is received from the first die, read second data corresponding to the read instruction from a shared cache of the second die; and the second second die is configured to, in a case where a write instruction is received from the first die, forward first data corresponding to the write instruction to a third die.

[0077] In this embodiment, the two second dies for implementing the CA function are divided into two different dies according to the read and write functions, one is used for operation according to the read instruction, and the other is used for operation according to the write instruction, that is, one is used to undertake the cache read operation, and the other is used to undertake the complete function of the remaining CA. The first second die can be referred to as a second die for implementing an upper cache (UCA), and the second second die can be referred to as a second die for implementing a lower cache (LCA).

[0078] In this embodiment, the two second dies for implementing the UCA and the LCA respectively are dedicated to the I / O data cache and read and write in the storage controller, the UCA shares the L3 cache in the first second die, and the LCA shares the L3 cache in the second second die. For example, the first die can send a read instruction to the first second die for implementing the UCA through inter-die communication, and read the second data from the corresponding L3 cache. The first die can send the corresponding first data to the first second die for implementing the LCA through inter-die communication, and the second second die can send the first data to the third die through inter-die communication, or temporarily write the first data into the corresponding L3 cache.

[0079] In a specific embodiment, the first second die can store I / O hot data and also store a plurality of data blocks in a prefetch mode; the second second die can directly send the first data to the third die, or delay sending the first data to the third die through a write back mode, and first temporarily write the first data into the corresponding L3 cache of the second second die.

[0080] In the embodiments of the present application, the two second dies for implementing the original CA function are deployed according to different functions of implementing read and write operations, so that the performance overhead caused by inter-die communication and data cache synchronization between the two second dies can be reduced as much as possible, and the two second dies can implement their respective read and write operations in parallel. Therefore, the embodiments of the present application can further horizontally split each functional module on the basis of horizontally splitting the I / O storage protocol stack function, thereby improving the overall I / O performance of the storage controller.

[0081] It can be understood that cache miss is also a common situation in actual use. Since the second second die is used to be responsible for functions other than read cache, if the first second die is cache-missed, the second second die can undertake the read operation of cache miss through inter-die communication.

[0082] According to an embodiment of the present application, the first second die is further configured to send the read instruction to the second second die in case that the second data corresponding to the read instruction is not read from the shared cache of the second die; the second second die is further configured to forward the read instruction to the third die in case that the read instruction is received from the first second die; the third die is further configured to determine the stripe information to which each data block for composing the second data belongs in case that the read instruction is received, and send the stripe information to the fourth die; and the fourth die is further configured to read each data block for composing the second data from the disk according to the stripe information from the third die.

[0083] In case that the first second die of the LCA misses, there is die-to-die communication between the two second dies, the first second die sends the read instruction to the second second die, and the second second die transparently transmits the read instruction to the third die. The read instruction can include the logical address (Logical Block Addressing, LBA) of the second data to be read after being divided into data blocks in the disk and the data length to be read. The third die can determine the logical address and the data length from the read instruction, determine the stripe information to which each data block for composing the second data belongs, and send the stripe information to the fourth die. Thus, the fourth die responsible for managing the disk can read each data block for composing the second data from the disk.

[0084] The third die can pre-establish a mapping relationship between the logical address and the stripe information according to the RAID level it is configured and / or the stripe it manages. Thus, the third die determines the stripe information of each data block for composing the second data according to the above mapping relationship after obtaining the LBA from the read request, and sends the above stripe information to the fourth die, so that the fourth die locates the physical address on the disk according to the stripe information and performs reading. After the fourth die reads each data block for composing the second data, it returns to the third die, and the third die recombines each data block into the second data and feeds back to the second second die; the second data is fed back to the host via the first die.

[0085] In the embodiments of the present application, the corresponding data is first read in the shared cache by the first second die for implementing the read operation, and in the case of cache miss in the first second die, the data is then read from the disk by the second second die, so as to minimize the overhead of directly reading data from the disk. In addition, the first second die does not need to interact with the third die at all, and the communication between the first second die and the second second die is only in the case of cache miss, thereby reducing the cross-die cache synchronization overhead in the overall storage controller.

[0086] In the implementation of the RAID function by the plurality of dies, the RAID of each processor core in each die is separated from the stripe managed by other RAIDs, but there is still a relatively large data sharing, which leads to a high cache synchronization overhead when sharing data across dies, especially in the scene of frequent I / O and modified I / O data, such as reading / writing data blocks or modifying the check blocks.

[0087] Therefore, according to one embodiment of the present application, in the case of at least two third dies, the plurality of first sub-blocks include a plurality of target data blocks obtained by splitting the first data and a check block for verifying the first data, the first third die is configured to: in the case of receiving the first data, split the first data to obtain a plurality of target data blocks; generate intermediate check data according to the plurality of target data blocks; and send the plurality of target data blocks to the fourth die to write the target data blocks into the disk by the fourth die; and the second third die is configured to: generate a check block for verifying the first data according to the intermediate check data from the first third die; and send the check block to the fourth die to write the check block into the disk by the fourth die.

[0088] The third die for implementing the RAID function can split the first data according to the RAID level it is configured to, and some RAID levels will generate corresponding check blocks after splitting the first data, such as RAID 5 and RAID 6. Therefore, the embodiments of the present application can split the original RAID function horizontally into a plurality of flows for performing different operations, such as splitting the operations of data processing and generating check blocks, and deploying them in different third dies. The first third die and the second third die are respectively a data processing branch and a check branch, and only the cross-die communication of the information required for generating the check block is involved between the two branches.

[0089] For example, the first third die can split the first data into a plurality of target data blocks according to the size of each block in the management stripe, and determine the stripe information of each target data block; then, the split target data blocks and their accompanying stripe information can be sent to the fourth die to realize the writing of the target data blocks. In addition, since the generation of the check block depends on the data of this writing, the first third die can also generate intermediate check data according to the plurality of target data blocks, and send the intermediate check data to the second third die to generate a check block according to the intermediate check data. Similarly, after the second third die generates the check block, it sends it to the fourth die, and the fourth die stores the check block in the same stripe as the target data block.

[0090] In the embodiments of the present application, the intermediate check data is the data communicated across the dies between the two third dies, the target data blocks split by the first third die are not synchronized across the dies to the second third die, and the check blocks generated by the second third die are not synchronized across the dies to the first third die, so that the transmission of the intermediate check data and the transmission of the modified data are only involved between the two third dies of the RAID, the cache synchronization overhead of the cross-die shared data is reduced, and the overall I / O performance of the storage controller is improved.

[0091] In one specific embodiment, if the third die is configured as RAID5, the first third die can be deployed as a data processing branch, and the second third die can be deployed as a check branch for generating one parity check block (referred to as P check block). Alternatively, in the case of more than two third dies, two or more third dies can be deployed as data processing branches, or two or more third dies can be deployed as check branches.

[0092] In another specific embodiment, the check blocks for verifying the first data are two, and the third dies are three. The second and third third dies are respectively configured to generate check blocks for verifying the first data according to the intermediate check data from the first third die, and send the respective generated check blocks to the fourth die to write the respective generated check blocks to the disk through the fourth die; wherein the check blocks generated by the second and third third dies are different.

[0093] It can be understood that in the case of generating two different check blocks, two different check branches are deployed on two different third dies for generating different check blocks. The operations performed by the second and third third dies can be similar, but they operate in parallel for different check blocks and do not involve cross-die transmission of check blocks between them.

[0094] For example, the third die is configured as RAID6, and two different check blocks are both parity check blocks, referred to as P check block and Q check block. At this time, the three third dies in the storage controller can be referred to as RAID-D, RAID-P, and RAID-Q branches respectively.

[0095] In one specific embodiment, the two-CPU multi-die architecture can include two second dies and three third dies, and is deployed as shown in Table 2 below.

[0096] The die numbers and processor cores in the two 24-core multi-core processors in Table 2 are similar to those in Table 1, and can be referred to Table 1. In the deployment scheme of Table 2, die2 in CPU0 is still the first die that undertakes the HL function, and die3 is still the fourth die that undertakes the VL function. The original CA service in Table 1 is horizontally divided into UCA and LCA, and is deployed in different second dies, such as die0 and die1; and the original RAID service in Table 1 is also horizontally divided into different RAID-D, RAID-P, and RAID-Q, which are respectively used for data processing, generating P check blocks, and generating Q check blocks.

[0097] Table 2

[0098]

[0099] In the embodiments of the present application, for the RAID level that needs to generate different check blocks, the data processing and the generation of different check blocks are respectively implemented by different third dies, so that only the first third die and the second and third third dies share cross-die data (intermediate check data), the generation of the two check blocks is completely parallel, and there is no cross-die communication between the two check blocks, which reduces the cache synchronization overhead of cross-die shared data and improves the overall I / O performance of the storage controller.

[0100] According to the embodiments of the present application, the first third die is further configured to: in the case that the first third die receives an updated write instruction from the second die, and the second and / or third third die receives intermediate check data, process the third data corresponding to the updated write instruction.

[0101] Different third dies are used to implement different RAID functions, and the operations implemented by multiple third dies can be regarded as pipeline operations of the RAID. Under the pipeline operations, after the first third die processes the first data and generates intermediate check data, the intermediate check data is sent to the second and third third dies, and the processing operation of the first third die on the current first data can be regarded as ending, and then the first third die can continue to receive the updated write instruction from the second die and process the new third data. At the same time, the second and third third dies process the intermediate check data of the first data, and the first, second and third third dies work simultaneously and in parallel.

[0102] For ease of understanding, the following will be described by taking a plurality of write instructions with a sequence as an example. For example, the first die continuously receives a plurality of data transmission instructions in a time sequence within a period of time, and the plurality of data transmission instructions are all converted into write instructions by the protocol, and the second die can send the plurality of write instructions to the third die in a time sequence. The data corresponding to each write instruction can be referred to as I / O data, and the plurality of I / O data arranged in a time sequence can be referred to as I / O1, I / O2, I / O3, …, and the data processed by the RAID-D branch, the RAID-P branch and the RAID-Q branch are simply referred to as I / O1-D, I / O1-P, I / O1-Q, I / O2-D, etc.

[0103] Taking the continuous reception of 8 I / O data as an example, the IO time sequence of the third die is shown in Table 3.

[0104] Each I / O data is processed in the first third die (RAID-D branch) first, and after the data processing is completed, the corresponding PQ check block generation task is generated and distributed to the second third die (RAID-P branch) and the third third die (RAID-Q branch) for implementation. As shown in Table 3, only at time 1, the first third die processes I / O1-D, and the second and third third dies do not process data, but at time 2-8 after that, the second and third third dies process the I / O data of the last time sequence of the first third die in parallel, such as time 2, the first third die processes I / O2-D, and the third third die processes I / O1-P and I / O1-Q in parallel, and the other time sequences are similar.

[0105] Table 3

[0106]

[0107] In the embodiments of the present application, the RAID function is realized through the pipeline among the three third dies, so that the CPU resources of each third die and each processor core in each third die are close to uniform, balanced and have high utilization through the operation mode of the pipeline.

[0108] The following will take two or three third dies to jointly realize the RAID function as an example to describe the three write types of RAID. It can be understood that if two third dies are used to jointly realize the RAID function, the plurality of first blocks can only include a plurality of target data blocks and one check block, the storage controller can include any one of the second and third third dies (the third die capable of realizing the RAID level) and the first third die. If three third dies are used to jointly realize the RAID function, the storage controller can include the above three third dies.

[0109] For ease of understanding, the following will take three third dies as an example to describe the three write types.

[0110] Figure 4 The corresponding schematic diagram of the relationship between the three write types and the stripe according to the embodiments of the present application is shown. As shown in Figure 4 Each processor core in each third die under the RAID function can manage a plurality of stripes, each stripe can be divided into 10 blocks, including 8 data blocks and 2 check blocks, such as D0~D7 data blocks, P check block and Q check block. Lowercase type refers to a write operation for a data block in a stripe, such as lowercase type for D0 data block; full write type refers to a write operation for D0~D7 data blocks (all data blocks in the stripe); uppercase type is between lowercase and full write types. For example, uppercase type can be for 2 data blocks, such as D0 and D1 data blocks, and so on, and can also be for 3…for 7 data blocks, such as D0~D6 data blocks.

[0111] According to the embodiments of the present application, the first third die is further configured to: in the case that the write type of the write instruction is the lowercase type for a single data block in the stripe, read the historical data block corresponding to the target data block from the target stripe through the fourth die, wherein the target stripe is the stripe corresponding to the write instruction; and based on the disk redundancy array level configured by the third die, perform logical operation on the historical data block and the target data block to obtain intermediate check data.

[0112] In this embodiment, the strip type can be determined according to the logical address and the data length of the write instruction. It can be understood that the RAID manages the disk through striping, and for the write instruction sent by the second die, the first third die can determine the write type and the target strip corresponding to the write instruction according to the logical address and the data length of the write instruction (the strip information in the foregoing includes the information of the target strip). Thus, the fourth die can read out the corresponding historical data block according to the received target strip and the correspondence between the strip and the physical address of the disk. Since each target strip also has the positions of the corresponding P parity block and Q parity block, the fourth die can also read the historical parity block (including the P historical parity block and the Q historical parity block) of the write instruction according to the target strip.

[0113] In this embodiment, since the generation of the parity block needs to use the historical data block, and the three third dies are divided into different dies according to their functions, and the first third die is used to process data, after the first third die cuts the first data into different target data blocks, the first third die also needs to read the historical data block corresponding to the target data block from the disk, and calculates the intermediate parity data from the historical data block and the target data block. The intermediate parity data can be understood as intermediate data related to the data and used for transmission between the second and third third dies.

[0114] It should be noted that there are various logics for calculating the parity block for various RAID levels, and thus the logic for determining the intermediate parity data according to the historical data block and the target data block can be determined according to the RAID level. For example, for RAID6, the historical data block and the target data block can be subjected to XOR operation to obtain the intermediate parity data.

[0115] In the embodiments of the present application, for the first third die used to process data, the reading of the historical data block and the calculation of the intermediate parity data related to the data are performed by the third die, so that the second and third third dies do not need to share the historical data block with the first third die, and for the write modification, the second and third third dies also do not need to consume resources to synchronize the historical data block in the cache, and only need to receive the intermediate parity data finally obtained by the write modification, thereby reducing the cross-die cache synchronization overhead.

[0116] According to the embodiments of the present application, the second and / or third third die is configured to receive the intermediate parity data sent by the first third die, and read the historical parity block from the target strip through the fourth die; perform logical operation on the historical parity block and the intermediate parity data to obtain the parity block based on the disk redundancy array level.

[0117] The same stripe stores both data blocks and parity blocks. Therefore, after determining the target stripe for the write instruction, the fourth die can read historical parity blocks from the target stripe containing the target data block. In this embodiment, for a storage controller with two third dies, one of the second or third third dies performs the parity block generation operation; for a storage controller with three third dies, both generate parity blocks, but the method by which they generate parity blocks can be determined according to the disk redundancy array level.

[0118] Figure 5 A flowchart illustrating the operation of lowercase types according to an embodiment of this application is shown. Figure 5 As shown, if the write type is lowercase, the first third die reads and writes the target data block ND to the read cache and reads the historical data block OD to the write cache. Based on the target data block ND and the historical data block OD, the intermediate parity data PD is determined as ND XOR OD. Furthermore, the first third die can also write the target data block ND to the disk corresponding to the target stripe through communication with the fourth die.

[0119] In this embodiment, reading and writing the target data block ND to the read cache is a fetch operation, which means reading the target data block after the first data split from the location of the split cache and writing it to the read cache, such as the Array Processing Unit (APU); at the same time, the historical data block OD can be read from the disk and written to the write cache, such as the Smart Cache Engine (SCE).

[0120] The second and third dies can read the historical parity block OP, calculate the new parity block NP = OP XOR PD, and write the new parity block NP to the disk corresponding to the target stripe through communication with the fourth die.

[0121] The third die reads the historical parity block OQ, calculates the new parity block NQ based on the intermediate parity data OP, and writes the new parity block NQ to the disk corresponding to the target stripe through communication with the fourth die. A complete RAID cycle ends after writing the target data block ND, parity block NP, and parity block NQ.

[0122] For example, for the PQ parity block supported by RAID6, the parity block NP, also known as the P parity block, can be obtained through an XOR operation; the parity block NQ, also known as the Q parity block, can be obtained through more complex operations, such as operations based on the Galois field.

[0123] In the embodiments of the present application, for the small write type with large write amplification, by transmitting the intermediate check data among the multiple third dies, the second and third third dies do not need to perform the operation of reading the history data block and do not need to consume resources to synchronize the history data block cached from the first third die, but only need to receive the intermediate check data, thereby reducing the cross-die cache synchronization overhead.

[0124] It should be noted that when performing the write operation on the target stripe, to achieve data consistency, the first third die can request the stripe lock for the target stripe in advance, and confirm whether the stripe lock is successfully obtained before preparing to write the target check block. If the stripe lock has been obtained, the write operation on the target stripe can be performed. The second and / or third third die writes the check block, and therefore, the stripe lock is released after the second and / or third third die successfully writes the check block.

[0125] The above has described the pipeline implementation of the two or three third dies in the embodiments of the present application. For example, the data block flow is implemented by the first third die, the P check block flow is implemented by the second third die, and the Q check block flow is implemented by the third third die. The following describes the specific operation of each implementation by taking the three third dies as an example.

[0126] Figure 6 An operation flowchart of the small write type according to one specific embodiment of the present application is shown. As shown in Figure 6 For each target data block, when the RAID small write operation needs to be performed, the RAID thread can be jumped to, the control block can be applied for, and the resources can be reserved. The RAID thread can be an FC-based fibre thread, which can apply for the TCB for tracking and managing the state of the write instruction, and at the same time, reserve the memory for storing the data, such as the history data block read from the disk, the history check block, and the intermediate check data and the new check block calculated. In addition to the memory, the resources also need to initialize the corresponding I / O structure, the initialized I / O structure (Standard IO) is referred to as sio, the stripe data element (SDE), and the I / O data packet (Package) is referred to as IPK. The above resources are used to build a complete I / O operation environment.

[0127] After the I / O operation environment is built, the target data block to be written can be obtained from a Volume Group (VG) Cache Line Buffer (CLB), and a history data block is read from a VL. The VL is a function implemented by the fourth die. After the history data block is read from the disk via the VL, a result (e.g., successful reading) is returned to the RAID thread. After the reading of the related data is completed, a stage 2 is started to process the data. For example, intermediate check data can be calculated according to the history data block and the target data block.

[0128] Subsequently, a stripe lock can be tested. If the test of the stripe lock indicates that the stripe lock has been acquired, the target data block can be written, and a P / Q check block process is started. If the stripe lock is not acquired, the stripe lock is waited for, and after the waiting is completed, the step of reading the target data block is returned to. In the data block process, the target data block is written through the VL, the disk is returned to the RAID thread, and a result (e.g., successful writing) is processed and returned. Related memory objects are released (actual return of the used memory to the memory pool) and deallocated, and thus the data IO process is completed.

[0129] The second and third P check block processes and Q check block processes implemented by the third dies are the same or similar, and the P check block process will be described below as an example.

[0130] After the intermediate check data is obtained from the data block process, a stage 1 of the P / Q check block is started, a history check block (P check block for the P check block process and Q check block for the Q check block process) is read from the VL, and then the disk is read and returned to the RAID thread, and a result is processed and returned. If the reading is successful, a stage 2 is started to calculate a new check block (the P check block process calculates a new P check block by using the calculation method of the P check block, and the Q check block process calculates a new Q check block by using the calculation method of the Q check block). Then, the check block is written, and the writing is implemented through the VL. After the disk is written and returned to the RAID thread, a result is processed and returned. Then, related memory objects are released and deallocated, the SDE is released, and the stripe lock is released. Thus, the P / Q check block process is completed. For the process including the P and Q check blocks, the stripe lock is released by any of the P / Q check block processes.

[0131] According to an embodiment of the present application, the first third die is further configured to, in a case where the write type of the write instruction is a full write type for a stripe, perform a logical operation on the split target data block to obtain intermediate check data, and send the intermediate check data to the second and / or third third die, so that the second and / or third third die performs a logical operation on the intermediate check data according to a Redundant Array of Independent Disks level to obtain a check block.

[0132] For full write types, since the target data blocks to be written constitute a complete stripe, there is no need to read historical data blocks and historical check blocks from the disk. The check block can be directly calculated based on multiple target data blocks. Multiple target data blocks and check blocks can overwrite the data within the original target stripe.

[0133] Figure 7 A flowchart illustrating the operation of a full-write type according to an embodiment of this application is shown. Figure 7 As shown, the first third die requests a stripe lock in advance when it starts processing the first data, and then segments the first data into multiple target data blocks. For each target data block, a fetch operation is performed, reading and writing the data block to the read cache (APU). For example, if n target data blocks are obtained, target data blocks 1 to n can be read and written to the read cache. If all target data blocks 1 to n are successfully read and written to the read cache, they can be written to disk; if any target data block fails to be successfully read and written to the read cache, it can wait for its successful read and write before writing all target data blocks to disk in batches. Figure 7 The verification of whether a stripe lock has been acquired is not shown in the figure. In the specific implementation, the write operation is performed on the target data block if all target data blocks 1 to n have been successfully read and written to the read cache and the stripe lock has been acquired.

[0134] If target data blocks 1 through n are successfully read and written to the read cache in the first third die, intermediate parity data can be calculated based on the n target data blocks. This intermediate parity data is then sent to the second and / or third third dies, whereby the second and / or third dies perform logical operations on the intermediate parity data according to the RAID level to calculate the parity block and write it to the disk. After writing the target data blocks and the parity block, a complete RAID cycle is finished.

[0135] In the embodiments of this application, for the full write type, the transmission of intermediate check data enables the second and third third dies to only receive the intermediate check data and directly use it to generate check blocks, reducing the cross-die cache synchronization overhead.

[0136] According to an embodiment of the present application, the first third die is further configured to: in a case where the write type of the write instruction is a large write type for a plurality of data blocks in a stripe, read, by the fourth die, a target historical data block from the target stripe, wherein the target historical data block is a remaining historical data block in the target stripe except for a historical data block corresponding to the target data block; perform a logical operation on the target historical data block and the target data block based on a redundant array of independent disks level to obtain intermediate check data, so as to send the intermediate check data to the second and / or third third die, so that the second and / or third third die performs a logical operation on the intermediate check data based on the redundant array of independent disks level to obtain a check block.

[0137] Figure 8 A scenario diagram of the large write type is shown according to an embodiment of the present application. For the large write type, as shown, Figure 8 The first data written by the write instruction is taken as an example for illustration, which is divided into D0' and D1' data blocks. The disk corresponding to the target stripe stores historical data blocks such as D0~D7 data blocks, wherein D0 and D1 data blocks are data blocks to be overwritten in this write operation. The first third die can read the target historical data blocks, i.e., D2~D7 data blocks, from the target stripe after the D0' and D1' data blocks are divided. Considering the write amplification of the large write type, the D0' and D1' data blocks and the read D2~D7 data blocks can be combined to form a complete stripe, and the intermediate check data is calculated in the full write type. The second and / or third third die also calculates the check block (such as P check block and Q check block) according to the intermediate check data in the full write type and writes it.

[0138] In an embodiment of the present application, for the large write type, the transmission of the intermediate check data can enable the second and third third dies to only receive the intermediate check data and directly use it to generate the check block, thereby reducing the cross-die cache synchronization overhead.

[0139] It should be noted that the operations implemented by the above storage controller or the operations implemented by the server including the storage controller can be referred to the above detailed description, and will not be described hereinafter.

[0140] Figure 9 A flowchart of a read-write method is shown according to an embodiment of the present application. As shown in Figure 9As shown, the data read-write method is applied to a server, the server comprising the storage controller, and the read-write method comprises: operation S910, at least one multi-core processor in the storage controller sequentially controls a plurality of processing units on a storage access path, receives a data transmission instruction, and performs data read-write on a disk according to the data transmission instruction; wherein the plurality of processing units on the storage access path are configured to: respectively implement ordered communication in a host interaction layer, a cache layer, a redundant array of independent disks layer, and a disk management layer according to a processing order of the data transmission instruction.

[0141] It can be understood that in the embodiments of the present application, the server can be provided with the storage controller, which can be regarded as a storage system implemented by at least one multi-core processor. The server can interface with various clients to implement storage or computing functions corresponding to each client. The server can implement read-write computing as one or more hosts (corresponding to the clients), and instructions corresponding to the read-write computing can be sent to the storage controller as data transmission instructions, so that the storage controller performs read-write on the disk according to the data transmission instructions. The operations implemented by the storage controller can be referred to the above, and will not be described here.

[0142] According to the embodiments of the present application, in the case of receiving the data transmission instruction sent by the server application layer, the data read-write on the disk comprises: the first processing unit performs protocol conversion on the data transmission instruction, and distributes the write instruction or the read instruction obtained after the protocol conversion; the second processing unit, in the case of receiving the write instruction from the first processing unit, forwards the first data corresponding to the write instruction to the third processing unit; and / or, in the case of receiving the read instruction from the first processing unit, reads the second data corresponding to the read instruction from the shared cache of the second processing unit; the third processing unit, in the case of receiving the first data, processes the first data based on the redundant array of independent disks level to obtain a plurality of first blocks, and sends the plurality of first blocks to the fourth processing unit; and the fourth processing unit writes the plurality of first blocks to the disk.

[0143] According to the embodiments of the present application, the storage controller comprises two multi-core processors, the first multi-core processor comprises the first processing unit, the second processing unit and the fourth processing unit; and the second multi-core processor comprises the third processing unit.

[0144] According to the embodiments of the present application, further comprising: the plurality of processing units included in the multi-core processors respectively perform cross-processing unit communication based on a first protocol; the second processing unit and the third processing unit and the fourth processing unit and the third processing unit respectively arranged between the two multi-core processors perform cross-processor cross-processing unit communication based on a second protocol, and the first protocol and the second protocol are different.

[0145] According to an embodiment of the present application, in the case that the second processing units are two; the method further comprises: in the case that the first second processing unit receives a read instruction from the first processing unit, reading second data corresponding to the read instruction from the shared cache of the second processing units; in the case that the second second processing unit receives a write instruction from the first processing unit, forwarding first data corresponding to the write instruction to the third processing unit.

[0146] According to an embodiment of the present application, in the case that the third processing units are three, the first data is processed based on a disk redundant array level to obtain a plurality of first blocks, and the plurality of first blocks are sent to the fourth processing unit, comprising: in the case that the first data is received by the first third processing unit, the first data is split to obtain a plurality of target data blocks; intermediate check data is generated according to the plurality of target data blocks; the plurality of target data blocks are sent to the fourth processing unit to write the target data blocks into the disk by the fourth processing unit; the second and third third processing units respectively generate check blocks for verifying the first data according to the intermediate check data from the first third processing unit; and the respective generated check blocks are sent to the fourth processing unit to write the respective generated check blocks into the disk by the fourth processing unit, wherein the check blocks generated by the second and third third processing units are different.

[0147] According to an embodiment of the present application, the present application further provides a server, comprising: a disk; a storage controller comprising at least one multi-core processor, each of the at least one multi-core processor comprising a plurality of processing units connected to each other; the at least one multi-core processor is configured to: sequentially control a plurality of processing units on a storage access path, receive a data transmission instruction and perform data read and write on the disk according to the data transmission instruction; wherein the plurality of processing units on the storage access path are configured to: respectively implement ordered communication in a host interaction layer, a cache layer, a disk redundant array layer and a disk management layer according to the processing order of the data transmission instruction. The operations implemented by the storage controller can refer to the above, and will not be described here.

[0148] According to embodiments of the present application, program code for implementing the computer programs provided by embodiments of the present application can be written in any combination of one or more programming languages, and can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. The programming language includes, but is not limited to, such languages as Java, C++, python, "C" language, or the like. The program code can execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the remote computing device, or entirely on the remote computing device or server. In the latter scenario, the remote computing device can be connected to the user's computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computing device, such as through the Internet using an Internet Service Provider.

[0149] The computer program instructions can also be loaded onto a computer or other programmable information processing apparatus to cause a series of operations to be performed on the computer or other programmable information processing apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable information processing apparatus implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0150] Those skilled in the art will understand that features recited in the various embodiments of the present application can be combined and / or integrated in a variety of ways, even if such combinations or integrations are not expressly noted in the present application. In particular, features recited in the various embodiments of the present application can be combined and / or integrated in ways that do not depart from the spirit and scope of the present application. All such combinations and / or integrations are within the scope of the present application.

[0151] The embodiments of the present application have been described above. However, these embodiments are merely for the purpose of illustration and are not intended to limit the scope of the present application. Although the embodiments are described separately above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. Those skilled in the art can make various substitutions and modifications without departing from the scope of the present application, and these substitutions and modifications shall fall within the scope of the present application.

Claims

1. A storage controller, characterized in that, It includes at least one multi-core processor, and each of the at least one multi-core processor includes a plurality of processing units interconnected with each other; The at least one multi-core processor is configured to: sequentially control multiple processing units located on the storage access path, receive data transmission instructions, and perform data read and write operations on the disk according to the data transmission instructions; The multiple processing units on the storage access path are configured to communicate in an orderly manner through the host interaction layer, cache layer, disk redundant array layer and disk management layer, respectively, according to the processing order of the data transmission instructions.

2. The storage controller according to claim 1, characterized in that, The processing unit includes a shared cache for storing data received or processed by the processing unit; The multiple processing units located on the storage access path include: a first processing unit, a second processing unit, a third processing unit, and a fourth processing unit configured to respectively implement the host interaction layer, the cache layer, the redundant array of disks layer, and the disk management layer; The first processing unit is configured to: receive the data transmission instruction and perform protocol conversion on the data transmission instruction; and distribute the write instruction or read instruction obtained after protocol conversion. The second processing unit is configured to: upon receiving the write instruction from the first processing unit, forward the first data corresponding to the write instruction to the third processing unit; and / or, upon receiving the read instruction from the first processing unit, read the second data corresponding to the read instruction from the shared cache of the second processing unit; The third processing unit is configured to: upon receiving the first data, process the first data based on the disk redundancy array level to obtain multiple first blocks, and send the multiple first blocks to the fourth processing unit; The fourth processing unit is configured to write multiple of the first blocks to the disk.

3. The storage controller according to claim 2, characterized in that, The multi-core processor comprises two units: the first multi-core processor includes the first processing unit, the second processing unit, and the fourth processing unit; the second multi-core processor includes the third processing unit.

4. The storage controller according to claim 3, characterized in that, The multi-core processor includes multiple processing units that communicate across processing units based on a first protocol; The second processing unit and the third processing unit, as well as the fourth processing unit and the third processing unit, which are respectively located between the two multi-core processors, communicate across processors and processing units based on a second protocol. The first protocol and the second protocol are different.

5. The storage controller according to claim 2, characterized in that, The second processing unit consists of two units; The first second processing unit is configured to: upon receiving the read instruction from the first processing unit, read second data corresponding to the read instruction from the shared cache of the second processing unit; The second processing unit is configured to: upon receiving the write instruction from the first processing unit, forward the first data corresponding to the write instruction to the third processing unit.

6. The storage controller according to claim 5, characterized in that, The first second processing unit is further configured to: send the read instruction to the second second processing unit if the second data corresponding to the read instruction is not read from the shared cache of the second processing unit; The second second processing unit is further configured to: upon receiving the read instruction from the first second processing unit, forward the read instruction to the third processing unit; The third processing unit is further configured to: upon receiving the read instruction, determine the stripe information to which each data block constituting the second data belongs, and send the stripe information to the fourth processing unit; The fourth processing unit is further configured to read from the disk, based on the stripe information from the third processing unit, the data blocks used to compose the second data.

7. The storage controller according to claim 2, characterized in that, The third processing unit comprises at least two units, and the plurality of first blocks include a plurality of target data blocks obtained by segmenting the first data and a verification block for verifying the first data. The first third processing unit is configured to: upon receiving the first data, segment the first data to obtain multiple target data blocks; Intermediate verification data is generated based on multiple target data blocks; the multiple target data blocks are sent to the fourth processing unit so that the target data blocks are written to the disk by the fourth processing unit; The second third processing unit is configured to: generate a verification block for verifying the first data based on the intermediate verification data from the first third processing unit; and send the verification block to the fourth processing unit so that the verification block is written to the disk by the fourth processing unit.

8. The storage controller according to claim 7, characterized in that, The verification blocks used to verify the first data consist of two parts; the third processing unit consists of three parts. The second and third third processing units are respectively configured to: generate a verification block for verifying the first data based on the intermediate verification data from the first third processing unit; and send the respective generated verification blocks to the fourth processing unit so that the respective generated verification blocks are written to the disk by the fourth processing unit. The second and third verification blocks generated by the third processing unit are different.

9. The storage controller according to claim 7 or 8, characterized in that, The first third processing unit is further configured to process the third data corresponding to the updated write instruction when the first third processing unit receives an updated write instruction from the second processing unit, and the second and / or third third processing units receive the intermediate verification data.

10. The storage controller according to claim 7 or 8, characterized in that, The first third processing unit is further configured to: When the write type of the write instruction is lowercase for a single data block within a stripe, the fourth processing unit reads the historical data block corresponding to the target data block from the target stripe, wherein the target stripe is the stripe corresponding to the write instruction; and Based on the disk redundancy array level configured for the third processing unit, logical operations are performed on the historical data block and the target data block to obtain the intermediate verification data.

11. The storage controller according to claim 10, characterized in that, The second and / or third third processing unit is configured to: receive the intermediate verification data sent by the first third processing unit, and read historical verification blocks from the target strip through the fourth processing unit; Based on the disk redundancy array level, logical operations are performed on the historical check block and the intermediate check data to obtain the check block.

12. The storage controller according to claim 7 or 8, characterized in that, The first third processing unit is further configured to: when the write type of the write instruction is a full write type for stripes, perform logical operations on the segmented target data block to obtain the intermediate verification data, and send the intermediate verification data to the second and / or the third third processing unit, so that the second and / or the third third processing unit performs logical operations on the intermediate verification data according to the disk redundancy array level to obtain the verification block.

13. The storage controller according to claim 7 or 8, characterized in that, The first third processing unit is further configured to: When the write type of the write instruction is uppercase for multiple data blocks within a stripe, the fourth processing unit reads the target historical data block from the target stripe, wherein the target historical data block is the remaining historical data block within the target stripe excluding the historical data block corresponding to the target data block; Based on the disk redundancy array level, logical operations are performed on the target historical data block and the target data block to obtain the intermediate verification data. The intermediate verification data is then sent to the second and / or third third processing unit, so that the second and / or third third processing unit performs logical operations on the intermediate verification data according to the disk redundancy array level to obtain the verification block.

14. A data read / write method, characterized in that, Applied to a server, the server including a storage controller as described in any one of claims 1 to 13, The read / write method includes: The storage controller contains at least one multi-core processor that sequentially controls multiple processing units located on the storage access path, receives data transmission instructions, and performs data read and write operations on the disk according to the data transmission instructions. The multiple processing units on the storage access path are configured to communicate in an orderly manner through the host interaction layer, cache layer, disk redundant array layer and disk management layer, respectively, according to the processing order of the data transmission instructions.

15. The method according to claim 14, characterized in that, The step of receiving data transmission instructions and performing data read / write operations on the disk according to the data transmission instructions includes: The first processing unit receives the data transmission instruction and performs protocol conversion on the data transmission instruction; it then distributes the write or read instructions obtained after the protocol conversion. When the second processing unit receives the write instruction from the first processing unit, it forwards the first data corresponding to the write instruction to the third processing unit; and / or, when it receives the read instruction from the first processing unit, it reads the second data corresponding to the read instruction from the shared cache of the second processing unit. Upon receiving the first data, the third processing unit processes the first data based on the disk redundancy array level to obtain multiple first blocks, and sends the multiple first blocks to the fourth processing unit. The fourth processing unit writes multiple of the first blocks to the disk.

16. The method according to claim 15, characterized in that, The storage controller includes two multi-core processors. The first multi-core processor includes the first processing unit, the second processing unit, and the fourth processing unit. The second multi-core processor includes the third processing unit.

17. The method according to claim 16, characterized in that, Also includes: The multi-core processor includes multiple processing units that communicate across processing units based on a first protocol; The second processing unit and the third processing unit, as well as the fourth processing unit and the third processing unit, which are respectively located between the two multi-core processors, communicate across processors and processing units based on a second protocol. The first protocol and the second protocol are different.

18. The method according to claim 15, characterized in that, When there are two second processing units, the method further includes: When the first second processing unit receives the read instruction from the first processing unit, it reads the second data corresponding to the read instruction from the shared buffer of the second processing unit. When the second processing unit receives the write instruction from the first processing unit, it forwards the first data corresponding to the write instruction to the third processing unit.

19. The method according to claim 15, characterized in that, When there are three third processing units, the first data is processed based on the disk redundancy array level to obtain multiple first blocks, and the multiple first blocks are sent to the fourth processing unit, including: Upon receiving the first data, the first third processing unit segments the first data to obtain multiple target data blocks; generates intermediate verification data based on the multiple target data blocks; and sends the multiple target data blocks to the fourth processing unit so that the target data blocks can be written to the disk by the fourth processing unit. The second and third third processing units respectively generate verification blocks for verifying the first data based on the intermediate verification data from the first third processing unit; and send the respective generated verification blocks to the fourth processing unit so that the fourth processing unit can write the respective generated verification blocks to the disk, wherein the verification blocks generated by the second and third third processing units are different.

20. A server, characterized in that, include: disk; A storage controller includes at least one multi-core processor, each of the at least one multi-core processor including a plurality of processing units interconnected with each other; The at least one multi-core processor is configured to: sequentially control multiple processing units located on the storage access path, receive data transmission instructions, and perform data read and write operations on the disk according to the data transmission instructions; The multiple processing units on the storage access path are configured to communicate in an orderly manner through the host interaction layer, cache layer, disk redundant array layer and disk management layer, respectively, according to the processing order of the data transmission instructions.

Images