Disk storage systems, methods and servers
By integrating the cache into the main controller chip, the problem of DRAM chips occupying space on the disk PCB board is solved, achieving more efficient data processing and improved read/write performance.
Patent Information
- Application Number
- CN202310572457.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-19
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2043-05-19
AI Technical Summary
In existing technologies, placing a large number of DRAM chips in the PCB board of a hard disk leads to a waste of space resources and restricts the read and write performance of the hard disk.
By integrating the cache into the main controller chip, data processing operations can be performed directly in the cache and/or disk through the main controller chip, reducing the space occupied by DRAM chips on the PCB board and enabling parallel execution of data processing tasks.
It effectively reduces data processing latency and information loss, improves disk data read and write performance, and reduces space resource waste and cabling costs.
Smart Images

Figure CN116774925B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer technology, and in particular to a disk storage system, method, and server. Background Technology
[0002] The cache is where the disk exchanges data with the external bus. The disk first transfers data to the cache, and then the cache exchanges data with the external data bus. The cache is a storage chip on the disk's circuit board, offering extremely fast access speeds compared to the hard drive platters. The size and speed of the cache are crucial factors directly related to the actual disk transfer speed; a large cache capacity can significantly improve the data hit rate, thereby improving the overall disk performance. Therefore, improving cache performance is a key factor in improving the overall read and write performance of the disk.
[0003] Typically, disk caches are implemented using Dynamic Random Access Memory (DRAM) chips, with each DRAM chip providing 2GB or 4GB of cache. Therefore, to improve cache performance, a large number of DRAM chips are usually added to the disk's printed circuit board (PCB). However, adding a large number of DRAM chips occupies significant space, wasting disk space resources, making the disk board increasingly bulky, increasing data processing time and information loss during data transmission, and ultimately limiting disk read / write performance. Summary of the Invention
[0004] This invention provides a disk storage system, method, and server to address the shortcomings of existing technologies where a large number of DRAM chips are placed on the PCB board of the disk, which occupies a lot of space and wastes disk space resources, thus restricting disk read and write performance. The invention aims to reduce the waste of disk space resources while improving disk data read and write performance.
[0005] The present invention provides a disk storage system, including a disk, a cache of the disk, and a main controller chip of the disk;
[0006] The main controller chip is externally connected to the disk, and the cache is integrated into the main controller chip;
[0007] The main controller chip is used to receive data processing instructions, obtain data to be processed and data processing operations for the data to be processed according to the data processing instructions, and perform the data processing operations on the data to be processed in the cache and / or the disk according to the storage information of the cache; wherein, the data processing operations include data reading operations and / or data writing operations.
[0008] According to a disk storage system provided by the present invention, the cache includes a plurality of cache blocks;
[0009] The main controller chip is specifically used for:
[0010] If the cache is determined to be the destination memory based on the storage information, the data to be processed is divided into multiple stripes; each stripe includes multiple sub-data blocks; the destination memory is used to provide storage space for data reading and writing of the data to be processed.
[0011] Based on multiple cache blocks, the data processing operation is performed in parallel on each of the sub-data blocks in the cache.
[0012] According to a disk storage system provided by the present invention, the cache is pre-stored with a set-association cache mapping table, and the set-association cache mapping table stores the association relationship between each set and each cache block;
[0013] The main controller chip is also used for:
[0014] Each of the aforementioned sub-data blocks is divided into a corresponding set;
[0015] Based on the set identifier of the corresponding set and the association relationship, the target cache block corresponding to each of the sub-data blocks is determined in the multiple cache blocks;
[0016] Based on the target cache block corresponding to each of the sub-data blocks, the data processing operation is performed in parallel on each of the sub-data blocks in the cache.
[0017] According to a disk storage system provided by the present invention, the main controller chip is further configured to:
[0018] When the data processing operation includes a data writing operation, the target data specification of the data to be processed is obtained;
[0019] Based on the storage information, determine the remaining storage space of the cache;
[0020] Based on the remaining storage space and the target data specifications, the data processing operation is performed on the data to be processed in the cache and / or the disk.
[0021] According to a disk storage system provided by the present invention, the main controller chip is further configured to:
[0022] Based on the remaining storage space, determine the maximum writable data size of the cache;
[0023] If the target data specification is greater than the maximum writable data specification, the cache and the disk are determined as the destination storage.
[0024] Based on the target data specification and the maximum writable data specification, the data to be processed is divided into data blocks to obtain a first data block and a second data block.
[0025] A data write operation is performed on the first data block in the buffer;
[0026] Perform a data write operation on the second data block in the disk;
[0027] Wherein, the data specification of the first data block is less than or equal to the maximum writable data specification; the second data block is a data block generated based on the data in the data to be processed, excluding the data in the first data block.
[0028] According to a disk storage system provided by the present invention, the main controller chip is further configured to:
[0029] If the data processing operation includes a data reading operation, determine the storage address of the data to be processed;
[0030] If, based on the storage address and the storage information, it is determined that both the disk and the cache contain the data to be processed, the cache is determined to be the destination memory.
[0031] The data to be processed is read from the cache according to the storage address.
[0032] According to the present invention, a disk storage system is provided, wherein the disk includes multiple hard disks;
[0033] Multiple hard drives are combined to form a disk array.
[0034] The present invention also provides a disk storage method, applied to the disk storage system described in any of the above claims, comprising:
[0035] Receive data processing instructions;
[0036] The data to be processed and the data processing operation of the data to be processed are obtained according to the data processing instructions.
[0037] Based on the storage information of the cache integrated on the main controller chip of the disk, the data processing operation is performed on the data to be processed in the cache and / or the disk.
[0038] The data processing operations include data reading operations or data writing operations.
[0039] According to a disk storage method provided by the present invention, the step of performing the data processing operation on the data to be processed in the cache and / or the disk based on the storage information of the cache integrated on the main controller chip of the disk includes:
[0040] Based on the storage information, a destination memory is determined in the cache and the disk; the destination memory is used to provide storage space for data reading and writing of the data to be processed.
[0041] If the cache is determined to be the destination memory, the data to be processed is divided into multiple stripes; each stripe includes multiple sub-data blocks;
[0042] Each of the aforementioned sub-data blocks is divided into a corresponding set;
[0043] Based on the set identifier of the corresponding set and the association relationship between each set and each cache block stored in the set association cache mapping table, the target cache block corresponding to each sub-data block is determined in the multiple cache blocks of the cache; the set association cache mapping table is pre-stored in the cache;
[0044] Based on the target cache block corresponding to each of the sub-data blocks, the data processing operation is performed in parallel on each of the sub-data blocks in the cache.
[0045] The present invention also provides a server, including a disk storage system and a processor as described in any of the above descriptions;
[0046] The processor is connected to the disk storage system and is used to send data processing instructions to the disk storage system.
[0047] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement any of the disk storage methods described above.
[0048] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the disk storage method described above.
[0049] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements any of the disk storage methods described above.
[0050] The disk storage system, method, and server provided by this invention integrate the cache into the main controller chip connected to the disk. The main controller chip directly performs data processing operations on the data to be processed in the cache and / or disk. This not only reduces the space resources required for a large number of DRAM chips on the upper part of the disk PCB board and reduces wiring costs, but also allows the main controller chip to directly control the cache to perform data processing operations locally. This effectively reduces the data processing latency and information loss caused by the main controller chip remotely accessing the disk and cache, and greatly improves the data read and write performance of the disk. Attached Figure Description
[0051] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.
[0052] Figure 1 This is a schematic diagram of the disk storage system provided by the present invention;
[0053] Figure 2 This is one of the flowcharts of the disk storage method provided by the present invention;
[0054] Figure 3 This is the second flowchart illustrating the disk storage method provided by the present invention;
[0055] Figure 4 This is a schematic diagram of the structure of the electronic device provided by the present invention.
[0056] Figure label:
[0057] 1: Disk storage system; 101: Disk; 102: Cache; 103: Main controller chip; 2: External device. Detailed Implementation
[0058] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.
[0059] The disk cache is a crucial component for improving disk read and write performance. Most disks rely on caches to enhance performance. However, with disk upgrades, such as the updates to the Peripheral Component Interconnect (PCIe) protocol and the Serial Attached SCSI (SAS) protocol, the overall data processing capacity of the disk has increased exponentially. This makes it difficult for existing cache capacities to support massive data flows. To better maintain the input / output (I / O) performance advantages of the hardware disk, a larger cache capacity is needed to provide better data processing performance.
[0060] Typically, a single DRAM chip provides 2GB or 4GB of cache, while current Gen4 RAID cards offer up to 32GB of cache. Consequently, a large number of DRAM chips need to be added to the disk's PCB board. This results in the DRAM chips used for caching occupying a significant amount of space on the PCB board, leading to a waste of space resources. This makes the disk board increasingly bulky, increases the data processing time of the cache, and causes information loss during data transmission, thereby limiting the disk's data read and write performance.
[0061] To address the aforementioned technical problems, this embodiment provides a disk storage system, method, and server. In this disk storage system, the disk's main controller chip is externally connected to the disk, and the disk's cache is integrated into the main controller chip. This allows the main controller chip to directly perform data processing operations on the data to be processed in the cache and / or the disk based on data processing instructions sent by external devices. On one hand, integrating the cache into the main controller chip reduces the space occupied on the disk's PCB board and lowers wiring costs. On the other hand, the main controller chip can directly control the cache to perform data processing operations locally, effectively reducing the data access latency caused by the main controller chip accessing the disk first, and then processing data in the cache only after accessing the disk, thus significantly improving the disk's data read / write performance.
[0062] The following is combined with Figure 1-4 The present invention describes a disk storage system, method, and server.
[0063] like Figure 1 The diagram shown is a structural schematic of the disk storage system provided in this embodiment. The disk storage system 1 includes at least a disk 101, a cache 102 of the disk 101, and a main controller chip 103 of the disk 101.
[0064] The main controller chip 103 is externally connected to the disk 101, and the cache 102 is integrated into the main controller chip 103;
[0065] The disk 101 referred to may be constructed from a single hard drive or from multiple hard drives; this embodiment does not specifically limit it in this way.
[0066] In some embodiments, the disk 101 includes multiple hard disks;
[0067] Multiple hard drives are combined to form a disk array.
[0068] Optionally, if disk 101 includes multiple independent hard drives, the multiple independent hard drives can be combined to form a Redundant Array of Independent Disks (RAID) to significantly improve the data throughput of the storage system by storing and reading data simultaneously through multiple hard drives, thereby improving the read and write performance of the disk.
[0069] The so-called main controller chip 103 is an external controller chip located on the disk 101, interconnected with the disk 101 via an independent bus. This main controller chip 103 could be a host controller chip, etc., but this embodiment does not specifically limit this. Accordingly, the main controller chip 103 can be connected to the disk 101 via a PCI Express (peripheral component interconnect express, high-speed serial computer expansion bus standard) bus, providing high-speed data transmission. Through this connection, the disk 101 can provide high-speed data transmission and processing capabilities, thereby achieving efficient data storage and protection, and improving the performance and reliability of the storage system.
[0070] The so-called Host controller chip is a control chip used in a host or server system to control I / O interfaces and devices. It can be a standalone integrated circuit or a modular chip assembly. The Host controller chip is mainly responsible for interacting with the system bus and related devices, coordinating data transmission and managing control signals. In this disk storage system 1, the Host controller chip is used to communicate and control the disk 101 and the cache 102 of the disk 101.
[0071] The cache 102 of disk 101 can be a high-speed cache memory (also called a cache), which is the place where disk 101 exchanges data with the external bus, and can realize both data reading and data writing. The cache 102 includes at least one image file to protect the data security in the cache 102. The cache 102 is directly integrated into the main controller chip 103.
[0072] The main controller chip 103 is used to receive data processing instructions sent by the external device 2, obtain the data to be processed and the data processing operation of the data to be processed according to the data processing instructions, and perform the data processing operation on the data to be processed in the cache 102 and / or the disk 101 according to the storage information of the cache 102; wherein, the data processing operation includes data reading operation and / or data writing operation.
[0073] The data to be processed may be video data, images, table data, text data, etc., and this embodiment does not specifically limit it.
[0074] The storage information here includes at least the data content stored in cache 102 and the storage space usage.
[0075] External device 2 here can be other devices such as a central processing unit (CPU) connected to disk 101.
[0076] Optionally, the main controller chip 103 is used to listen to the data processing instructions sent by the external device 2 in real time. When a data processing instruction is received, the data processing instruction can be parsed to obtain the data to be processed and the data processing operation of the data to be processed.
[0077] Then, based on the storage information of the buffer 102, the destination memory is determined in the buffer 102 and the disk 101, that is, the memory used to provide storage space for data reading and writing of the data to be processed is determined in the buffer 102 and the disk 101, so that data processing operations can be performed on the data to be processed in the destination memory.
[0078] Here, the method for determining the destination storage can be based on different types of data processing operations. Different information is retrieved from the storage information, namely historical storage data or storage space occupancy. The destination storage is then determined based on this historical data or storage space occupancy. For example, if the data processing operation is a data read operation, historical storage data is retrieved from the storage information. Based on this historical data, it is first determined whether the data to be processed can be retrieved from the cache 102, and then whether the cache 102 is the destination storage. Only if the cache 102 is not the destination storage is it further determined whether the disk 101 can be used as the destination storage. Similarly, if the data processing operation is a data write operation, storage space occupancy is retrieved from the storage information. Based on this storage space occupancy, it is first determined whether the cache 102 has sufficient memory space for data writing, and then whether the cache 102 is the destination storage. Only if the cache 102 is not the destination storage is it further determined whether the disk 101 can be used as the destination storage.
[0079] Compared to related technologies that improve disk read / write performance by adding a large amount of DRAM to the disk's PCB, this embodiment integrates the cache directly into the main controller chip. This avoids the complex PCB layout and wasted space caused by multiple DRAMs, effectively reducing the PCB space occupied and minimizing space waste and interference from PCB wiring. This ensures a lighter overall disk with less wasted space and avoids various problems caused by overly complex wiring. Furthermore, the direct integration of the cache with the main controller chip brings it closer to the chip, allowing the main controller chip to access the cache directly. This makes the cache easier to access, reducing remote interactions between the main controller chip, the disk, and the cache, improving cache utilization efficiency, and thus effectively improving disk read / write performance. Additionally, it avoids data loss caused by the disk's supercapacitors failing to provide adequate data protection due to a large amount of DRAM, effectively ensuring data security.
[0080] The disk storage system provided in this embodiment integrates the cache into the main controller chip connected to the disk. The main controller chip directly performs data processing operations on the data to be processed in the cache and / or disk. This not only reduces the space resources required for a large number of DRAM chips on the upper part of the disk PCB board and reduces wiring costs, but also allows the main controller chip to directly control the cache to perform data processing operations locally. This effectively reduces the data processing latency and information loss caused by the main controller chip remotely accessing the disk and cache, and greatly improves the data read and write performance of the disk.
[0081] In some embodiments, the cache 102 includes a plurality of cache blocks;
[0082] The main controller chip 103 is specifically used for:
[0083] If the cache 102 is determined to be the destination memory based on the storage information, the data to be processed is divided into multiple stripes; each stripe includes multiple sub-data blocks; the destination memory is used to provide storage space for data reading and writing of the data to be processed.
[0084] Based on multiple cache blocks, the data processing operation is performed in parallel on each of the sub-data blocks in the cache 102.
[0085] The cache 102 is pre-configured with multiple cache blocks, each of which has data read and data write functions.
[0086] Optionally, when the main controller chip 103 determines that the buffer 102 is the destination memory based on the storage information, it indicates that the buffer 102 can provide storage space for data reading and writing of the data to be processed. At this time, the data to be processed can be divided into multiple stripes to divide the data to be processed into multiple data blocks.
[0087] Here, the partitioning method can be based on the data size of the cache block and the processing performance of each cache block, so that the partitioned data blocks are compatible with the processing performance of the cache blocks; or it can be based on a pre-defined data specification, such as 16KB, etc. This example does not make a specific limitation on this.
[0088] Next, the main controller chip 103 divides the processing tasks of each sub-data block into corresponding cache blocks so that multiple cache blocks can perform data reading and data writing operations on each sub-data block in parallel in the cache 102.
[0089] In related technologies, a cache can only perform data reading or data writing tasks at the same time. That is, when a data reading task is being performed in the cache, a data reading task cannot be performed, which leads to waiting delays in data processing and waste of cache resources.
[0090] In this embodiment, multiple cache blocks are used to perform data reading or writing operations on each sub-data block in the data to be processed in parallel. This allows data reading tasks to be performed in parallel on some cache blocks, while data writing operations can be performed synchronously on other cache blocks. This effectively reduces waiting latency and cache resource waste, thereby improving the utilization rate of cache resources and increasing read and write speed.
[0091] In some embodiments, the cache 102 pre-stores a set-associated cache mapping table, which stores the association relationship between each set and each cache block;
[0092] The main controller chip 103 is also used for:
[0093] Each of the aforementioned sub-data blocks is divided into a corresponding set;
[0094] Based on the set identifier of the corresponding set and the association relationship, the target cache block corresponding to each of the sub-data blocks is determined in the multiple cache blocks;
[0095] Based on the target cache block corresponding to each of the sub-data blocks, the data processing operation is performed in parallel on each of the sub-data blocks in the cache 102.
[0096] The cache mapping relationship of cache 102 is represented by a set-associative cache mapping table. This set-associative cache mapping table is pre-stored in cache 102 and is used to represent the association relationship between each set and each cache block. In this cache mapping relationship, cache blocks are grouped to form a set, and main memory blocks can enter any cache block of a specific set.
[0097] Optionally, the main controller chip 103 can sequentially divide each sub-data block into the corresponding set according to the number of sub-data blocks that each set can process, or it can divide each sub-data block into the corresponding set according to the association relationship between each sub-data block and each set. This embodiment does not specifically limit this.
[0098] Then, based on the set identifier of each sub-data block and the association between each set and each cache block, the main controller chip 103 determines the target cache block corresponding to each sub-data block from among multiple cache blocks. That is, the cache block used to process the data processing tasks of each sub-data block. In this way, the corresponding target cache block can be allocated to each sub-data block for parallel data processing more accurately and conveniently, thereby improving data read and write performance.
[0099] Next, the main controller chip 103 divides the processing tasks of each sub-data block into their respective target cache blocks, so that the target cache blocks corresponding to each sub-data block can perform data reading and data writing operations on each sub-data block in parallel in the cache 102.
[0100] In this embodiment, by dividing each sub-data block into a corresponding set, and allocating a corresponding target cache block to each sub-data block according to the association between each set and each cache block, the target cache blocks corresponding to each sub-data block can perform data writing operations synchronously and in parallel, which can effectively improve the utilization of cache resources and improve read and write speed.
[0101] In some embodiments, the main controller chip 103 is further configured to:
[0102] When the data processing operation includes a data writing operation, the target data specification of the data to be processed is obtained;
[0103] Based on the storage information, determine the remaining storage space of the cache 102;
[0104] Based on the remaining storage space and the target data specifications, the data processing operation is performed on the data to be processed in the cache 102 and / or the disk 101.
[0105] Optionally, when data processing operations include data writing operations, the main controller chip 103 needs to obtain the target data specifications of the data to be processed; and obtain the storage space occupancy information from the storage information, so as to further determine the remaining storage space of the cache 102 based on the storage space occupancy information, so as to determine whether the cache 102 has sufficient storage space to write the data to be processed based on the remaining storage space and the target data specifications, and then determine the target memory in the cache 102 and the disk 101 based on the determination result; when the cache 102 is the target cache 102, the data processing operation can be performed on the data to be processed in the cache 102; when the disk 101 is the target cache 102, the data processing operation can be performed on the data to be processed in the disk 101; when both the disk 101 and the cache 102 are target caches 102, the data processing operation can be performed on the data to be processed in both the disk 101 and the cache 102.
[0106] In this embodiment, based on the remaining storage space of the buffer 102, it is prioritized to perform data processing operations on the data to be processed in the buffer 102. This is to improve data writing performance when the buffer 102 has sufficient storage space for writing the data to be processed.
[0107] In some embodiments, the main controller chip 103 is further configured to:
[0108] Based on the remaining storage space, determine the maximum writable data specification of the cache 102;
[0109] If the target data specification is greater than the maximum writable data specification, the cache 102 and the disk 101 are determined as the destination storage.
[0110] Based on the target data specification and the maximum writable data specification, the data to be processed is divided into data blocks to obtain a first data block and a second data block.
[0111] A data write operation is performed on the first data block in the buffer 102;
[0112] Perform a data write operation on the second data block in the disk 101;
[0113] Wherein, the data specification of the first data block is less than or equal to the maximum writable data specification; the second data block is a data block generated based on the data in the data to be processed, excluding the data in the first data block.
[0114] Optionally, the main controller chip 103 is also used to determine the maximum available storage space of the cache 102 based on the remaining storage space and the safety margin, and further determine the maximum writable data specification of the cache 102 based on the maximum available storage space of the cache 102; the safety margin here is the storage space required to maintain the correct operation of the memory, which can be adaptively determined according to the cache performance of the cache, such as 40% of the storage space of the cache.
[0115] Here, the maximum available storage space can be determined by subtracting the remaining storage space from the safety margin.
[0116] This embodiment determines the maximum usable storage space of the cache 102 based on the remaining storage space and safety margin, which can maximize the use of the cache resources of the cache while ensuring the safe operation of the cache, thereby effectively improving data reading performance.
[0117] Next, the main controller chip 103 can compare the target data specification with the maximum writable data specification. If it is determined that the target data specification is less than or equal to the maximum writable data specification, the buffer 102 can be used as the destination memory to perform a data write operation on the data to be processed separately in the buffer 102.
[0118] Here, when performing a data write operation on the data to be processed individually in the cache 102, the data to be processed can be divided into multiple stripes; based on the multiple cache blocks of the cache, data processing operations are performed on each sub-data block in parallel in the cache.
[0119] When the target data specification is determined to be greater than the maximum writable data specification, the cache 102 and the disk 101 are used together as the destination storage. In order to maximize the use of the cache storage resources and improve the data writing performance, the data to be processed can be divided into data blocks according to the target data specification and the maximum writable data specification to obtain the first data block and the second data block; wherein, the data specification of the first data block is less than or equal to the maximum writable data specification.
[0120] Next, a data write operation is performed on the first data block in the cache 102 in parallel, and a data write operation is performed on the second data block in the disk 101.
[0121] Here, when performing a data write operation on the first data block in the cache 102, the first data block can be divided into multiple stripes, so that data processing operations can be performed in parallel on each sub-data block of the first data block in the cache based on multiple cache blocks of the cache.
[0122] Normally, the disk only requires the data to be written to the cache to complete the write operation. Therefore, in order to maximize the use of the cache's storage space and improve the write speed, when the target data size is less than or equal to the maximum writable data size, the data to be processed is written directly in the cache. Only when the target data size is greater than the maximum writable data size, the cache and disk are used to write the data to be processed in parallel to achieve efficient writing of the data to be processed.
[0123] In some embodiments, the main controller chip 103 is further configured to:
[0124] If the data processing operation includes a data reading operation, determine the storage address of the data to be processed;
[0125] If, based on the storage address and the storage information, it is determined that both the disk 101 and the cache 102 store the data to be processed, then the cache 102 is determined to be the destination storage.
[0126] The data to be processed is read from the cache 102 according to the storage address.
[0127] Optionally, for data processing operations including data reading operations, the main controller chip 103 is also used to determine the storage address of the data to be processed, and to obtain the data content stored in the buffer 102 from the storage information to determine whether the buffer 102 stores the data to be processed. At the same time, based on the storage address of the data to be processed and the data content stored in the disk 101, it is determined whether the disk 101 stores the data to be processed.
[0128] If it is determined that only one of the disk 101 and the cache 102 stores the data to be processed, the memory storing the data to be processed is used as the destination memory, and the data to be processed is read from the destination memory according to the storage address.
[0129] If it is determined that both disk 101 and cache 102 contain data to be processed, cache 102 is preferred as the destination memory so that the data to be processed can be read from cache 102 according to the storage address.
[0130] In data retrieval, if the data to be read can be found in the cache, the time required for disk seeks is significantly reduced. Therefore, to improve data retrieval performance, when both disk and cache store data to be processed, data should be read first from the cache. This allows for direct retrieval of the same data and instructions when processing the same data or executing the same instructions repeatedly, reducing data interaction time and disk seek time. Consequently, the response time of the storage system is greatly reduced, enabling efficient reading of data to be processed and improving the performance of the storage system.
[0131] The disk storage method provided by the present invention is described below. The disk storage method described below can be referred to in correspondence with the disk storage system described above.
[0132] like Figure 2 The diagram shown is a flowchart of a disk storage method provided in this embodiment. The main controller chip 103 in the disk storage system 1 is the execution entity of this method. The system includes a disk 101, a cache 102 of the disk 101, and the main controller chip 103 of the disk 101. The specific structure is as follows: Figure 1 As shown, it will not be elaborated further here.
[0133] like Figure 2 As shown, the method includes:
[0134] Step 201: Receive data processing instructions sent by an external device;
[0135] Step 202: Obtain the data to be processed and the data processing operation of the data to be processed according to the data processing instruction;
[0136] Step 203: Based on the storage information of the cache integrated on the main controller chip of the disk, perform the data processing operation on the data to be processed in the cache and / or the disk;
[0137] The data processing operations include data reading operations or data writing operations.
[0138] Optionally, the main controller chip is used to listen for data processing instructions sent by external devices in real time. Upon receiving a data processing instruction, it can parse the instruction to obtain the data to be processed and the data processing operation.
[0139] Then, based on the storage information of the cache, the destination memory is determined in the cache and the disk, that is, the memory used to provide storage space for data reading and writing of the data to be processed is determined in the cache and the disk, so that data processing operations can be performed on the data to be processed in the destination memory.
[0140] Here, the method for determining the destination storage can be based on the type of data processing operation. Historical storage data or storage space usage information can be retrieved from the storage information to determine the destination storage. For example, if the data processing operation is a data read operation, historical storage data is retrieved from the storage information. Based on this historical data, it is first determined whether the data to be processed can be retrieved from the cache, and then whether the cache is the destination storage. Only if the cache is not the destination storage is it further determined whether the disk can be used as the destination storage. Similarly, if the data processing operation is a data write operation, storage space usage information is retrieved from the storage information. Based on this storage space usage, it is first determined whether the cache has sufficient memory space for data writing, and then whether the cache is the destination storage. Only if the cache is not the destination storage is it further determined whether the disk can be used as the destination storage.
[0141] The disk storage method provided in this embodiment integrates the cache into the main controller chip connected to the disk. The main controller chip directly performs data processing operations on the data to be processed in the cache and / or disk. This not only reduces the space resources required for a large number of DRAM chips on the upper part of the disk PCB board and reduces wiring costs, but also allows the main controller chip to directly control the cache to perform data processing operations locally. This effectively reduces the data processing latency and information loss caused by the main controller chip remotely accessing the disk and cache, and greatly improves the data read and write performance of the disk.
[0142] like Figure 3 As shown, in some embodiments, performing the data processing operation on the data to be processed in the cache and / or the disk based on the storage information of the cache integrated on the main controller chip of the disk includes:
[0143] Step 301: Based on the storage information, determine the destination memory in the cache and the disk; the destination memory is used to provide storage space for data reading and writing of the data to be processed;
[0144] Step 302: If the buffer is determined to be the destination memory, the data to be processed is divided into multiple stripes; each stripe includes multiple sub-data blocks;
[0145] Step 303: Divide each of the sub-data blocks into the corresponding sets;
[0146] Step 304: Based on the set identifier of the corresponding set and the association relationship between each set and each cache block stored in the set association cache mapping table, determine the target cache block corresponding to each sub-data block in the multiple cache blocks of the cache; the set association cache mapping table is pre-stored in the cache;
[0147] Step 305: Based on the target cache block corresponding to each of the sub-data blocks, perform the data processing operation in parallel on each of the sub-data blocks in the cache.
[0148] Optionally, the main controller chip can determine the destination memory from the cache and the disk based on the storage information. When the cache is determined to be the destination memory based on the storage information, it indicates that the cache can provide storage space for data reading and writing of the data to be processed. At this time, the data to be processed can be divided into multiple stripes to segment the data into multiple data blocks.
[0149] Here, the partitioning method can be based on the data size of the cache block and the processing performance of each cache block, or it can be based on a pre-defined data specification, such as 16KB. This example does not specifically limit this method.
[0150] Next, the main controller chip can sequentially divide each sub-data block into the corresponding set according to the number of sub-data blocks that each set can process, or it can divide each sub-data block into the corresponding set according to the association between each sub-data block and each set. This embodiment does not specifically limit this.
[0151] Then, based on the set identifier of the corresponding set of each sub-data block and the association between each set and each cache block, the main controller chip determines the target cache block corresponding to each sub-data block among multiple cache blocks, that is, the cache block used to process the data processing task of each sub-data block.
[0152] Next, the main controller chip divides the processing tasks of each sub-data block into their respective target cache blocks, so that the target cache blocks corresponding to each sub-data block can perform data reading and data writing operations on each sub-data block in parallel in the cache.
[0153] In this embodiment, multiple cache blocks are used to perform data reading or writing operations on each sub-data block in the data to be processed in parallel. This enables data reading tasks to be performed in parallel on some cache blocks, while data writing operations can be performed synchronously on other cache blocks. This effectively reduces waiting latency and cache resource waste, thereby improving the utilization rate of cache resources and increasing read and write speed.
[0154] This embodiment also provides a server, which includes a disk storage system 1 and a processor; the processor is connected to the disk storage system 1 and is used to send data processing instructions to the disk storage system 1.
[0155] The server also includes at least one battery to power the various components in the server while protecting the data security in the cache 102 and disk 101.
[0156] The processor here can be a CPU or other processor, which sends data processing instructions to the disk storage system 1 through the data bus interaction interface. When the main controller chip 103 of the disk 101 receives the data processing instructions, it obtains the data to be processed and the data processing operation of the data to be processed according to the data processing instructions. Based on the storage information of the cache 102, it performs the data processing operation on the data to be processed in the cache 102 and / or the disk 101.
[0157] The server provided in this embodiment integrates the cache into the main controller chip connected to the disk. The main controller chip directly performs data processing operations on the data to be processed in the cache and / or disk. This not only reduces the space resources required for a large number of DRAM chips on the upper part of the disk PCB board and reduces wiring costs, but also allows the main controller chip to directly control the cache to perform data processing operations locally. This effectively reduces the data processing latency and information loss caused by the main controller chip remotely accessing the disk and cache, and greatly improves the data read and write performance of the disk.
[0158] Figure 4 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 4 As shown, the electronic device may include a processor 401, a communications interface 402, a memory 403, and a communication bus 404, wherein the processor 401, communications interface 402, and memory 403 communicate with each other via the communication bus 404. The processor 401 can invoke logical instructions in the memory 403 to execute a disk storage method, which includes: receiving a data processing instruction; acquiring data to be processed and data processing operations on the data to be processed according to the data processing instruction; and performing the data processing operations on the data to be processed in the cache and / or the disk according to the storage information of the cache integrated on the main controller chip of the disk; wherein the data processing operations include data reading operations or data writing operations.
[0159] Furthermore, the logical instructions in the aforementioned memory 403 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0160] On the other hand, the present invention also provides a computer program product, the computer program product including a computer program, the computer program being able to be stored on a non-transitory computer-readable storage medium, the computer program being executed by a processor, the computer being able to execute the disk storage method provided by the above methods, the method including: receiving a data processing instruction; obtaining data to be processed and data processing operations of the data to be processed according to the data processing instruction; and performing the data processing operations on the data to be processed in the cache and / or the disk according to the storage information of the cache integrated on the main controller chip of the disk; wherein the data processing operations include data reading operations or data writing operations.
[0161] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When executed by a processor, the computer program is implemented to perform the disk storage method provided by the above methods. The method includes: receiving a data processing instruction; acquiring data to be processed and data processing operations on the data to be processed according to the data processing instruction; and performing the data processing operations on the data to be processed in the cache and / or the disk according to the storage information of the cache integrated on the main controller chip of the disk. The data processing operations include data reading operations or data writing operations.
[0162] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.
[0163] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.
[0164] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A disk storage system, characterized in that, Includes a disk, a cache for the disk, and a main controller chip for the disk; The main controller chip is externally connected to the disk, and the cache is integrated into the main controller chip; The main controller chip is used to receive data processing instructions, obtain data to be processed and data processing operations for the data to be processed according to the data processing instructions, and perform the data processing operations on the data to be processed in the cache and / or the disk according to the storage information of the cache; wherein, the data processing operations include data reading operations and / or data writing operations; The cache includes multiple cache blocks; The main controller chip is specifically used for: If the cache is determined to be the destination memory based on the storage information, the data to be processed is divided into multiple stripes; each stripe includes multiple sub-data blocks; the destination memory is used to provide storage space for data reading and writing of the data to be processed. Based on multiple cache blocks, the data processing operation is performed in parallel on each of the sub-data blocks in the cache.
2. The disk storage system according to claim 1, characterized in that, The cache pre-stores a set-association cache mapping table, which stores the association relationship between each set and each cache block; The main controller chip is also used for: Each of the aforementioned sub-data blocks is divided into a corresponding set; Based on the set identifier of the corresponding set and the association relationship, the target cache block corresponding to each of the sub-data blocks is determined in the multiple cache blocks; Based on the target cache block corresponding to each of the sub-data blocks, the data processing operation is performed in parallel on each of the sub-data blocks in the cache.
3. The disk storage system according to claim 1, characterized in that, The main controller chip is also used for: When the data processing operation includes a data writing operation, the target data specification of the data to be processed is obtained; Based on the storage information, determine the remaining storage space of the cache; Based on the remaining storage space and the target data specifications, the data processing operation is performed on the data to be processed in the cache and / or the disk.
4. The disk storage system according to claim 3, characterized in that, The main controller chip is also used for: Based on the remaining storage space, determine the maximum writable data size of the cache; If the target data specification is greater than the maximum writable data specification, the cache and the disk are determined as the destination storage. Based on the target data specification and the maximum writable data specification, the data to be processed is divided into data blocks to obtain a first data block and a second data block. A data write operation is performed on the first data block in the buffer; Perform a data write operation on the second data block in the disk; Wherein, the data specification of the first data block is less than or equal to the maximum writable data specification; the second data block is a data block generated based on the data in the data to be processed, excluding the data in the first data block.
5. The disk storage system according to claim 1, characterized in that, The main controller chip is also used for: If the data processing operation includes a data reading operation, determine the storage address of the data to be processed; If, based on the storage address and the storage information, it is determined that both the disk and the cache contain the data to be processed, the cache is determined to be the destination memory. The data to be processed is read from the cache according to the storage address.
6. The disk storage system according to any one of claims 1-5, characterized in that, The disk includes multiple hard drives; Multiple hard drives are combined to form a disk array.
7. A disk storage method, applied to the disk storage system as described in any one of claims 1 to 6, characterized in that, include: Receive data processing instructions; The data to be processed and the data processing operation of the data to be processed are obtained according to the data processing instructions. Based on the storage information of the cache integrated on the main controller chip of the disk, the data processing operation is performed on the data to be processed in the cache and / or the disk. The data processing operations include data reading operations or data writing operations.
8. The disk storage method according to claim 7, characterized in that, The step of performing the data processing operation on the data to be processed in the cache and / or the disk based on the storage information of the cache integrated on the main controller chip of the disk includes: Based on the storage information, a destination memory is determined in the cache and the disk; the destination memory is used to provide storage space for data reading and writing of the data to be processed. If the cache is determined to be the destination memory, the data to be processed is divided into multiple stripes; each stripe includes multiple sub-data blocks; Each of the aforementioned sub-data blocks is divided into a corresponding set; Based on the set identifier of the corresponding set and the association relationship between each set and each cache block stored in the set association cache mapping table, the target cache block corresponding to each sub-data block is determined in the multiple cache blocks of the cache; the set association cache mapping table is pre-stored in the cache; Based on the target cache block corresponding to each of the sub-data blocks, the data processing operation is performed in parallel on each of the sub-data blocks in the cache.
9. A server, characterized in that, Includes the disk storage system and processor as described in any one of claims 1-6; The processor is connected to the disk storage system and is used to send data processing instructions to the disk storage system.
Citation Information
Patent Citations
Distributed cache system in a drive array
CN102016807A
Data processing method and device, electronic equipment and storage medium
CN115933994A
