A data read-write processing method in a storage system and an electronic device

Abstract

The application discloses a data read-write processing method and an electronic device in a storage system, relates to the technical field of computer storage, and realizes the asynchronization and isolation of read-write operations by constructing a three-layer resource processing architecture, decoupling host read-write requests into independent controllable processing stages. The first resource focuses on recording metadata, the second resource serves as a temporary carrier for data transfer, and the third resource serves as a persistent cache management center. This architecture enables the write operation to return a response to the host as soon as the data reaches the second resource, greatly reducing the write delay; the read operation realizes non-blocking access through the concurrent control mechanism of the third resource. Overall, the response speed and throughput of the storage system are improved, and the performance bottleneck of mutual blocking of read and write due to competition for the same data resource in the related architecture is solved.

Description

Technical Field

[0001] This application relates to the field of computer storage technology, and in particular to a data read / write processing method and electronic device in a storage system. Background Technology

[0002] In the business processing of modern storage systems, the coupling between read I / O (Input and Output) and write I / O operations significantly limits system performance. This manifests in two main ways: 1. Write I / O severely restricts read I / O performance: Write I / O involves modifying data content, typically involving a lengthy data processing flow, and has a serial dependency with other write I / O operations. More importantly, based on data consistency requirements, storage systems usually employ locking mechanisms for synchronization when processing the same data block. When a write I / O operation is modifying a data block, it holds an exclusive lock on that block. During this period, all subsequent arriving read I / O operations requesting access to the same data block must queue and wait until the write I / O completes and releases the lock. This directly leads to a significant increase in read I / O operation latency, a decrease in overall throughput, and read performance severely hampered by write operations. 2. Read I / O also affects write I / O response time: Although concurrent processing (shared locks) is possible between read I / O operations, based on the same lock synchronization mechanism, subsequent write I / O operations requesting to modify the same data block must wait for all ongoing read I / O operations on that data block to complete and release the shared lock before they can acquire an exclusive lock and begin execution. This means that concurrent holding of read operations will block the execution of write operations, significantly extending the response time of write I / O. In summary, in the relevant processing model, there is a serious performance coupling and mutual influence between read I / O and write I / O. This serialized processing method of mutual exclusion between read and write has become a key bottleneck for improving the overall IOPS (Input / Output Operations Per Second) and reducing latency of the storage system, especially in high-concurrency, mixed read / write load application scenarios, where its performance limiting effect is even more prominent. Summary of the Invention

[0003] This application provides a data read / write processing method and electronic device in a storage system, so as to at least solve the problem of serious performance coupling and mutual influence between read IO and write IO in related technologies.

[0004] This application provides a data read / write processing method in a storage system. The method includes: allocating a first resource for a received read / write request, wherein the first resource is used to manage the processing status of the target data corresponding to the read / write request; dividing the target data into one or more data blocks according to a preset granularity, and allocating a second resource for each of the one or more data blocks; determining a third resource corresponding to the address of the data block corresponding to the second resource, wherein the third resource is used to maintain the data status and access permissions within the corresponding address range; in response to a write request, writing one or more data blocks into the second resource respectively, and determining a first access permission status of the third resource, so as to determine whether to perform a write merging operation based on the first access permission status, wherein the write merging operation is used to merge the data in the second resource into the third resource; in response to the second resource having completed data writing, returning a write completion response to the host; in response to a read request, determining a second access permission status of the third resource, and determining whether to read data from the third resource or backend storage into the corresponding second resource based on the second access permission status, and aggregating the data in the second resource and returning it to the host; in response to the second resource having completed data return, returning a read completion response to the host.

[0005] This application also provides an electronic device, including: a memory for storing a computer program; and a processor for executing the computer program to implement at least the following steps in a storage system for data read / write processing: allocating a first resource for a received read / write request, wherein the first resource is used to manage the processing status of target data corresponding to the read / write request; dividing the target data into one or more data blocks according to a preset granularity, and allocating a second resource for each of the one or more data blocks; determining a third resource corresponding to the address of the data block corresponding to the second resource, wherein the third resource is used to maintain the data status and access permissions within the corresponding address range; in response to a write request, writing one or more data blocks to the second resource respectively, and determining a first access permission status of the third resource, so as to determine whether to perform a write merging operation based on the first access permission status, wherein the write merging operation is used to merge the data in the second resource into the third resource; in response to the second resource having completed data writing, returning a write completion response to the host; in response to a read request, determining a second access permission status of the third resource, and determining whether to read data from the third resource or backend storage to the corresponding second resource based on the second access permission status, and aggregating the data in the second resource and returning it to the host; in response to the second resource having completed data return, returning a read completion response to the host.

[0006] This application constructs a three-layer resource processing architecture, decoupling host read and write requests into independent and controllable processing stages, thus achieving asynchronous and isolated read and write operations. The first resource focuses on recording metadata, the second resource serves as a temporary carrier for data transfer, and the third resource acts as a persistent cache management center. This architecture allows write operations to return a response to the host as soon as the data reaches the second resource, significantly reducing write latency; read operations achieve non-blocking access through the concurrency control mechanism of the third resource. Overall, this improves the response speed and throughput of the storage system and solves the performance bottleneck of mutual blocking between read and write operations due to competition for the same data resource in related architectures. Attached Figure Description

[0007] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0008] Figure 1 This is a flowchart illustrating the data read / write processing method in the storage system of the first embodiment;

[0009] Figure 2 This is a diagram of the internal structure of the electronic device in the second embodiment. Detailed Implementation

[0010] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0011] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0012] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0013] In the first embodiment, such as Figure 1As shown, a data read / write processing method in a storage system is provided. The method includes: allocating a first resource for a received read / write request, wherein the first resource is used to manage the processing status of the target data corresponding to the read / write request; dividing the target data into one or more data blocks according to a preset granularity, and allocating a second resource for each of the one or more data blocks; determining a third resource corresponding to the address of the data block corresponding to the second resource, wherein the third resource is used to maintain the data status and access permissions within the corresponding address range; responding to a write request, writing one or more data blocks to the second resource respectively, and determining the first access permission status of the third resource, so as to determine whether to perform a write merging operation based on the first access permission status, wherein the write merging operation is used to merge the data in the second resource into the third resource; responding to the second resource having completed data writing, returning a write completion response to the host; responding to a read request, determining the second access permission status of the third resource, and determining whether to read data from the third resource or backend storage to the corresponding second resource based on the second access permission status, and aggregating the data in the second resource and returning it to the host; responding to the second resource having completed data return, returning a read completion response to the host.

[0014] Specifically, the above method constructs a three-layer resource processing architecture, decoupling host read and write requests into independent and controllable processing stages, thus achieving asynchronous and isolated read and write operations. The first resource focuses on recording metadata, the second resource serves as a temporary carrier for data transfer, and the third resource acts as a persistent cache management center. This architecture allows write operations to return a response to the host as soon as the data reaches the second resource, significantly reducing write latency; read operations achieve non-blocking access through the concurrency control mechanism of the third resource. Overall, this improves the response speed and throughput of the storage system and solves the performance bottleneck of mutual blocking between read and write operations due to competition for the same data resource in related architectures.

[0015] In a specific embodiment, the metadata of the read / write request includes at least the volume ID, the starting LBA address, the requested data size, and the read / write type. The second resource receives the target data corresponding to the I / O request from the receiving host according to the LBA order.

[0016] In a specific embodiment, during the system initialization phase, four resource pools are prepared: a first resource pool, a second resource pool, a third resource pool, and memory resources. The memory resources are divided into 4KB memory pages according to the operating system's standard granularity. Here, the first resource pool is divided into the second resource pool at a granularity of 32KB. The pseudocode for other resources is as follows: `Struct hio{uint32 lun_id; uint32 lab; uint32 size; bool read; list head; uint32 pio_completed; uint32 pio_allocate;}Struct pio{uint32 lun_id; uint32 lba; uint32 size; uint32 offset; uint8 flag; list_node hio_node; list_node cdcb_node; void *data_ptr;}Struct cdcb{uint32 lun_id; uint32 lba; uint64 valid_page_mask; uint64 modified_page_mask; spinlock lock; uint8 ...` staus;uint8 status_count;uint8 write_queued_count;list queued_list;void*data_ptr;}.

[0017] Furthermore, the target data is divided into one or more data blocks according to a preset granularity, and a second resource is allocated to each of the one or more data blocks. This includes: determining multiple integer multiple address intervals required to cover the data range based on the data range of the target data, wherein the size of the integer multiple address intervals corresponds to the preset granularity; determining the offset parameter of the second resource based on the starting position of the target data within the corresponding integer multiple address interval of the second resource; determining the data volume parameter of the second resource based on the amount of target data within the corresponding integer multiple address interval of the second resource; and determining the memory space required for the second resource based on the offset parameter and the data volume parameter, and allocating memory space of the corresponding size for the second resource.

[0018] Specifically, the above method introduces an address range partitioning method based on preset granularity, systematically transforming host requests with arbitrary starting positions and data lengths into multiple internal processing units of uniform size and address alignment. The setting of offset and data volume parameters precisely describes the actual location and range of unaligned data, achieving efficient and unified processing of fragmented input / output requests. This partitioning method lays the foundation for subsequent concurrent processing, precise memory allocation, and cache management, ensuring the rationality of resource allocation and the continuity of data processing.

[0019] In a specific embodiment, the preset data granularity is 32KB (i.e., 64 sectors, 1 sector = 512B); read / write request: write request; target data: starting logical block address (LBA): 63; data size (Size): 128 sectors; data range: starting from LBA63, 128 consecutive sectors, i.e., the address range [63, 190] (63+128-1). Step 1: Determine the partitioning base and alignment. Partitioning base: an address range that is an integer multiple of 32KB (64 sectors). That is, the range is: [0, 63], [64, 127], [128, 191], [192, 255]... Alignment operation: align the data range [63, 190] with these integer multiple address ranges. This means that the data range will span the following ranges: the latter half of the range [0, 63], the entire range [64, 127], and the first half of the range [128, 191]. Step 2: Handle address misalignment and set offset: The starting address of the target data (LBA63) is not on the starting boundary (LBA0) of the interval [0, 63], i.e., "the starting address is not aligned with an integer multiple of the address". Therefore, set the offset parameter for the second resource (which we call PIO-1) covering the interval [0, 63], offset parameter = 63. This indicates that the valid data in PIO-1 starts from the 63rd sector position within the entire interval [0, 63] it is responsible for. Step 3: Determine the data portion covered by each interval and set the data size: It is necessary to determine the corresponding data portion size (data size parameter) for each integer multiple of the address interval covered by the data range [63, 190]. For the interval [0, 63]: it covers the data range [63, 63], data size parameter = 1 (sector). Because the interval starts from LBA63 and ends at LBA63 (only 1 sector), it exceeds the portion of the target data within this interval. For the interval [64, 127]: It covers the entire interval, i.e., the data range [64, 127], with a data size parameter of 64 (sectors), which is the full size of the interval. Since the data starts at the boundary of this interval (LBA64), its offset parameter is 0. For the interval [128, 191]: It covers the data range [128, 190], with a data size parameter of 63 (sectors). This is because there are 63 sectors from LBA128 to LBA190. The data starts at the boundary of this interval (LBA128), with its offset parameter being 0. Step 4: Allocate Secondary Resources: Based on the above analysis, allocate three secondary resources (PIOs) for the data range [63, 190]: PIO-1: corresponding to the interval [0, 63], parameters: offset = 63, data size = 1; PIO-2: corresponding to the interval [64, 127], parameters: offset = 0, data size = 64; PIO-3: corresponding to the interval [128, 191]. Parameters: offset = 0, data size = 63.

[0020] Further, allocating memory space of a corresponding size for the second resource includes: determining the number of complete memory pages required by the second resource based on the memory space size required by the second resource; requesting multiple consecutive or non-consecutive memory pages from the memory resource pool based on the number of complete memory pages, wherein the memory resource pool manages memory space in units of memory pages of a preset data size; and assigning the logical address or physical address of the multiple memory pages to the storage pointer of the second resource, wherein the storage pointer is used to indicate that the second resource accesses the corresponding memory space.

[0021] Specifically, the above method achieves fine-grained management of memory resource allocation by allocating memory based on the number of complete memory pages. Allocating memory from the resource pool in units of fixed-size memory pages effectively avoids memory fragmentation and improves memory utilization, regardless of whether the requested physical pages are contiguous or non-contiguous. Assigning the obtained memory address to the storage pointer of the second resource establishes a clear access path between the resource and physical memory, ensuring efficient and secure data transmission and providing a stable and reliable memory space for temporary data caching.

[0022] Further, based on the memory space required by the second resource, the number of complete memory pages required by the second resource is determined, including: determining a conversion factor based on the memory page size and the smallest storage unit size of the target data; dividing the offset parameter by the conversion factor and rounding down to obtain the starting memory page boundary; dividing the sum of the offset parameter and the data size parameter by the conversion factor and rounding up to obtain the ending memory page boundary; subtracting the starting memory page boundary from the ending memory page boundary and adding one to obtain the number of complete memory pages required by the second resource.

[0023] Specifically, the above method provides a way to accurately calculate the number of memory pages required for unaligned data. By calculating the start and end memory page boundaries, combined with rounding down and up operations, the minimum complete set of memory pages spanned by a data range of arbitrary offset and length can be accurately determined. This calculation model ensures that just enough memory space is allocated to the data, avoiding both data access out-of-bounds errors due to underestimation and memory waste caused by over-allocation. It is a key algorithmic guarantee for achieving efficient and accurate memory resource management.

[0024] In a specific embodiment, the memory page size is 4KB, and the preset conversion factor is 4KB ÷ 512B = 8, meaning 1 memory page = 8 sectors. Taking PIO1 as an example: the starting memory page boundary is 63 ÷ 8 = 7.875 → rounded down = 7, the ending memory page boundary is (63 + 1) ÷ 8 = 8 → rounded up = 8, and the total number of complete memory pages is 8 - 7 + 1 = 2.

[0025] Further, determining the third resource corresponding to the data block address corresponding to the second resource includes: determining a corresponding integer multiple address range based on the data block address of the second resource, and searching in the corresponding resource pool for a third resource that manages the corresponding integer multiple address range; in response to the existence of a third resource that manages the corresponding integer multiple address range, determining the found third resource as the third resource corresponding to the data block address corresponding to the second resource; in response to the absence of a third resource that manages the corresponding integer multiple address range, requesting a new third resource from the corresponding resource pool, and assigning the corresponding integer multiple address range to the new third resource, so that the new third resource is used as the third resource corresponding to the data block address corresponding to the second resource.

[0026] Specifically, the above method implements a dynamic, on-demand creation and global shared lookup mechanism for third-party resources. By mapping data block addresses to integer multiples of address ranges and searching for or creating corresponding third-party resources in the resource pool, it ensures that each data block has one and only one unified manager. This mechanism avoids the idle waste that may result from static pre-allocation of resources, while guaranteeing the consistency of state when multiple concurrent accesses to the same data block occur. It forms the cornerstone of cache sharing and concurrency control and is a core technology for achieving data consistency and efficient cache reuse.

[0027] In a specific embodiment, the range [0, 63] is determined based on the PIO1 data block address; a search is performed in the CDCB pool → it does not exist; a new CDCB is requested and assigned the range [0, 63]; it is then determined as the corresponding resource.

[0028] Further, determining the first access permission state of the third resource, or determining the second access permission state of the third resource, includes: acquiring the spinlock of the third resource, and determining the current lock state and write wait count of the third resource; in response to a read / write request being a write request, and the current lock state being idle, determining the access permission state of the third resource to be write exclusive, modifying the lock state of the third resource to write exclusive, and incrementing the write wait count; in response to a read / write request being a read request, the current lock state being idle, and the write wait count being zero, determining the access permission state of the third resource to be read shared, modifying the lock state of the third resource to read shared, and incrementing the read shared count of the third resource; in response to... If a read / write request is a write request and the current lock state is either write exclusive or read shared, the access permission state of the third resource is determined to be prohibited, and the write wait count is incremented. If a read / write request is a read request and the current lock state is either write exclusive or read shared, the access permission state of the third resource is determined to be prohibited. If a read / write request is a read request and the write wait count is not zero, the access permission state of the third resource is determined to be prohibited. The spinlock of the third resource is released. The write wait count is used to record the number of second resources in the third resource that are waiting for or performing write exclusive access, and the read shared count is used to record the number of second resources in the third resource that are performing read shared access.

[0029] Specifically, the above method defines a sophisticated read / write concurrency control state machine and locking protocol. Through state checking and modification protected by spinlocks, atomic acquisition of write exclusive and read shared access permissions is achieved. In particular, a precondition of "write wait count being zero" is introduced for read shared access, implementing a write-priority scheduling strategy and effectively preventing write operations from starving due to continuous read access. The independent maintenance of the write wait count and read shared count allows the system to accurately track the concurrent access status of each cache block, providing a core mechanism for achieving high-performance read / write isolation and fair scheduling.

[0030] Further, based on the first access permission state, determining whether to perform a write merge operation includes: in response to the first access permission state being in an access-prohibited state, prohibiting the execution of the write merge operation, adding the second resource to the waiting queue of the third resource, and marking the write task of the second resource as completed; in response to the first access permission state being in a write-exclusive state, determining to perform a write merge operation, merging the data in the second resource into the third resource, and updating the data status identifier of the third resource; wherein, the data status identifier includes at least a valid data identifier and a dirty data identifier, and the valid data identifier is used to indicate whether the data unit in the third resource is valid, and the dirty data identifier is used to indicate whether the data unit in the third resource has been modified and needs to be written to the backend storage.

[0031] Specifically, the above method defines the complete process of synchronous data update and lock release. After successfully acquiring exclusive write privileges, the data is copied to a third resource and the validity and modification flags are updated, ensuring the atomicity and traceability of cached data updates. By restoring the lock state to idle and decrementing the write wait count, resource occupation is safely released. The design of immediately marking the task as completed and adding it to the waiting queue when privileges are not successfully acquired allows the host write request response to be accelerated, achieving decoupling between low-latency response and background asynchronous persistence. This is a key innovation for improving write performance.

[0032] In a specific embodiment, the mask bits for both the valid data mask and the dirty data mask are set to 1, with one mask bit corresponding to one sector. If a third resource manages 64 sectors (32KB), then both masks are 64-bit bitmaps, where the 0 / 1 state of each bit precisely corresponds to the data state of a specific sector.

[0033] Furthermore, after marking the write task of the second resource as completed, or updating the data status identifier of the third resource, the data read / write processing method in the storage system further includes: in response to the completion of the write merge operation, restoring the lock state of the third resource from the write exclusive state to the idle state, and decrementing the write wait count; in response to the completion of the write merge operation or the second resource being added to the wait queue, releasing the spinlock of the third resource, and waking up the waiting second resource to execute the corresponding write task according to the current state of the wait queue. Specifically, waking up the waiting second resource to execute the corresponding write task according to the current state of the wait queue includes: in response to the existence of a waiting second resource in the wait queue, waking up the waiting second resource and executing the corresponding write task; in response to the absence of a waiting second resource in the wait queue, marking the write task of the second resource as completed, and releasing the second resource back to the second resource pool.

[0034] Specifically, the above method introduces a waiting queue inspection and wake-up mechanism, forming a complete task scheduling closed loop. This mechanism ensures that when a write operation completes, other waiting write tasks on the same cache block can be woken up in a timely manner, reducing task queuing latency and improving resource utilization and system throughput. The process of the task retrying to acquire the lock after being woken up is seamlessly connected with the initial lock acquisition mechanism, forming a fair and efficient serialized execution chain that effectively manages concurrent write competition for the same hot data.

[0035] Further, based on the second access permission status, determining whether to read data from the third resource or backend storage to the corresponding second resource includes: in response to the second access permission status being in a prohibited access state, prohibiting the reading of data from the third resource or backend storage to the corresponding second resource, and adding the second resource to the waiting queue of the third resource; in response to the second access permission status being in a read shared state, determining whether the data in the third resource meets the data requirements of the second resource based on the offset parameter, data volume parameter, and valid data identifier of the third resource; in response to the data in the third resource meeting the data requirements of the second resource, reading data from the third resource to the corresponding second resource; in response to the data in the third resource not meeting the data requirements of the second resource, reading data from the backend storage to the corresponding second resource.

[0036] Furthermore, after reading data from the third resource to the corresponding second resource, or reading data from the backend storage to the corresponding second resource, the data read / write processing method in the storage system further includes: acquiring the spinlock of the third resource and decrementing the read shared count of the third resource; in response to the decremented read shared count being zero, changing the lock state of the third resource from the read shared state to the idle state; and releasing the spinlock of the third resource.

[0037] Specifically, the above method implements intelligent cache hit detection and secure lock release for read requests. By combining request parameters and cache validity indicators, it determines whether data is fully cached, intelligently decides the data source, and maximizes caching benefits. After data reading is complete, an atomic operation decrements the read sharing count, and restores the idle state when the count reaches zero, ensuring the accuracy and security of lock state transitions. This process guarantees that multiple read requests can safely share the cache and releases resources promptly after all read operations are completed, balancing read concurrency performance and resource management efficiency.

[0038] In a specific embodiment, a check and wake-up mechanism for the waiting queue can also be introduced after the read lock is released. The specific method is similar to the check and wake-up mechanism for the waiting queue introduced after the write lock is released.

[0039] Furthermore, the data read / write processing method in the storage system also includes: scanning multiple used third resources and designating third resources with non-zero dirty data identifiers as third resources to be written to disk, wherein used third resources indicate third resource instances that have been allocated from the third resource pool and have not yet been released; requesting a fourth resource from the second resource pool for the third resource to be written to disk and assigning the data block address of the third resource to be written to disk to the fourth resource, wherein the fourth resource is used to temporarily cache modified data to be written to the backend storage; acquiring read shared access permissions for the third resource to be written to disk; in response to successfully acquiring read shared access permissions for the third resource to be written to disk, determining the range of modified data to be written to the backend storage based on the dirty data identifier of the third resource to be written to disk; copying the data within the modified data range to the fourth resource and clearing the dirty data identifier of the third resource to be written to disk; releasing read shared access permissions for the third resource to be written to disk and writing the data in the fourth resource to the backend storage; releasing the fourth resource; in response to failing to acquire read shared access permissions for the third resource to be written to disk, adding the fourth resource to the waiting queue of the third resource to be written to disk.

[0040] Specifically, the above method implements a background asynchronous disk write mechanism for dirty data, which is crucial for ensuring data persistence and system performance. It scans used cache blocks and selects blocks containing dirty data to initiate disk write tasks, incorporating background data flushing operations into a unified concurrency control framework. The disk write process accesses cache blocks in a read-shared manner, avoiding blocking foreground read / write requests. The range of data to be written is precisely determined based on the dirty data mask and copied as needed, followed by immediately clearing the dirty data flag, ensuring data consistency and avoiding duplicate disk writes. Disk write operations are executed asynchronously after the read lock is released, minimizing the impact on system response time. If the lock is not successfully acquired, the disk write task enters a waiting queue, ensuring fair task scheduling and eventual completion. This mechanism ensures reliable data write-to-disk while maintaining high overall system throughput and low latency through resource reuse, fine-grained control, and asynchronous design.

[0041] It should be understood that, although Figure 1 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 1 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0042] In a second embodiment, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it performs the following steps: allocating a first resource for a received read / write request, wherein the first resource is used to manage the processing status of the target data corresponding to the read / write request; dividing the target data into one or more data blocks according to a preset granularity, and allocating a second resource for each of the one or more data blocks; determining a third resource corresponding to the address of the data block corresponding to the second resource, wherein the third resource is used to maintain the data status and access permissions within the corresponding address range; in response to a write request, writing one or more data blocks to the second resource respectively, and determining a first access permission status of the third resource, so as to determine whether to perform a write merging operation based on the first access permission status, wherein the write merging operation is used to merge the data in the second resource into the third resource; in response to the second resource having completed writing all data, returning a write completion response to the host; in response to a read request, determining a second access permission status of the third resource, and determining whether to read data from the third resource or backend storage to the corresponding second resource based on the second access permission status, and aggregating the data in the second resource and returning it to the host; in response to the second resource having completed returning all data, returning a read completion response to the host.

[0043] When the program instructions are read and executed by one or more processors, they can also perform operations corresponding to the steps in the above method embodiments, as described above, and will not be repeated here. Reference Figure 2 This example illustrates the architecture of an electronic device, which may include a processor 210, a video display adapter 211, a disk drive 212, an input / output interface 213, a network interface 214, and a memory 220. The processor 210, video display adapter 211, disk drive 212, input / output interface 213, network interface 214, and memory 220 can communicate with each other via a communication bus 230.

[0044] The processor 210 can be implemented using a general-purpose central processing unit (CPU), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solution provided in this application.

[0045] The memory 220 can be implemented as a read-only memory (ROM), random access memory (RAM), static storage device, dynamic storage device, etc. The memory 220 can store the operating system 221 for controlling the operation of the electronic device 200, and the basic input / output system (BIOS) 222 for controlling the low-level operations of the electronic device 200. Additionally, it can store a web browser 223, data storage management 224, and an icon / font processing system 225. The icon / font processing system 225 can be the application program that specifically implements the aforementioned steps in this embodiment. In summary, when implementing the technical solution provided in this application through software or firmware, the relevant program code is stored in the memory 220 and executed by the processor 210.

[0046] Input / output interface 213 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components in the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touch screens, microphones, various sensors, etc., and output devices may include displays, speakers, vibrators, indicator lights, etc.

[0047] Network interface 214 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0048] Bus 230 includes a pathway for transmitting information between various components of the device, such as processor 210, video display adapter 211, disk drive 212, input / output interface 213, network interface 214, and memory 220.

[0049] In addition, the electronic device 200 can also obtain information on specific acquisition conditions from a virtual resource object acquisition condition information database (not shown in the figure) for condition judgment.

[0050] It should be noted that although the above-described electronic device 200 only shows a processor 210, a video display adapter 211, a disk drive 212, an input / output interface 213, a network interface 214, a memory 220, and a bus 230, in specific implementations, the electronic device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the solution of this application, and does not necessarily include all the components shown in the figures.

[0051] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause an electronic device (which may be a personal computer, cloud server, or network device, etc.) to execute the methods of various embodiments or some parts of the embodiments of this application.

[0052] In a third embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, it performs the following steps: allocating a first resource for a received read / write request, wherein the first resource is used to manage the processing status of the target data corresponding to the read / write request; dividing the target data into one or more data blocks according to a preset granularity, and allocating a second resource for each of the one or more data blocks; determining a third resource corresponding to the address of the data block corresponding to the second resource, wherein the third resource is used to maintain the data status and access permissions within the corresponding address range; in response to a write request, writing one or more data blocks to the second resource respectively, and determining the first access permission status of the third resource, so as to determine whether to perform a write merging operation based on the first access permission status, wherein the write merging operation is used to merge the data in the second resource into the third resource; in response to the second resource having completed data writing, returning a write completion response to the host; in response to a read request, determining the second access permission status of the third resource, and determining whether to read data from the third resource or backend storage to the corresponding second resource based on the second access permission status, and aggregating the data in the second resource and returning it to the host; in response to the second resource having completed data return, returning a read completion response to the host.

[0053] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0054] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0055] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application.

[0056] In a fourth embodiment, a computer program product is provided, on which a computer program is stored. When the computer program is executed by a processor, it performs the following steps: allocating a first resource for a received read / write request, wherein the first resource is used to manage the processing status of the target data corresponding to the read / write request; dividing the target data into one or more data blocks according to a preset granularity, and allocating a second resource for each of the one or more data blocks; determining a third resource corresponding to the address of the data block corresponding to the second resource, wherein the third resource is used to maintain the data status and access permissions within the corresponding address range; in response to a write request, writing one or more data blocks to the second resource respectively, and determining the first access permission status of the third resource, so as to determine whether to perform a write merging operation based on the first access permission status, wherein the write merging operation is used to merge the data in the second resource into the third resource; in response to the second resource having completed writing all data, returning a write completion response to the host; in response to a read request, determining the second access permission status of the third resource, and determining whether to read data from the third resource or backend storage to the corresponding second resource based on the second access permission status, and aggregating the data in the second resource and returning it to the host; in response to the second resource having completed returning all data, returning a read completion response to the host. In a fourth embodiment, a computer program product is also provided, including a non-volatile computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it performs the following steps: allocating a first resource for a received read / write request, wherein the first resource is used to manage the processing state of the target data corresponding to the read / write request; dividing the target data into one or more data blocks according to a preset granularity, and allocating a second resource for each of the one or more data blocks; determining a third resource corresponding to the address of the data block corresponding to the second resource, wherein the third resource is used to maintain the data state and access permissions within the corresponding address range; and responding to the read / write request by writing a write request. The method requests the following steps: Write one or more data blocks to a second resource, determine the first access permission status of a third resource, and determine whether to perform a write merge operation based on the first access permission status. The write merge operation merges data from the second resource into the third resource. In response to the second resource having completed writing data, a write completion response is returned to the host. In response to a read request being a read request, the second access permission status of the third resource is determined, and based on the second access permission status, it is determined whether to read data from the third resource or backend storage into the corresponding second resource, and the data in the second resource is aggregated and returned to the host. In response to the second resource having completed writing data, a read completion response is returned to the host. Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer program product, and when executed, it can include the processes of the embodiments of the above methods.

[0057] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0058] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application.

Claims

1. A data read / write processing method in a storage system, characterized in that, The method includes: Allocate a first resource for the received read / write request, wherein the first resource is used to manage the processing status of the target data corresponding to the read / write request; The target data is divided into one or more data blocks according to a preset granularity, and a second resource is allocated to each of the one or more data blocks. A third resource corresponding to the data block address corresponding to the second resource is determined, wherein the third resource is used to maintain the data status and access permissions within the corresponding address range; In response to the read / write request being a write request, the one or more data blocks are written to the second resource respectively, and the first access permission status of the third resource is determined, so as to determine whether to perform a write merge operation based on the first access permission status, wherein the write merge operation is used to merge the data in the second resource into the third resource; In response that all data has been written to the second resource, a write completion response is returned to the host; In response to the read / write request being a read request, the second access permission status of the third resource is determined, and based on the second access permission status, it is determined whether to read data from the third resource or backend storage to the corresponding second resource, and the data in the second resource is aggregated and returned to the host; In response to the fact that the second resource has completed the data return, a read completion response is returned to the host.

2. The method according to claim 1, characterized in that, The step of dividing the target data into one or more data blocks according to a preset granularity, and allocating second resources to each of the one or more data blocks, includes: Based on the data range of the target data, determine multiple integer multiple address intervals required to cover the data range, wherein the size of the integer multiple address intervals corresponds to the preset granularity; Based on the starting position of the target data within the address range that is an integer multiple of the address of the second resource, the offset parameter of the second resource is determined; The data volume parameter of the second resource is determined based on the amount of target data within the address range that is an integer multiple of the address of the second resource. Based on the offset parameter and the data volume parameter, determine the memory space required for the second resource, and allocate a corresponding amount of memory space for the second resource.

3. The method according to claim 2, characterized in that, The step of allocating memory space of a corresponding size to the second resource includes: Based on the memory space required by the second resource, determine the number of complete memory pages required by the second resource; Based on the number of complete memory pages, multiple memory pages, either contiguous or non-contiguous, are requested from the memory resource pool, wherein the memory resource pool manages memory space in units of memory pages of a preset data size; The logical or physical addresses of the plurality of memory pages are assigned to the storage pointer of the second resource, wherein the storage pointer is used to instruct the second resource to access the corresponding memory space.

4. The method according to claim 3, characterized in that, Determining the number of complete memory pages required by the second resource based on the memory space size required by the second resource includes: The conversion factor is determined based on the memory page size and the minimum storage unit size of the target data; Divide the offset parameter by the conversion factor and round down to obtain the starting memory page boundary; Divide the sum of the offset parameter and the data volume parameter by the conversion factor and round up to obtain the end memory page boundary; Subtracting the starting memory page boundary from the ending memory page boundary and then adding one yields the total number of memory pages required by the second resource.

5. The method according to claim 1, characterized in that, The step of determining the third resource corresponding to the data block address corresponding to the second resource includes: Based on the data block address of the second resource, determine the corresponding integer multiple address range, and search in the corresponding resource pool to see if there is a third resource that manages the corresponding integer multiple address range; In response to the existence of a third resource that manages an address range that is an integer multiple of the corresponding address range, the found third resource is determined to be the third resource corresponding to the data block address of the second resource; In response to the absence of a third resource that manages the address range that is an integer multiple of the corresponding address range, a new third resource is requested from the corresponding resource pool, and the address range that is an integer multiple of the corresponding address range is assigned to the new third resource, so that the new third resource is used as the third resource corresponding to the data block address of the second resource.

6. The method according to claim 1, characterized in that, Determining the first access permission status of the third resource, or determining the second access permission status of the third resource, includes: Acquire the spinlock of the third resource, and determine the current lock state and write wait count of the third resource; In response to the read / write request being a write request and the current lock state being idle, the access permission state of the third resource is determined to be write exclusive, the lock state of the third resource is modified to write exclusive, and the write wait count is incremented. In response to the read / write request being a read request, the current lock state being the idle state, and the write wait count being zero, the access permission state of the third resource is determined to be a read-shared state, the lock state of the third resource is modified to the read-shared state, and the read-shared count of the third resource is incremented. In response to the read / write request being a write request, and the current lock state being either a write exclusive state or a read shared state, the access permission state of the third resource is determined to be a prohibited access state, and the write wait count is incremented; In response to the read / write request being a read request, and the current lock state being either a write exclusive state or a read shared state, the access permission state of the third resource is determined to be the prohibited access state. In response to the read / write request being a read request and the write wait count being non-zero, the access permission status of the third resource is determined to be the prohibited access status; Release the spinlock of the third resource; The write wait count is used to record the number of second resources in the third resource that are waiting or performing exclusive write access, and the read share count is used to record the number of second resources in the third resource that are performing read share access.

7. The method according to claim 6, characterized in that, The step of determining whether to perform a write merge operation based on the first access permission status includes: In response to the first access permission state being the prohibited access state, the write merge operation is prohibited, the second resource is added to the waiting queue of the third resource, and the write task of the second resource is marked as completed; In response to the first access permission state being the write exclusive state, it is determined to perform the write merge operation, merge the data in the second resource into the third resource, and update the data status identifier of the third resource; The data status identifier includes at least a valid data identifier and a dirty data identifier. The valid data identifier is used to indicate whether the data unit in the third resource is valid, and the dirty data identifier is used to indicate whether the data unit in the third resource has been modified and needs to be written to the backend storage.

8. The method according to claim 6 or 7, characterized in that, The step of determining whether to read data from the third resource or backend storage to the corresponding second resource based on the second access permission status includes: In response to the second access permission status being the prohibited access status, reading data from the third resource or the backend storage to the corresponding second resource is prohibited, and the second resource is added to the waiting queue of the third resource; In response to the second access permission state being the read shared state, based on the offset parameter of the second resource, the data volume parameter, and the valid data identifier of the third resource, it is determined whether the data in the third resource meets the data requirements of the second resource; In response to the fact that the data in the third resource meets the data requirements of the second resource, data is read from the third resource into the corresponding second resource; In response to the fact that the data in the third resource does not meet the data requirements of the second resource, data is read from the backend storage and sent to the corresponding second resource.

9. The method according to claim 8, characterized in that, After reading data from the third resource to the corresponding second resource, or after reading data from the backend storage to the corresponding second resource, the method further includes: Acquire the spinlock of the third resource and decrement the read share count of the third resource; In response to the read-share count being zero after being decremented, the lock state of the third resource is changed from the read-share state to the idle state; Release the spinlock of the third resource.

10. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the steps of the data read / write processing method in the storage system as described in any one of claims 1 to 9.

Images