Data storage method and device, data reading method and device, electronic equipment and storage medium

Abstract

The invention provides a data storage method and device, a data reading method and device, electronic equipment and a storage medium, and the data storage method comprises the steps: writing to-be-stored data and a physical block address of the to-be-stored data into a data disk refreshing unit corresponding to a logic storage unit under the condition that the to-be-stored data is written into the logic storage unit; under the condition that the data in the data disk refreshing unit corresponding to the logic storage unit reaches a capacity threshold value, writing the data in the data disk refreshing unit corresponding to the logic storage unit into a solid-state storage medium; the corresponding relation between the logic storage unit and the data disk refreshing unit is predefined. According to the data storage method and device, the data reading method and device, the electronic equipment and the storage medium provided by the invention, data merging and disk falling of logic storage unit granularity can be realized based on the temporal locality, the spatial locality and the sequential locality of the data, the distribution continuity of the data on the solid-state storage medium can be improved, and the storage efficiency is improved. The access efficiency and the reading efficiency of data reading can be improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a data storage, data reading method, device, electronic equipment and storage medium. Background Art

[0002] All-Flash Array (AFA) is a high-performance storage system composed entirely of solid-state storage media (SSD). Through LSA (Log-Structured Architecture) or similar log structure architectures, all-flash disk arrays can realize the conversion of random small blocks of data into sequential large blocks, and use append writes instead of overwrite writes, which can improve the input / output (IO) performance of the system while increasing the life of the back-end solid-state storage media, thereby improving the reliability of all-flash disk arrays.

[0003] The LSA architecture can fully manage the data organization of the back-end disk array space, and maintain the correspondence between the logical address and the back-end physical address by maintaining a large number of logical to metadata physical (LP) mappings, thus realizing automatic thin configuration and on-demand allocation of the back-end solid-state storage media space. The LSA architecture can achieve data reduction such as data deduplication and compression by maintaining metadata such as physical layout (PL) and hash or compression parameters (HP). By intelligently selecting compression algorithms and deduplication strategies, and optimizing the storage and access methods of these metadata, the space utilization and performance of the back-end solid-state storage media can be further improved.

[0004] However, in the related technology, when data is stored under the LSA architecture, it may cause discontinuous distribution of data on the solid-state storage medium. This discontinuity will seriously affect the access efficiency and reading efficiency of subsequent data reading, especially when performing pre-read operations, more disk sectors need to be read to obtain the required data, thereby increasing the disk reading amount.

[0005] In addition, according to the principles of temporal locality, spatial locality, and sequential locality of data, IO requests issued in the same time period are usually accessed at the same time, so batch reading can be used to improve performance. However, the data integration (merge) mechanism of the data flushing unit (FLU) under the LSA architecture breaks the logical or physical locality of the data, causing the data that could have been read in batches to become scattered, thereby reducing the access efficiency and reading efficiency of data reading.

[0006] Therefore, how to improve the access efficiency and reading efficiency of data reading by optimizing the data storage strategy under the LSA architecture is a technical problem that needs to be solved urgently in this field. Summary of the invention

[0007] The present invention provides a data storage and data reading method, device, electronic device and storage medium, which are used to solve the defect that the data storage strategy under the LSA architecture in the prior art may affect the access efficiency and reading efficiency of subsequent data reading, and improve the access efficiency of data reading by optimizing the data storage strategy under the LSA architecture.

[0008] The present invention provides a data storage method, comprising the following steps.

[0009] In the case where the data to be stored has been written into the logical storage unit, generating a physical block address of the data to be stored based on the logical sequence address of the data to be stored; Writing the data to be stored and the physical block address of the data to be stored into the data flushing unit corresponding to the logical storage unit; When the data in the data flushing unit corresponding to the logical storage unit reaches a capacity threshold, writing the data in the data flushing unit corresponding to the logical storage unit into a solid-state storage medium; Among them, the corresponding relationship between the logical storage unit and the data disk flushing unit is predefined.

[0010] According to a data storage method provided by the present invention, after generating the physical block address of the data to be stored based on the logical sequence address of the data to be stored, the method further comprises: When it is determined based on the physical block addresses of the data to be stored that the physical block addresses of multiple data to be stored written into the logical storage unit are continuous, the multiple data to be stored are determined as a continuous address space data group, and the real address of each data to be stored in the continuous address space data group is maintained in a tree data structure maintained by the metadata physical mapping.

[0011] According to a data storage method provided by the present invention, after determining the plurality of data to be stored as a continuous address space data group, the method further comprises: storing each to-be-stored data in the continuous address space data group into a cache space in a write cache state, and performing power-off protection processing on each to-be-stored data in the continuous address space data group; storing each to-be-stored data in the continuous address space data group and the physical block address of each to-be-stored data in the continuous address space data group to the opposite end storage device of the cache space; After writing the data in the data flushing unit corresponding to the logical storage unit into the solid-state storage medium, the method further includes: The cache space is switched to a read cache state.

[0012] According to a data storage method provided by the present invention, when the data to be stored has been written into a logical storage unit, before generating a physical block address of the data to be stored based on the logical sequence address of the data to be stored, the method further comprises: Determine a data flushing unit corresponding to the logical storage unit based on the historical data flow of the logical storage unit; The determining, based on the historical data flow of the logical storage unit, a data flushing unit corresponding to the logical storage unit comprises: Obtaining historical data flow of the logical storage unit; When the historical data flow of the logical storage unit exceeds the data flow threshold, the logical storage unit is determined as a first-type logical storage unit; when the historical data of the logical storage unit does not exceed the data flow threshold, the logical storage unit is determined as a second-type logical storage unit; For the first-type logical storage units, an idle data disk flushing unit is allocated to each of the first-type logical storage units as the data disk flushing unit corresponding to each of the first-type logical storage units. For the second logical storage units, based on the historical data traffic of each of the second-type logical storage units, multiple second-type logical storage units are determined as a logical storage unit group, and then an idle data disk flushing unit is allocated to each logical storage unit group as the data disk flushing unit corresponding to each second-type logical storage unit in each of the logical storage unit groups.

[0013] The present invention also provides a data reading method, comprising the following steps: In case of receiving a data read request including a physical block address of data to be read, judging whether the data to be read is data in a continuous address space data group based on the physical block address of the data to be read; In the case where it is determined that the data to be read is data in a continuous address space data group, searching in a cache space in a read cache state whether the continuous address space data group exists; In the case where it is determined that the continuous address space data group exists in the cache space, extracting the to-be-read data from the cache space; The data to be read and other data in the continuous address space data group where the data to be read is located are stored based on the data storage method described above.

[0014] According to a data reading method provided by the present invention, after searching whether the continuous address space data group exists in the cache space in the read cache state, the method further includes: When it is determined that the continuous address space data group does not exist in the cache space, the continuous address space data group is pre-read into the cache space according to the granularity of the redundant array of independent disks.

[0015] The present invention also provides a data storage device, comprising the following modules: An address generation module, configured to generate a physical block address of the data to be stored based on a logical sequence address of the data to be stored when the data to be stored has been written into the logical storage unit; A data writing module, used for writing the data to be stored and the physical block address of the data to be stored into the data flushing unit corresponding to the logical storage unit; A data flushing module, used for writing the data in the data flushing unit corresponding to the logical storage unit into a solid-state storage medium when the data in the data flushing unit corresponding to the logical storage unit reaches a capacity threshold; Among them, the corresponding relationship between the logical storage unit and the data disk flushing unit is predefined.

[0016] The present invention also provides a data reading device, comprising the following modules: A data parsing module, configured to, upon receiving a data read request including a physical block address of data to be read, determine whether the data to be read is data in a continuous address space data group based on the physical block address of the data to be read; A data search module, configured to search for the existence of the continuous address space data group in the cache space in the read cache state when it is determined that the data to be read is data in the continuous address space data group; A data extraction module, configured to extract the to-be-read data from the cache space when it is determined that the continuous address space data group exists in the cache space; The data to be read and other data in the continuous address space data group where the data to be read is located are stored based on the data storage method described above.

[0017] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the processor implements any one of the above-mentioned data storage methods and / or the data reading method.

[0018] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-mentioned data storage methods and / or the above-mentioned data reading methods.

[0019] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any one of the above-mentioned data storage methods and / or the above-mentioned data reading methods.

[0020] The data storage and data reading method, device, electronic device and storage medium provided by the present invention can write the data to be stored into a data disk flushing unit corresponding to the logical storage unit after generating the physical block address of the data to be stored based on the logical sequence address of the data to be stored when the data to be stored has been written into the logical storage unit. Furthermore, when the data in the data disk flushing unit corresponding to the logical storage unit reaches the capacity threshold, the data in the data disk flushing unit corresponding to the logical storage unit is written into a solid-state storage medium. Based on the principles of temporal locality, spatial locality and sequential locality of data, data merging and storage at the granularity of logical storage units can be realized, the data distribution continuity on the solid-state storage medium can be improved, the disk reading amount for reading continuous data can be reduced, the access efficiency and reading efficiency of data reading can be improved, and the performance of the storage system can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0022] Figure 1 It is a schematic diagram of the data structure under the LSA architecture in the related technology.

[0023] Figure 2 It is a flow chart of the data storage method provided by the present invention.

[0024] Figure 3 This is one of the data structure diagrams under the LSA architecture in the data storage method provided by the present invention.

[0025] Figure 4 This is the second data structure schematic diagram under the LSA architecture in the data storage method provided by the present invention.

[0026] Figure 5 It is a flow chart of the data reading method provided by the present invention.

[0027] Figure 6 It is a structural schematic diagram of the data storage device provided by the present invention.

[0028] Figure 7 It is a structural schematic diagram of the data reading device provided by the present invention.

[0029] Figure 8 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0031] In the description of the invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0032] In the description of the present application, the terms "first", "second", etc. are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are usually a class, and the number of objects is not limited. For example, the first object can be one or more. In addition, in the description of the present application, "and / or" represents at least one of the connected objects, and the character " / " generally represents that the front and back associated objects are in an "or" relationship.

[0033] It should be noted that the all-flash disk array is based on the data organization method of the LSA architecture. It uses an append write method to convert random small blocks of data into sequential large blocks and write them to the disk. This can reduce the write amplification of the redundant array of independent disks (RAID). Full stripe write can fully utilize the RAID performance; it can also fully utilize the large block write characteristics of solid-state storage media, reduce junk data, reduce disk wear, and increase the life of solid-state storage media.

[0034] Figure 1 This is a schematic diagram of the data structure under the LSA architecture in the related technology. Figure 1As shown, the LSA architecture in the related technology implements space management of different granularities. The main management units include: space management unit BLOCK (128M), data flush unit FLU (2N) and grain (8K), which reduce the granularity successively, similar to the space division and management inside the solid-state storage medium disk.

[0035] BLOCK is the largest management unit and the basic unit of garbage collection. By calculating the inefficiency (i.e. the proportion of invalid data) in BLOCK, when a certain threshold is reached, the system will migrate the data and recycle the entire BLOCK to improve data utilization and storage performance. At the same time, the back-end trim operation also helps to further optimize the use of storage space.

[0036] FLU is a management unit for stripe width settings for typical RAID loads. The FLU setting fully utilizes the full stripe write principle of RAID, reduces RAID write amplification, and thus improves system performance. The granularity of FLU is 2M, which can effectively manage data while avoiding the extra overhead caused by too small management units.

[0037] Grain is the smallest unit of data and data management, and the granularity of grain is 8K. Grain plays a key role in data reading, writing, recovery, and data disaster recovery and reconstruction. FLU consists of a large number of grains, each of which consists of two parts: header and data. The header is used to identify the reverse metadata block to ensure data consistency and recoverability.

[0038] The LSA architecture divides data, maintains metadata, and performs bitmap management according to grains. Through BLOCK-level garbage collection and trim operations, the system can continuously reclaim invalid space and improve storage space utilization. The FLU setting makes full use of the characteristics of RAID, reduces write amplification, and improves the overall performance of the system. At the same time, grain-level data processing and metadata management ensure the efficiency and accuracy of data reading and writing. In addition, flexible data recovery capabilities: Since each grain contains a complete header and data part, the system can quickly locate and recover data during data disaster recovery and reconstruction, ensuring data reliability and availability.

[0039] The LSA architecture achieves efficient reading, writing, management, and recovery of data through fine data unit division and efficient space management strategies. The three management units of different granularities, BLOCK, FLU, and grain, cooperate with each other to form the basis of high performance and high reliability of the system. This architectural design not only improves the utilization of storage space and the overall performance of the system, but also enhances the reliability and recovery capabilities of data.

[0040] The data storage strategy under the LSA architecture includes: when the host data reaches the LSA log volume (LSFS processing module), the LSA log volume will first allocate a BLOCK and select an idle FLU from the BLOCK to store the data.

[0041] The LSA log volume merges IO requests from different logical volumes (LUNs) into the same FLU. This merger can reduce the number of accesses to the storage medium and improve write efficiency. The merged data is organized according to grains. Here, grain refers to the smallest unit of data organization, which can be bytes, sectors, or other larger units. At the same time, each FLU will insert a header to record the metadata of the FLU, such as timestamp, serial number, data type, etc.

[0042] When the data in the FLU reaches its capacity limit, the FLU is considered full. The LSA log volume is full. The data in the FLU will be written to the RAID. In RAID, data is usually distributed to multiple disks in a stripe manner. When all the data in a stripe is written, the data in the stripe will be written to the disk together (that is, written to the solid-state storage medium) to ensure data persistence.

[0043] In the LSA architecture, FLU is organized into a pool as the basic unit of data organization. This pool can span multiple physical storage devices to form a logical storage resource pool. The IO of all logical volumes (LUNs) will be mixed and stored in the FLU pool. This hybrid storage method can further improve the utilization of storage resources, because the IO requests of different LUNs may overlap in time and space. By merging these requests, the number of accesses to the storage medium can be reduced.

[0044] The core of the basic process of data storage under the above-mentioned LSA architecture is to realize the random-to-sequential conversion of pool granularity, and full-stripe write fully utilizes the write amplification effect of the back-end solid-state storage medium. However, the related technology has the following defects in data storage under the LSA architecture: First, under the LSA architecture, all LUN data is written to the same FLU. This method may cause discontinuous distribution of data on the solid-state storage medium for sequential or random small blocks of data. This discontinuity causes the system to require more disk seek operations to access discontinuous data blocks in subsequent read operations, especially prefetch operations, which increases the disk read volume, thereby seriously affecting the access efficiency and read efficiency of data reading.

[0045] Second, according to the principles of temporal locality, spatial locality, and sequential locality of data, IO requests issued in the same time period are usually accessed at the same time, so batch reading can be used to improve performance. However, the data integration (merge) mechanism of FLU under the LSA architecture breaks the logical or physical locality of data, causing data that could have been read in batches to become scattered, thereby reducing the access efficiency and reading efficiency of data reading.

[0046] Among them, temporal locality means that if a certain data is accessed, it is likely to be accessed again in the near future; spatial locality means that if a certain data is accessed, the data adjacent to it may also be accessed in the near future. Sequential locality means that when accessing data sequentially, the next data item accessed is usually the physical or logical successor of the current data.

[0047] Third, before the IO operation of writing data to the FLU under the LSA architecture is completed, the host thread or process that initiates the above IO operation will be blocked and unable to continue to perform subsequent operations until the above IO operation is completed and the result is returned, resulting in the host's computing resources being unable to be effectively utilized while waiting for the above IO operation to be completed.

[0048] Moreover, if the above IO operation cannot be completed immediately, the thread or process that initiated the above IO operation will be suspended (i.e., enter a waiting or sleeping state) until the above IO operation is completed. During the waiting period, the thread or process cannot perform any other operations. This method will directly lead to an increase in the host IO response latency, because the host must wait for the IO operation to be completed before continuing to process other tasks, which seriously increases the host IO response latency. Among them, the host IO response latency refers to the time difference between the host issuing an IO request and receiving a response to the completion of the IO operation. The host IO response latency includes the execution time of the IO operation itself and any time increased due to synchronization or blocking waiting.

[0049] In this regard, the present invention provides a data storage method. The data storage method provided by the present invention aims at the technical problem that the data integration mechanism of the FLU of the original pool granularity cannot fully identify the host semantics and data flow, and the data of different LUNs are mixed together, which is not conducive to data reading. Based on the principles of temporal locality, spatial locality and sequential locality and the access characteristics of data sequences, the data storage strategy under the LSA architecture is optimized, and the data merging and storage of LUN granularity are realized. The data is pre-read into the cache by utilizing the characteristics that the data is written and accessed at the same time, and the access efficiency and reading efficiency of the data reading are improved. For the same data flow, large blocks are pre-read into the cache to improve the subsequent cache read hit rate, and the host IO response delay under the LSA architecture can be reduced by using the write cache synchronization, which can improve the storage system performance.

[0050] Combine the following Figure 1-Figure 5 The data storage method and data reading method of the present invention are described.

[0051] Figure 2 It is a flow chart of the data storage method provided by the present invention. Figure 2 As shown, the method includes the following steps: Step 201: when the data to be stored has been written into the logical storage unit, a physical block address of the data to be stored is generated based on the logical sequence address of the data to be stored.

[0052] It should be noted that the data storage device that is the execution subject of the embodiment of the present invention can be configured in electronic devices such as computers and servers.

[0053] It should be noted that the host-side logical storage unit LUN is the basic unit for data service bearing. The logical storage unit LUN provides a storage interface independent of the underlying solid-state storage medium for upper-layer applications (such as databases, file systems, etc.). Through the logical storage unit LUN, applications can easily store, access and manage data.

[0054] On the host side, a data stream to be operated may contain a mixture of small blocks of randomly accessed write data and large blocks of sequentially accessed write data. Small blocks of randomly accessed write data usually occur when discontinuous data blocks need to be accessed, such as database index access. Large blocks of sequentially accessed write data are common in scenarios such as batch data processing and file transfer. When the data is accessed again, its access pattern is likely to follow the characteristics of the data when it was written. For example, if the data is written sequentially, then in subsequent read operations, the data is also likely to be accessed sequentially.

[0055] Therefore, unlike the data storage strategy in the related art in which the data of all logical storage units LUN are written into the same FLU under the LSA architecture, in the data storage method provided by the present invention, when accessing data for each logical storage unit LUN, a data flushing unit FLU is applied from the resource pool as the data flushing unit FLU corresponding to each logical storage unit LUN. The data flushing unit FLU corresponding to any logical storage unit LUN is used to store the data of the above logical storage unit LUN.

[0056] Specifically, for any logical storage unit LUN i , after the data to be stored has been written into the logical storage unit LUN iIn the case of a physical block address (PBA) of the data to be stored, the physical block address (PBA) of the data to be stored can be generated based on the logical sequence address (LSA) of the data to be stored according to the predefined address mapping mechanism and algorithm; wherein, i Represents any non-negative integer.

[0057] Step 202: write the data to be stored and the physical block address of the data to be stored into the data flushing unit corresponding to the logical storage unit; Among them, the corresponding relationship between the logical storage unit and the data disk flushing unit is predefined.

[0058] Specifically, after the physical block address of the data to be stored is generated, the data to be stored and the physical block address of the data to be stored can be written into the logical storage unit LUN. i In the corresponding data flush disk unit.

[0059] It should be noted that in the embodiment of the present invention, when the data to be stored and the physical block address of the data to be stored are written into the data flushing unit corresponding to the logical storage unit, according to the data storage strategy under the LSA architecture, the order of small block random services is randomly changed, large blocks are written to the disk to reduce write amplification, and full stripes are written to the disk.

[0060] As an optional embodiment, when the data to be stored has been written into the logical storage unit, before generating the physical block address of the data to be stored based on the logical sequence address of the data to be stored, the method also includes: determining the data flushing unit corresponding to the logical storage unit based on the historical data flow of the logical storage unit.

[0061] It should be noted that the logical storage unit LUN i The corresponding data flushing unit can be a logical storage unit LUN in which the data to be stored is written i Previously, based on logical storage unit LUN i The historical data flow is determined.

[0062] In the logical storage unit LUN i When the historical data flow is small, the logical storage unit LUN can be allocated based on the idle status of the data flushing unit FLU in the BLOCK. i The corresponding data flushing unit and the data flushing units corresponding to other logical storage units LUNs with smaller historical data traffic are determined to be the same idle data flushing unit.

[0063] In the logical storage unit LUN iIn the case of large historical data traffic, an idle data flushing unit FLU can be determined as a logical storage unit LUN based on the idle status of the data flushing unit FLU in the BLOCK. i Corresponding data flushing unit.

[0064] As an optional embodiment, determining a data flushing unit corresponding to a logical storage unit based on historical data traffic of the logical storage unit includes: acquiring historical data traffic of the logical storage unit.

[0065] Specifically, in the embodiment of the present invention, the logical storage unit LUN can be obtained by means of data query. i Historical data flow.

[0066] It should be noted that the logical storage unit LUN in the embodiment of the present invention is i The historical data traffic can be written to the logical storage unit LUN when the data to be stored is stored i The data flow within the previously preset time period. The above preset time period may be determined based on prior knowledge and / or actual conditions, and the above preset time period is not specifically limited in the embodiments of the present invention.

[0067] When the historical data traffic of the logical storage unit exceeds the data traffic threshold, the logical storage unit is determined as a first type of logical storage unit. When the historical data of the logical storage unit does not exceed the data traffic threshold, the logical storage unit is determined as a second type of logical storage unit.

[0068] Specifically, obtain the logical storage unit LUN i After comparing the historical data traffic of the logical storage unit LUN i The size of the historical data flow and the data flow threshold.

[0069] If the logical storage unit LUN i If the historical data traffic exceeds the data traffic threshold, the logical storage unit LUN can be determined. i The historical data flow is large, so the logical storage unit LUN i is determined as a first-type logical storage unit.

[0070] If the logical storage unit LUN i If the historical data traffic does not exceed the data traffic threshold, the logical storage unit LUN can be determined. i The historical data traffic is small, so the logical storage unit LUN i Determined as a second-type logical storage unit.

[0071] It should be noted that the data flow threshold in the embodiment of the present invention may be determined based on prior knowledge and / or actual conditions. The specific value of the data flow threshold in the embodiment of the present invention is not limited.

[0072] For the first type of logical storage units, an idle data disk flushing unit is allocated to each first type of logical storage unit as the data disk flushing unit corresponding to each first type of logical storage unit. For the second logical storage units, based on the historical data traffic of each second type of logical storage unit, multiple second type logical storage units are determined as a logical storage unit group, and then an idle data disk flushing unit is allocated to each logical storage unit group as the data disk flushing unit corresponding to each second type logical storage unit in each logical storage unit group.

[0073] Specifically, when determining the logical storage unit LUN i In the case of the first type of logical storage unit, it can be a logical storage unit LUN i Allocate an idle data flush unit FLU as a logical storage unit LUN i The corresponding data flush unit FLU.

[0074] Determine the logical storage unit LUN i In the case of a second type of logical storage unit, the logical storage unit LUN can be determined based on i The historical data traffic of the logical storage unit LUN i It is determined as a logical storage unit group with several other logical storage units, and then an idle data flushing unit FLU can be allocated to the above logical storage unit group as the data flushing unit FLU corresponding to each logical storage unit in the above logical storage unit, and then the logical storage unit LUN can be determined. i The corresponding data flush unit FLU.

[0075] Figure 3 This is one of the data structure diagrams under the LSA architecture in the data storage method provided by the present invention. Figure 3 As shown, in the related art, a data flushing unit FLU stores data of multiple logical storage units LUN. In the data storage method provided in the present application, a data flushing unit FLU only stores data in logical storage units LUN1 and LUN4, and another data flushing unit FLU only stores data in logical storage unit LUN3.

[0076] The embodiment of the present invention allocates an idle data flushing unit FLU to the logical storage unit LUN based on the historical data traffic of the logical storage unit LUN. As the data flushing unit FLU corresponding to the logical storage unit LUN, the data in some logical storage unit LUNs with smaller historical data traffic can be fully merged into one data flushing unit FLU, thereby reducing the resource occupancy and time waiting of the data flushing unit.

[0077] Step 203: When the data in the data flushing unit corresponding to the logical storage unit reaches a capacity threshold, the data in the data flushing unit corresponding to the logical storage unit is written into a solid-state storage medium.

[0078] Specifically, in the logical storage unit LUN i When the data in the corresponding data flushing unit FLU reaches the capacity threshold, the logical storage unit LUN can be i The data in the corresponding data flushing unit FLU is written into the solid-state storage medium SSD.

[0079] It should be noted that the value of the capacity threshold in the embodiment of the present invention can be determined based on the maximum data capacity of the data flushing unit FLU. For example, the capacity threshold can be the maximum data capacity of the data flushing unit FLU; or, the capacity threshold can also be 90% of the maximum data capacity of the data flushing unit FLU. The value of the capacity threshold is not specifically limited in the embodiment of the present invention.

[0080] The embodiment of the present invention writes the data to be stored into a data flush unit corresponding to the logical storage unit after generating a physical block address of the data to be stored based on the logical sequence address of the data to be stored when the data to be stored has been written into the logical storage unit. Furthermore, when the data in the data flush unit corresponding to the logical storage unit reaches the capacity threshold, the data in the data flush unit corresponding to the logical storage unit is written into a solid-state storage medium. This can realize data merging and storage at the granularity of the logical storage unit based on the principles of temporal locality, spatial locality and sequential locality of the data, improve the distribution continuity of data on the solid-state storage medium, reduce the disk read amount for reading continuous data, improve the access efficiency and reading efficiency of data reading, and improve the performance of the storage system.

[0081] As an optional embodiment, after generating the physical block address of the data to be stored based on the logical serial address of the data to be stored, the method further includes: when it is determined that the physical block addresses of multiple data to be stored written into the logical storage unit are continuous based on the physical block address of the data to be stored, the multiple data to be stored are determined as a continuous address space data group, and the real address of each data to be stored in the continuous address space data group is maintained in the tree data structure maintained by the metadata physical mapping.

[0082] It should be noted that based on writing to the logical storage unit LUN i The physical block address of the data to be stored is determined when writing to the logical storage unit LUN i When the multiple data to be stored are continuous data, the multiple data to be stored can be determined as a continuous address space data group, and then the real addresses of the multiple data to be stored can be recorded in the Btree+ maintained by the metadata physical (LP) mapping.

[0083] Among them, Btree+ (usually referred to as B+ tree) is a self-balancing tree data structure, which is a variant of B tree and is widely used in the index structure of databases and operating systems. B+ tree supports efficient search, insertion, deletion and sequential access operations by keeping data in order.

[0084] When data is read, if it is detected that the data to be read hits the above continuous address space data group, the data in the above continuous address space data group can be sequentially extracted into the read cache.

[0085] The embodiment of the present invention determines multiple data to be stored as a continuous address space data group by determining that the physical block addresses of multiple data to be stored written into a logical storage unit are continuous based on the physical block addresses of the data to be stored, and maintains the real address of each data to be stored in the continuous address space data group in a tree data structure maintained by the metadata physical mapping. This can pre-read continuous data into the cache when reading data, improve the access efficiency and reading efficiency of data reading, improve the subsequent cache read hit rate, and further improve the performance of the storage system.

[0086] As an optional embodiment, after determining multiple data to be stored as a continuous address space data group, the method also includes: storing each data to be stored in the continuous address space data group to a cache space in a write cache state, and performing power-off protection processing on each data to be stored in the continuous address space data group.

[0087] Each to-be-stored data in the continuous address space data group and the physical block address of each to-be-stored data in the continuous address space data group are stored in the opposite end storage device of the cache space.

[0088] After writing the data in the data flushing unit corresponding to the logical storage unit into the solid-state storage medium, the method further includes: converting the cache space to a read cache state.

[0089] Figure 4 This is the second schematic diagram of the data structure under the LSA architecture in the data storage method provided by the present invention. Figure 4As shown, when multiple data to be stored are determined as a continuous address space data group, each data to be stored in the continuous address space data group is stored in a cache space in a write cache state, and power-off protection processing is performed on each data to be stored in the continuous address space data group.

[0090] After storing each data to be stored in the continuous address space data group in the cache space in the write cache state, each data to be stored in the continuous address space data group can be stored in the opposite storage device of the cache space to improve data disaster recovery and reconstruction capabilities.

[0091] Write each data to be stored in the continuous address space data group into the logical storage unit LUN i The corresponding data can be returned to the host after the disk flush unit FLU, regardless of the logical storage unit LUN i The corresponding data flushing unit FLU is stored on disk, which can reduce the host delay.

[0092] After writing the data in the data disk flushing unit corresponding to the logical storage unit into the solid-state storage medium, the above-mentioned cache space can be changed from a write cache state to a read cache state, so that when a read request for the above-mentioned data is received, the above-mentioned data can be read from the cache immediately instead of from the solid-state storage medium, thereby greatly improving the cache hit rate of access immediately after data storage, improving data reading efficiency, and using write cache to synchronously reduce the host IO response delay under the LSA architecture, thereby improving the performance of the storage system.

[0093] Figure 5 is a flow chart of the data reading method provided by the present invention. The data reading method provided by the present invention is implemented based on the above data storage method. Figure 5 As shown, the method includes: Step 501, when receiving a data read request including a physical block address of data to be read, judging whether the data to be read is data in a continuous address space data group based on the physical block address of the data to be read.

[0094] Step 502: When it is determined that the data to be read is data in a continuous address space data group, it is searched whether there is a continuous address space data group in the cache space in the read cache state.

[0095] Step 503: When it is determined that there is a continuous address space data group in the cache space, extract the data to be read from the cache space.

[0096] The data to be read and other data in the continuous address space data group where the data to be read is located are stored based on the above data storage method.

[0097] As an optional embodiment, after searching whether there is a continuous address space data group in the cache space in the read cache state, the method also includes: when it is determined that there is no continuous address space data group in the cache space, pre-reading the continuous address space data group into the cache space according to the granularity of the independent disk redundant array.

[0098] It should be noted that the specific steps of storing the data to be read and other data in the continuous address space data group where the data to be read is located can refer to the contents of the above embodiments, and will not be repeated in the embodiments of the present invention.

[0099] It should be noted that, in the case where the above-mentioned continuous address space data group is replaced in the above-mentioned cache space due to the cache elimination strategy, data pre-fetching of the above-mentioned continuous address space data group can be performed according to the stripe size and layout of the RAID configuration. Among them, RAID striping usually stores data in a dispersed manner on multiple disks to improve parallel access capabilities and fault tolerance. Therefore, pre-fetching at the stripe granularity can ensure that the data of the entire stripe can be efficiently accessed when needed. The data of the entire stripe is read from the disk and stored in the memory so that subsequent read requests can be directly satisfied from the cache.

[0100] The pre-fetched continuous address space data group is stored in the cache space, and the relevant cache metadata and the internal mapping of the data block location and status are updated.

[0101] By prefetching data, the chance of directly hitting the cache when a subsequent read request for the above-mentioned continuous address space data group is received can be increased, thereby reducing the number of accesses and delays to the solid-state storage medium. The data prefetch operation can balance the load between the solid-state storage medium and the cache, allowing the storage system to process read requests more efficiently. For applications that need to frequently access continuous data blocks (such as databases, video streams, etc.), this optimization can significantly improve the response speed and overall performance of data reading, and effectively solve the IO performance bottleneck and disaster recovery problems in small block random load scenarios. Through the integration of cache and FLU, asynchronous write process, read cache optimization, and intelligent prefetching strategy, not only the read and write performance of the system is improved, but also the reliability and availability of data are enhanced.

[0102] The embodiments of the present invention can improve the access efficiency and reading efficiency of data reading by pre-reading continuous data into cache when reading data, and can also improve the subsequent cache reading hit rate, thereby improving the performance of the storage system.

[0103] Figure 6 This is a schematic diagram of the structure of the data storage device provided by the present invention. Figure 6The data storage device provided by the present invention is described. The data storage device described below and the data storage method provided by the present invention described above can be referred to in correspondence with each other. Figure 6 As shown, the device includes: an address generation module 601, a data writing module 602 and a data flushing module 603.

[0104] The address generation module 601 is used to generate a physical block address of the data to be stored based on the logical sequence address of the data to be stored when the data to be stored has been written into the logical storage unit.

[0105] The data writing module 602 is used to write the data to be stored and the physical block address of the data to be stored into the data flushing unit corresponding to the logical storage unit.

[0106] The data flushing module 603 is used to write the data in the data flushing unit corresponding to the logical storage unit into the solid-state storage medium when the data in the data flushing unit corresponding to the logical storage unit reaches the capacity threshold.

[0107] Among them, the corresponding relationship between the logical storage unit and the data disk flushing unit is predefined.

[0108] Specifically, the address generation module 601, the data writing module 602 and the data flushing module 603 are electrically connected.

[0109] The data storage device in the embodiment of the present invention, by generating a physical block address of the data to be stored based on the logical sequence address of the data to be stored when the data to be stored has been written into the logical storage unit, writes the data to be stored into the data flushing unit corresponding to the logical storage unit, and then writes the data in the data flushing unit corresponding to the logical storage unit into the solid-state storage medium when the data in the data flushing unit corresponding to the logical storage unit reaches the capacity threshold. Based on the principles of temporal locality, spatial locality and sequential locality of data, the data merging and storage at the granularity of the logical storage unit can be realized, which can improve the distribution continuity of data on the solid-state storage medium, reduce the disk read amount for reading continuous data, improve the access efficiency and reading efficiency of data reading, and improve the performance of the storage system.

[0110] Figure 7 Schematic diagram of the structure of the data reading device provided by the present invention. Figure 7 The data reading device provided by the present invention is described. The data reading device described below and the data reading method provided by the present invention described above can be referred to each other. Figure 7 As shown, the device includes: a data parsing module 701, a data searching module 702 and a data extracting module 703.

[0111] The data analysis module 701 is used to determine whether the data to be read is data in a continuous address space data group based on the physical block address of the data to be read when receiving a data read request including the physical block address of the data to be read.

[0112] The data search module 702 is used to search whether there is a continuous address space data group in the cache space in the read cache state when it is determined that the data to be read is data in the continuous address space data group.

[0113] The data extraction module 703 is used to extract the to-be-read data from the cache space when it is determined that there is a continuous address space data group in the cache space.

[0114] The data to be read and other data in the continuous address space data group where the data to be read is located are stored based on the above data storage method.

[0115] Specifically, the data parsing module 701, the data searching module 702 and the data extracting module 703 are electrically connected.

[0116] The data reading device in the embodiment of the present invention can improve the access efficiency and reading efficiency of data reading by pre-reading continuous data into the cache when reading data, and can also improve the subsequent cache read hit rate, thereby improving the performance of the storage system.

[0117] Figure 8 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 8As shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830 and a communication bus 840, wherein the processor 810, the communication interface 820, and the memory 830 complete mutual communication through the communication bus 840. The processor 810 may call the logic instructions in the memory 830 to execute the data storage method and / or the data reading method. The data storage method includes: generating a physical block address of the data to be stored based on the logical sequence address of the data to be stored when the data to be stored has been written into the logical storage unit; writing the data to be stored and the physical block address of the data to be stored into the data brush unit corresponding to the logical storage unit; when the data in the data brush unit corresponding to the logical storage unit reaches the capacity threshold, writing the data in the data brush unit corresponding to the logical storage unit into the solid-state storage medium; wherein the corresponding relationship between the logical storage unit and the data brush unit is predefined. The data reading method includes: when receiving a data reading request including the physical block address of the data to be read, judging whether the data to be read is data in a continuous address space data group based on the physical block address of the data to be read; when it is determined that the data to be read is data in a continuous address space data group, searching in a cache space in a read cache state whether there is a continuous address space data group; when it is determined that there is a continuous address space data group in the cache space, extracting the data to be read from the cache space; wherein the data to be read and other data in the continuous address space data group where the data to be read is located are stored based on the above data storage method.

[0118] In addition, the logic instructions in the above-mentioned memory 830 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0119] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium, and when the computer program is executed by a processor, the computer can execute the data storage method and / or data reading method provided by the above methods. The data storage method includes: when the data to be stored has been written into the logical storage unit, based on the logical sequence address of the data to be stored, generating the physical block address of the data to be stored; writing the data to be stored and the physical block address of the data to be stored into the data flushing unit corresponding to the logical storage unit; when the data in the data flushing unit corresponding to the logical storage unit reaches the capacity threshold, writing the data in the data flushing unit corresponding to the logical storage unit into the solid-state storage medium; wherein the correspondence between the logical storage unit and the data flushing unit is predefined. The data reading method includes: when receiving a data reading request including the physical block address of the data to be read, judging whether the data to be read is data in a continuous address space data group based on the physical block address of the data to be read; when it is determined that the data to be read is data in a continuous address space data group, searching in a cache space in a read cache state whether there is a continuous address space data group; when it is determined that there is a continuous address space data group in the cache space, extracting the data to be read from the cache space; wherein the data to be read and other data in the continuous address space data group where the data to be read is located are stored based on the above data storage method.

[0120] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the processor executes the data storage method and / or data reading method provided by the above methods. The data storage method includes: generating a physical block address of the data to be stored based on the logical sequence address of the data to be stored when the data to be stored has been written into the logical storage unit; writing the data to be stored and the physical block address of the data to be stored into the data flushing unit corresponding to the logical storage unit; when the data in the data flushing unit corresponding to the logical storage unit reaches the capacity threshold, writing the data in the data flushing unit corresponding to the logical storage unit into the solid-state storage medium; wherein the correspondence between the logical storage unit and the data flushing unit is predefined. The data reading method includes: when receiving a data reading request including the physical block address of the data to be read, judging whether the data to be read is data in a continuous address space data group based on the physical block address of the data to be read; when it is determined that the data to be read is data in a continuous address space data group, searching in a cache space in a read cache state whether there is a continuous address space data group; when it is determined that there is a continuous address space data group in the cache space, extracting the data to be read from the cache space; wherein the data to be read and other data in the continuous address space data group where the data to be read is located are stored based on the above data storage method.

[0121] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0122] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0123] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A data storage method, characterized in that: include: In the case where the data to be stored has been written into the logical storage unit, generating a physical block address of the data to be stored based on the logical sequence address of the data to be stored; Writing the data to be stored and the physical block address of the data to be stored into the data flushing unit corresponding to the logical storage unit; When the data in the data flushing unit corresponding to the logical storage unit reaches a capacity threshold, writing the data in the data flushing unit corresponding to the logical storage unit into a solid-state storage medium; Among them, the corresponding relationship between the logical storage unit and the data disk flushing unit is predefined.

2. The data storage method according to claim 1, characterized in that: After generating the physical block address of the data to be stored based on the logical sequence address of the data to be stored, the method further includes: When it is determined based on the physical block addresses of the data to be stored that the physical block addresses of multiple data to be stored written into the logical storage unit are continuous, the multiple data to be stored are determined as a continuous address space data group, and the real address of each data to be stored in the continuous address space data group is maintained in a tree data structure maintained by the metadata physical mapping.

3. The data storage method according to claim 2, characterized in that: After determining the plurality of data to be stored as a continuous address space data group, the method further includes: storing each to-be-stored data in the continuous address space data group into a cache space in a write cache state, and performing power-off protection processing on each to-be-stored data in the continuous address space data group; storing each to-be-stored data in the continuous address space data group and the physical block address of each to-be-stored data in the continuous address space data group to the opposite end storage device of the cache space; After writing the data in the data flushing unit corresponding to the logical storage unit into the solid-state storage medium, the method further includes: The cache space is switched to a read cache state.

4. The data storage method according to any one of claims 1 to 3, characterized in that: In the case where the data to be stored has been written into the logical storage unit, before generating the physical block address of the data to be stored based on the logical sequence address of the data to be stored, the method further includes: Determine a data flushing unit corresponding to the logical storage unit based on the historical data flow of the logical storage unit; The determining, based on the historical data flow of the logical storage unit, a data flushing unit corresponding to the logical storage unit comprises: Obtaining historical data flow of the logical storage unit; When the historical data flow of the logical storage unit exceeds the data flow threshold, the logical storage unit is determined as a first-type logical storage unit; when the historical data of the logical storage unit does not exceed the data flow threshold, the logical storage unit is determined as a second-type logical storage unit; For the first-type logical storage units, an idle data disk flushing unit is allocated to each of the first-type logical storage units as the data disk flushing unit corresponding to each of the first-type logical storage units. For the second logical storage units, based on the historical data traffic of each of the second-type logical storage units, multiple second-type logical storage units are determined as a logical storage unit group, and then an idle data disk flushing unit is allocated to each logical storage unit group as the data disk flushing unit corresponding to each second-type logical storage unit in each of the logical storage unit groups.

5. A data reading method, characterized in that: include: In case of receiving a data read request including a physical block address of data to be read, judging whether the data to be read is data in a continuous address space data group based on the physical block address of the data to be read; In the case where it is determined that the data to be read is data in a continuous address space data group, searching in a cache space in a read cache state whether the continuous address space data group exists; In the case where it is determined that the continuous address space data group exists in the cache space, extracting the to-be-read data from the cache space; The data to be read and other data in the continuous address space data group where the data to be read is located are stored based on the data storage method according to claim 3.

6. The data reading method according to claim 5, characterized in that: After searching whether the continuous address space data group exists in the cache space in the read cache state, the method further includes: When it is determined that the continuous address space data group does not exist in the cache space, the continuous address space data group is pre-read into the cache space according to the granularity of the redundant array of independent disks.

7. A data storage device, characterized in that: include: An address generation module, configured to generate a physical block address of the data to be stored based on a logical sequence address of the data to be stored when the data to be stored has been written into the logical storage unit; A data writing module, used for writing the data to be stored and the physical block address of the data to be stored into the data flushing unit corresponding to the logical storage unit; A data flushing module, used for writing the data in the data flushing unit corresponding to the logical storage unit into a solid-state storage medium when the data in the data flushing unit corresponding to the logical storage unit reaches a capacity threshold; Among them, the corresponding relationship between the logical storage unit and the data disk flushing unit is predefined.

8. A data reading device, characterized in that: include: A data parsing module, configured to, upon receiving a data read request including a physical block address of data to be read, determine whether the data to be read is data in a continuous address space data group based on the physical block address of the data to be read; A data search module, configured to search for whether the continuous address space data group exists in the cache space in the read cache state when it is determined that the data to be read is data in the continuous address space data group; A data extraction module, configured to extract the to-be-read data from the cache space when it is determined that the continuous address space data group exists in the cache space; The data to be read and other data in the continuous address space data group where the data to be read is located are stored based on the data storage method according to claim 3.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the processor implements the data storage method according to any one of claims 1 to 4, and / or the data reading method according to claim 5 or 6.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the computer program implements the data storage method according to any one of claims 1 to 4 and / or the data reading method according to claim 5 or 6.

Images