Storage write amplification (WA) evaluation method and device, electronic equipment and medium

Abstract

The embodiment of the invention provides a storage write amplification WA evaluation method and device, electronic equipment and a medium. The virtual storage space is constructed for the target service, each write request is simulated and executed in the virtual storage space, and when it is simulated that the virtual storage space at least accommodating data written by the current write request exists, the storage state of each virtual storage position corresponding to the virtual storage space changes; therefore, the mode of determining the WA value of the target business when the return visit is ended does not need to really write the data of the write request into a real physical storage medium to obtain a WA value result, dependence on hardware is removed, hardware cost is saved, running time of real writing of the physical storage medium is saved, and the WA value is rapidly obtained.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a storage write amplification (WA) evaluation method, device, electronic device, and medium. Background Art

[0002] At present, in actual applications, write amplification (WA) exists in storage systems. The so-called write amplification refers to the situation where the amount of data written to the storage medium exceeds the amount of data written by the business logic. Write amplification will cause the storage medium to wear out and affect the life of the storage medium. Reasonable storage methods can effectively reduce the write amplification of the storage system when storing target business data. Accurate and fast evaluation of write amplification is crucial to determining the most suitable storage algorithm.

[0003] Currently, the write amplification (WA) value is evaluated by deploying and running the storage algorithm in the storage system to store the target business data. This WA evaluation method has the problems of long cycle and high hardware cost. Summary of the invention

[0004] In view of this, embodiments of the present application provide a storage write amplification evaluation method, device, electronic device, and medium to achieve accurate and rapid evaluation of a WA value.

[0005] The present application provides a storage write amplification evaluation method, the method comprising:

[0006] Parsing the read and write I / O request file of the target business to obtain each I / O request in the I / O request file; the I / O request includes a read request and a write request;

[0007] Obtain a write request from each obtained I / O request, traverse the obtained write requests in order, and use the currently traversed write request as the current write request;

[0008] Replay the current write request, determine whether there is a virtual storage space in the virtual storage device that can accommodate at least the data written by the current write request, and if so, update the storage state of the virtual storage location corresponding to the virtual storage space in the virtual storage device to a to-be-occupied state, the storage state is used to indicate to-be-occupied, and when it is determined that the data written by the current write request causes the existence of invalid data, update the storage state of the virtual storage location corresponding to the invalid data in the virtual storage device to a failed state, the failed state is used to indicate to-be-erased; and traverse the next write request of the current write request from all other write requests that have not been traversed, and return to replay the current write request; if not, output an alarm indication, and end the replay;

[0009] After the playback is finished, the WA value is determined based on the number of virtual storage locations currently in a to-be-occupied state and the number of logical storage locations corresponding to each write request in the I / O request file.

[0010] The embodiment of the present application also provides a storage write amplification evaluation device, the device comprising:

[0011] A parsing module, used for parsing the I / O request file of the target business to obtain each I / O request in the I / O request file;

[0012] An evaluation module, used for sequentially traversing each write request from each obtained I / O request, and taking the traversed write request as the current write request;

[0013] Replay the current write request, determine whether there is a virtual storage space in the virtual storage device that can accommodate at least the data written by the current write request, and if so, update the storage state of the virtual storage location corresponding to the virtual storage space in the virtual storage device to a to-be-occupied state, the storage state is used to indicate to-be-occupied, and when it is determined that the data written by the current write request causes the existence of invalid data, update the storage state of the virtual storage location corresponding to the invalid data in the virtual storage device to a failed state, the failed state is used to indicate to-be-erased; and traverse the next write request of the current write request from all other write requests that have not been traversed, and return to replay the current write request; if not, output an alarm indication, and end the replay;

[0014] The determination module is used to determine the WA value based on the number of virtual storage locations currently in a to-be-occupied state and the number of logical storage locations corresponding to each write request in the I / O request file after the playback ends.

[0015] An embodiment of the present application also provides an electronic device, comprising: a processor and a computer-readable storage medium for storing computer program instructions, wherein the computer program instructions, when executed by the computer-readable storage medium, enable the processor to execute the steps of the above method.

[0016] An embodiment of the present application also provides a machine-readable storage medium, which stores computer program instructions. When the computer program instructions are executed, the steps of the above method can be implemented.

[0017] It can be seen from the above technical solution that in this embodiment, the I / O request file of the target business is parsed, each write request is traversed in order from each obtained I / O request, the traversed write request is used as the current write request, the current write request is replayed, and it is determined whether there is a virtual storage space in the virtual storage device that can accommodate at least the data written by the current write request. If so, the storage state of the virtual storage position corresponding to the virtual storage space in the virtual storage device is updated to a to-be-occupied state, and the storage state is used to indicate to-be-occupied, and when determining the invalid data caused by the data written by the current write request, the storage state of the virtual storage position corresponding to the invalid data in the virtual storage device is updated to a failed state, and the failed state is used to indicate to-be-erased; and the next write request of the current write request is traversed from all other write requests that have not been traversed, and the current write request is returned for replay; if not, an alarm indication is output, and the replay is ended. After the replay is ended, the WA value is determined based on the number of virtual storage positions currently in the to-be-occupied state and the number of logical storage positions corresponding to each write request in the I / O request file.

[0018] This method constructs a virtual storage space for the target business, simulates the execution of each write request in the virtual storage space, and simulates the storage state changes of each virtual storage location corresponding to the virtual storage space when there is a virtual storage space that can accommodate at least the data written by the current write request, so as to determine the WA value of the target business when the return visit ends. This method does not require the data of the write request to be actually written to the real physical storage medium to obtain the WA value, removes the dependence on hardware, saves hardware costs, and saves the running time of the real write of the physical storage medium, so as to achieve rapid acquisition of the WA value. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A schematic diagram of a process for a method provided in an embodiment of the present application;

[0020] Figure 2 A schematic diagram of storage status information of each virtual storage location in a virtual storage device provided in an embodiment of the present application;

[0021] Figure 3 Another schematic diagram of a method according to an embodiment of the present invention;

[0022] Figure 4 A schematic diagram of the structure of the device provided in the embodiment of the present application;

[0023] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0024] In order to enable those skilled in the art to better understand the technical solutions provided by the embodiments of the present application and to make the above-mentioned purposes, features and advantages of the embodiments of the present application more obvious and understandable, the technical solutions in the embodiments of the present application are further described in detail below in conjunction with the accompanying drawings.

[0025] To facilitate understanding of this solution, the existing technical problems are described in detail before describing this solution:

[0026] In storage systems, the sequential write performance of common physical storage media such as Solid State Disks (SSDs) is better than the random write performance. Redirect On Write (ROW) is a key technology to achieve the conversion of random writes to sequential writes. Therefore, using ROW can improve the storage performance of storage systems.

[0027] The SSD in the storage system is a traditional block device with a minimum storage unit. For very small data, it may be necessary to fill it when writing. For example, if the minimum storage data is 8K, but the written data is only 2K, it is necessary to fill 2K with zeros to supplement the 8K storage, which will cause the storage space rate to decrease. In order to avoid the decrease in storage space utilization, IO aggregation is required. For example, if each IO writes 2K, 4 IOs are aggregated and written together. In other words, IO aggregation can improve the space utilization of the storage system.

[0028] As can be seen from the above description, in order to improve the storage performance and storage space utilization of the storage system, ROW and I / O aggregation is used. However, ROW and I / O aggregation will cause write amplification.

[0029] Specifically, redirection on write, when data modification is required, instead of directly modifying the original data block, write the updated data to a new location and update the related metadata to point to the new location. This is limited by the writing method and the fact that the minimum writing unit of the storage medium SSD is a storage page, while the minimum erasing unit is a storage block. This results in the need to read out all the data in the old data block when the user wants to update a small part of the data, merge the valid data in the old block with the new data, and then write all back to a new blank block, erasing the data in the old block to free up space. This will lead to write amplification. Here, reading out all the data in the old data block, merging the valid data in the old block with the new data, and then writing all back to a new blank block, erasing the data in the old block to free up space is called garbage collection (GC).

[0030] For example, if you want to modify a 4KB file in the SSD, but the SSD controller cannot update only the 4KB, it needs to first read the entire 512KB block containing the 4KB, and write the original 512KB minus the valid data excluding the 4KB change plus the new 4KB data into a new blank block, and finally erase the 512KB block that has been read out. In this way, in order to modify the 4KB data, the SSD is equivalent to erasing and writing 512KB data, so the write amplification reaches 128 times.

[0031] I / O aggregation is used to aggregate multiple write instructions. When a write instruction needs to be modified (overwrite), the overwritten data is very small, but all the data in the old data block formed by the aggregation needs to be read out, and the valid data in the old block is merged with the new data, and then all is written back to a new blank block, and the data in the old block is erased to free up space, which will still cause write amplification.

[0032] In summary, write amplification will exist in the storage system, which will not only cause the storage medium to wear out, but also affect the life of the storage medium.

[0033] Different businesses and storage systems vary greatly. If ROW and GC can be configured with parameters based on industry data characteristics and access patterns (that is, the storage algorithm used to write business data to the storage medium can be selected based on industry data characteristics and access patterns), it can minimize write amplification while taking into account performance. In other words, a reasonable storage algorithm can effectively reduce the write amplification of the storage system when storing target business data. Based on this, accurately and quickly evaluating the write amplification of the test program in the storage system when storing the target business is crucial to determining the most suitable storage algorithm.

[0034] Currently, the write method results are evaluated by deploying and running the storage algorithm in the storage system to store the data of the target business. However, this evaluation method requires the storage algorithm to be deployed in the storage system and run to actually write the data to the SSD. The physical storage media and server installation and deployment are complex, which makes the evaluation cycle very long and the hardware cost high. This cannot achieve accurate and rapid evaluation of the write amplification results of the storage algorithm for the target business.

[0035] Based on this, in order to solve the above problems, the embodiments of the present application provide a storage write amplification evaluation method, device, electronic device and medium to accurately and quickly evaluate the write amplification results of the storage algorithm for the target business.

[0036] The method provided in the embodiment of the present application is described below:

[0037] Please refer to Figure 1 , Figure 1 This is a flow chart of the method provided in the embodiment of the present application. Figure 1 As shown, the process may include the following steps:

[0038] S101, parsing a read and write I / O request file of a target service to obtain each I / O request in the I / O request file; the I / O request includes a read request and a write request.

[0039] Parse the I / O request file of the target business to obtain each I / O request in the I / O request file.

[0040] In this embodiment, the I / O request file is an existing Block I / O Trace file, and the obtained block input and output trace Block I / O Trace file of the target business is parsed to obtain each I / O request recorded in the file; the Block I / O Trace file of any business is used to record and store each I / O request requested by the business within a specified historical time period.

[0041] In this embodiment, the Block I / O Trace file can only record each I / O request from the business level (ie, the logical level), and cannot record the actual storage address in the storage medium corresponding to each I / O request.

[0042] As an embodiment, the Block I / O Trace file may be obtained from a trace storage module of a service end of other running storage systems storing the target service.

[0043] S102, obtaining a write request from each obtained I / O request, traversing the obtained write requests in order, and taking the currently traversed write request as the current write request.

[0044] In this embodiment, the Block I / O Trace file may be used to record the order of each write request for traversal.

[0045] S103, replay the current write request, determine whether there is virtual storage space in the virtual storage device that can accommodate at least the data written by the current write request, if so, update the storage status of the virtual storage location corresponding to the virtual storage space in the virtual storage device to a to-be-occupied state, the storage status is used to indicate to-be-occupied, and when it is determined that the data written by the current write request causes invalid data, update the storage status of the virtual storage location corresponding to the invalid data in the virtual storage device to a failed state, the failed state is used to indicate to-be-erased; and traverse the next write request of the current write request from all other write requests that have not been traversed, and return to replay the current write request; if not, output an alarm indication and end the replay.

[0046] The specific implementation of the above step S103 will be described in detail later and will not be repeated here.

[0047] S104, after the playback is finished, the WA value is determined based on the number of virtual storage locations currently in a to-be-occupied state and the number of logical storage locations corresponding to each write request in the I / O request file.

[0048] As an embodiment, the method further includes the following steps: during playback of each write request, when the number of virtual storage locations in a failed state meets the set erasure condition, the failed data in a failed state is erased and the valid data in the same virtual storage block as the failed data in a to-be-occupied state is migrated (this process is called garbage collection GC).

[0049] The storage state of the virtual storage location corresponding to the erased invalid data in the virtual storage device is changed from an invalid state to an idle state; the storage state of the virtual storage location before migration corresponding to the migrated valid data is changed from a to-be-occupied state to an idle state, and the storage state of the virtual storage location after migration corresponding to the migrated valid data is changed from an idle state to a to-be-occupied state.

[0050] When the number of virtual storage locations in a failed state meets the set migration stop condition, the migration is stopped.

[0051] Accordingly, the implementation of the above step S104 is changed to: based on the number of virtual storage locations currently in a to-be-occupied state after the playback ends, the number of virtual storage locations occupied by the migrated valid data, and the number of logical storage locations corresponding to each I / O request. This specific implementation of step S104 will be described in detail in the following text in the form of a specific embodiment, and will not be repeated here.

[0052] So far, completed Figure 1 The process shown.

[0053] pass Figure 1As can be seen from the process shown, in this embodiment, the I / O request file of the target business is parsed, each write request is traversed in order from each obtained I / O request, the traversed write request is used as the current write request, the current write request is replayed, and it is determined whether there is a virtual storage space in the virtual storage device that can accommodate at least the data written by the current write request. If so, the storage state of the virtual storage position corresponding to the virtual storage space in the virtual storage device is updated to a to-be-occupied state, and the storage state is used to indicate to-be-occupied. When determining the invalid data caused by the data written by the current write request, the storage state of the virtual storage position corresponding to the invalid data in the virtual storage device is updated to a failed state, and the failed state is used to indicate to-be-erased; and the next write request of the current write request is traversed from all other write requests that have not been traversed, and the current write request is returned for replay; if not, an alarm indication is output, and the replay is ended. After the replay is ended, the WA value is determined based on the number of virtual storage positions currently in the to-be-occupied state and the number of logical storage positions corresponding to each write request in the I / O request file.

[0054] This method constructs a virtual storage space for the target business, simulates the execution of each write request in the virtual storage space, and simulates the storage state changes of each virtual storage location corresponding to the virtual storage space when there is a virtual storage space that can accommodate at least the data written by the current write request, so as to determine the WA value of the target business when the return visit ends. This method does not require the data of the write request to be actually written to the real physical storage medium to obtain the WA value, removes the dependence on hardware, saves hardware costs, and saves the running time of the real write of the physical storage medium, so as to achieve rapid acquisition of the WA value.

[0055] The specific implementation of the above step S103 is described in detail below:

[0056] There are many specific implementation methods for determining whether there is a virtual storage space in the virtual storage device that can accommodate at least the data written by the current write request in the above step S103. For example, as an embodiment, the storage status of each virtual storage location in the virtual storage device and the data written by the current write request are input into an external storage algorithm. If the storage algorithm outputs a virtual storage space that can accommodate the data written by the current write request, it is determined that there is a virtual storage space that can accommodate at least the data written by the current write request. If the storage algorithm outputs a virtual storage location that can accommodate the data written by the current write request, it is determined that there is no virtual storage space that can accommodate at least the data written by the current write request.

[0057] Specifically, in a known virtual storage device, the virtual storage space is composed of multiple virtual storage blocks and multiple storage pages on each block. To determine whether there is a virtual storage space in the virtual storage device that can accommodate at least the data written by the current write request, the purpose is to search the virtual storage space for the page (i.e., the physical block address PBA) that accommodates the data written by the current write request. When replaying the first write request, the configuration information that the virtual storage space is composed of multiple virtual storage blocks and multiple storage pages on each block, and the current write request carrying the logical block address LBA are sent to the external storage algorithm to determine whether the external storage algorithm calculates the PBA corresponding to the LBA. If calculated, the next write request of the current write request is traversed. If not calculated, an alarm indication is output and the playback ends.

[0058] Correspondingly, in the above step S103, when it is determined that there is virtual storage space in the virtual storage device to accommodate the data written by the current write request, the specific implementation method of updating the storage status of the virtual storage location corresponding to the virtual storage space in the virtual storage device to a to-be-occupied state can be: based on the calculation of the PBA corresponding to the LBA, the bitmap mark corresponding to the physical block address PBA in the existing first bitmap is updated from the initial setting mark to the first mark, for example, from 0 to 1.

[0059] Accordingly, the specific implementation method for determining whether the data written by the current write request leads to invalid data can be as follows: if the correspondence between the historical LBA and PBA recorded locally is used to find out whether there is a historical LBA that is the same as the LBA carried in the current write request, and the historical PBA corresponding to the historical LBA is different from the current PBA, if a historical LBA that is the same as the LBA is found, and the historical PBA corresponding to the historical LBA is different from the current PBA, it is determined that there is invalid data, and the data at the historical PBA is determined as invalid data, and the correspondence between the LBA and the historical PBA is changed to the correspondence between the LBA and the current PBA. If no data is found, it is determined that there is no invalid data, and the correspondence between the LBA and the current PBA is directly recorded.

[0060] Based on the above description, the specific implementation method of updating the storage status of the virtual storage location corresponding to the failed data in the virtual storage device to a failed status in the above step S103 may be: based on the PBA corresponding to the failed data, the bitmap mark corresponding to the physical block address PBA in the existing second bitmap is updated from the initial setting mark to the second mark, for example, from 0 to 1.

[0061] The specific implementation of the above garbage collection process steps is described in detail below.

[0062] The specific implementation method of changing the storage state of the virtual storage location corresponding to the erased failed data in the virtual storage device from a failed state to an idle state as mentioned above may be: based on the physical block address PBA corresponding to the erased failed data, the bitmap mark corresponding to the physical block address PBA in the existing second bitmap bitmap is updated from the second mark to the initial setting mark, for example, from 1 to 0.

[0063] The specific implementation method of changing the storage state of the virtual storage location before migration corresponding to the migrated valid data mentioned above from a to-be-occupied state to an idle state may be: based on the physical block address PBA corresponding to the virtual storage location before migration corresponding to the migrated valid data, the bitmap mark corresponding to the physical block address PBA in the existing first bitmap is updated from the second mark to the initial setting mark, for example, from 1 to 0.

[0064] The specific implementation method of changing the storage state of the migrated virtual storage location corresponding to the migrated valid data mentioned above from the idle state to the to-be-occupied state may be:

[0065] Based on the physical block address PBA corresponding to the migrated virtual storage location corresponding to the migrated valid data, the bitmap mark corresponding to the physical block address PBA in the existing first bitmap is updated from the initial setting mark to the second mark, for example, from 0 to 1.

[0066] The above-mentioned determination of whether the number of virtual storage locations in a failed state meets the set erasure condition can be achieved in the following manner. For example, as an embodiment, when the ratio of the number of pages in a failed state in the virtual storage device to all pages in the virtual storage device is greater than a set recovery ratio threshold, it is determined that the set erasure condition is met.

[0067] The total number of failed pages in the virtual storage device can be calculated in the following manner: after each write request is played back, the page corresponding to the historical PBA corresponding to the LBA carried in the write request is determined to be failed data, the number of failed pages in a block is counted, and the corresponding count is increased to update the total number of failed pages in the block. The total number of failed pages in all blocks is counted to update the total number of failed pages in the virtual storage device.

[0068] Perform garbage collection on a block, count the number of invalid pages deleted from the block during the garbage collection process, and reduce the corresponding count to update the total number of invalid pages in the block. Count the number of invalid pages deleted from all blocks, and reduce the corresponding count to update the total number of invalid pages in the virtual storage device.

[0069] The specific implementation method of erasing the invalid data in the invalid state and migrating the valid data in the same virtual storage block as the invalid data in the to-be-occupied state can be as follows: inputting the storage state of each virtual storage location in the virtual storage device into an external recycling algorithm to obtain a recycling strategy, which is used to indicate which block to select, deleting the invalid pages in the block, and migrating the valid pages.

[0070] The above garbage collection process is described in detail above.

[0071] The specific implementation of the above step S104 is described in detail below.

[0072] The specific implementation method mentioned above based on the number of virtual storage locations currently in the to-be-occupied state after the playback ends, the number of virtual storage locations occupied by migrated valid data, and the number of logical storage locations corresponding to each I / O request can be: the ratio of the sum of the total number of pages currently in the to-be-occupied state and the total number of migrated pages after the playback ends to the number of pages corresponding to each I / O request at the logical level is the above-mentioned WA value.

[0073] Exemplarily, the total number of pages currently in the to-be-occupied state can be calculated in the following way: after replaying a write request, the first bitmap is updated once, a block is garbage collected, and the first bitmap is also updated once. Each time the first bitmap is updated, the number of newly added pages in the to-be-occupied state in the block corresponding to the first bitmap is counted by comparing the bitmaps before and after the update, or the number of pages in the to-be-occupied state that are reduced in the block due to the migration of valid data, so as to increase or decrease the count to update the total number of pages in the to-be-occupied state in the block.

[0074] The total number of migrated pages can be calculated in the following way: after obtaining the migration strategy, count the number of valid pages migrated in each block, increase the corresponding count and update the total number of migrated pages in the virtual storage device.

[0075] The specific implementation of the above step S104 is described in detail above.

[0076] In order to explain this scheme in more detail, Figure 2 and Figure 3 The method of this application is described in more detail:

[0077] The process is as follows:

[0078] 1. Parse the device input parameters.

[0079] In this step, the above parameters at least include that the virtual storage space is composed of a plurality of virtual storage blocks, and a plurality of storage pages on each block.

[0080] 2. Parse the I / O request file Block I / O Trace file of the target business to obtain each I / O request from it.

[0081] 3. Traverse each write request in order from the obtained I / O requests, and use the traversed write request as the current write request.

[0082] 4. Call the external storage algorithm (i.e., the storage algorithm to be tested) to determine whether to find the PBA corresponding to the LBA carried in the current write request. If found, execute the subordinate step 5 and continue to send the next write request. Otherwise, end the playback.

[0083] 5. Update the storage status information of each virtual storage location in the virtual storage device based on the found PBA.

[0084] For more information about the storage status of each virtual storage location in a specific virtual storage device, see Figure 2 , Figure 2 The specific updating method of each information in the above steps S103 and S104 is referred to, and will not be repeated here.

[0085] 6. After the return visit, obtain statistical information, and take the ratio of the sum of the total number of pages in the to-be-occupied state and the total number of migrated pages in the statistical information to the number of pages corresponding to each I / O request at the logical level as the above WA value.

[0086] The method provided in the embodiment of the present application is described above. The device provided in the embodiment of the present application is described below:

[0087] See also Figure 4 , Figure 4 This is a diagram of the device structure provided in the embodiment of the present application. Figure 4 As shown, the device includes: an analysis module 401, an evaluation module 402 and a determination module 403.

[0088] A parsing module 401 is used to parse the I / O request file of the target business to obtain each I / O request in the I / O request file;

[0089] An evaluation module 402 is used to sequentially traverse each write request from each obtained I / O request, and use the traversed write request as the current write request;

[0090] Replay the current write request, determine whether there is a virtual storage space in the virtual storage device that can accommodate at least the data written by the current write request, and if so, update the storage state of the virtual storage location corresponding to the virtual storage space in the virtual storage device to a to-be-occupied state, the storage state is used to indicate to-be-occupied, and when it is determined that the data written by the current write request causes the existence of invalid data, update the storage state of the virtual storage location corresponding to the invalid data in the virtual storage device to a failed state, the failed state is used to indicate to-be-erased; and traverse the next write request of the current write request from all other write requests that have not been traversed, and return to replay the current write request; if not, output an alarm indication, and end the replay;

[0091] The determination module 403 is used to determine the WA value based on the number of virtual storage locations currently in a to-be-occupied state and the number of logical storage locations corresponding to each write request in the I / O request file after the playback ends.

[0092] As an embodiment, the evaluation module is further used for:

[0093] In the process of replaying each write request, when the number of virtual storage locations in a failed state meets the set erasure condition, the failed data in a failed state is erased and the valid data in the same virtual storage block as the failed data in a to-be-occupied state is migrated;

[0094] The storage state of the virtual storage location corresponding to the erased invalid data in the virtual storage device is changed from the invalid state to the idle state; the storage state of the virtual storage location before the migration corresponding to the migrated valid data is changed from the to-be-occupied state to the idle state, and the storage state of the virtual storage location after the migration corresponding to the migrated valid data is changed from the idle state to the to-be-occupied state;

[0095] When the number of virtual storage locations in the invalid state meets the set migration stop condition, the migration is stopped;

[0096] The WA value is determined based on the number of virtual storage locations currently in a to-be-occupied state after the playback ends, the number of virtual storage locations occupied by migrated valid data, and the number of logical storage locations corresponding to each I / O request.

[0097] As an embodiment, updating the storage state of the virtual storage location corresponding to the virtual storage space in the virtual storage device to a to-be-occupied state includes:

[0098] Based on a physical block address PBA in a virtual physical storage space corresponding to the virtual storage location, updating a bitmap mark corresponding to the physical block address PBA in an existing first bitmap from an initial setting mark to a first mark;

[0099] Updating the storage state of the virtual storage location corresponding to the failed data in the virtual storage device to a failed state includes:

[0100] Based on the physical block address PBA corresponding to the virtual storage location corresponding to the failed data, the bitmap mark corresponding to the physical block address PBA in the existing second bitmap is updated from the initial setting mark to the second mark.

[0101] As an embodiment, changing the storage state of the virtual storage location corresponding to the erased invalid data in the virtual storage device from the invalid state to the idle state includes:

[0102] Based on the physical block address PBA corresponding to the virtual storage location corresponding to the erased failed data, the bitmap mark corresponding to the physical block address PBA in the existing second bitmap is updated from the second mark to the initial setting mark;

[0103] Changing the storage state of the virtual storage location before migration corresponding to the migrated valid data from a to-be-occupied state to an idle state includes:

[0104] Based on the physical block address PBA corresponding to the virtual storage location before the migration corresponding to the migrated valid data, the bitmap mark corresponding to the physical block address PBA in the existing first bitmap is updated from the second mark to the initial setting mark;

[0105] Changing the storage state of the migrated virtual storage location corresponding to the migrated valid data from an idle state to a to-be-occupied state includes:

[0106] Based on the physical block address PBA corresponding to the migrated virtual storage location corresponding to the migrated valid data, the bitmap mark corresponding to the physical block address PBA in the existing first bitmap is updated from the initial setting mark to the second mark.

[0107] As an embodiment, determining whether there is a virtual storage space in the virtual storage device that can accommodate at least the data written by the current write request includes:

[0108] The storage status of each virtual storage location in the virtual storage device and the data written by the current write request are input into the external storage algorithm. If the storage algorithm outputs a virtual storage location that accommodates the data written by the current write request, it is determined that there is a virtual storage space that can accommodate at least the data written by the current write request. If the storage algorithm outputs a virtual storage location that accommodates the data written by the current write request, it is determined that there is no virtual storage space that can accommodate at least the data written by the current write request.

[0109] As an embodiment, any write request carries a logical block address LBA; the device further includes:

[0110] The storage module is used to search for a record of whether there is a historical LBA that is the same as the LBA carried in the current write request in the correspondence relationship between the LBA and the PBA recorded locally, and the historical PBA corresponding to the historical LBA is different from the current PBA;

[0111] If found, the correspondence between LBA and historical PBA is changed to the correspondence between LBA and current PBA;

[0112] If no record is found, the corresponding relationship between LBA and current PBA is recorded;

[0113] When a record is found, it is determined that the data written by the current write request results in the existence of invalid data, and the data at the historical PBA is invalid data.

[0114] So far, completed Figure 4 Structural description of the device shown.

[0115] See also Figure 5 , Figure 5 This is a structural diagram of an electronic device provided in an embodiment of the present application. Figure 5 As shown, the hardware structure may include: a processor and a machine-readable storage medium, the machine-readable storage medium storing machine-executable instructions that can be executed by the processor; the processor is used to execute the machine-executable instructions to implement the method disclosed in the above example of this application.

[0116] Based on the same application concept as the above method, an embodiment of the present application also provides a machine-readable storage medium, on which a number of computer instructions are stored. When the computer instructions are executed by a processor, the method disclosed in the above example of the present application can be implemented.

[0117] Exemplarily, the above-mentioned machine-readable storage medium can be any electronic, magnetic, optical or other physical storage device that can contain or store information, such as executable instructions, data, etc. For example, the machine-readable storage medium can be: RAM (Radom Access Memory), volatile memory, non-volatile memory, flash memory, storage drive (such as hard disk drive), solid state drive, any type of storage disk (such as CD, DVD, etc.), or similar storage medium, or a combination thereof.

[0118] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A storage write amplification (WA) evaluation method, characterized in that: The method comprises: Parsing the read and write I / O request file of the target business to obtain each I / O request in the I / O request file; the I / O request includes a read request and a write request; Obtain a write request from each obtained I / O request, traverse the obtained write requests in order, and use the currently traversed write request as the current write request; Replay the current write request, determine whether there is a virtual storage space in the virtual storage device that can accommodate at least the data written by the current write request, and if so, update the storage state of the virtual storage location corresponding to the virtual storage space in the virtual storage device to a to-be-occupied state, wherein the storage state is used to indicate to-be-occupied, and when it is determined that the data written by the current write request causes the existence of invalid data, update the storage state of the virtual storage location corresponding to the invalid data in the virtual storage device to a invalid state, wherein the invalid state is used to indicate to-be-erased; and traverse the next write request of the current write request from all other write requests that have not been traversed, and return to replay the current write request; if not, output an alarm indication, and end the replay; After the playback is finished, the WA value is determined based on the number of virtual storage locations currently in a to-be-occupied state and the number of logical storage locations corresponding to each write request in the I / O request file.

2. The method according to claim 1, characterized in that The method further comprises: In the process of replaying each write request, when the number of virtual storage locations in a failed state meets the set erasure condition, the failed data in a failed state is erased and the valid data in the same virtual storage block as the failed data in a to-be-occupied state is migrated; The storage state of the virtual storage location corresponding to the erased invalid data in the virtual storage device is changed from the invalid state to the idle state; the storage state of the virtual storage location before migration corresponding to the migrated valid data is changed from the to-be-occupied state to the idle state, and the storage state of the virtual storage location after migration corresponding to the migrated valid data is changed from the idle state to the to-be-occupied state; When the number of virtual storage locations in a failed state meets a set migration stop condition, the migration is stopped; The WA value is determined based on the number of virtual storage locations currently in a to-be-occupied state after playback ends, the number of virtual storage locations occupied by migrated valid data, and the number of logical storage locations corresponding to each I / O request.

3. The method according to claim 1, characterized in that: The updating of the storage state of the virtual storage location corresponding to the virtual storage space in the virtual storage device to a to-be-occupied state comprises: Based on a physical block address PBA in a virtual physical storage space corresponding to the virtual storage location, updating a bitmap mark corresponding to the physical block address PBA in an existing first bitmap from an initial setting mark to a first mark; The updating of the storage state of the virtual storage location corresponding to the invalid data in the virtual storage device to the invalid state comprises: Based on the physical block address PBA corresponding to the virtual storage location corresponding to the failed data, the bitmap mark corresponding to the physical block address PBA in the existing second bitmap is updated from the initial setting mark to the second mark.

4. The method according to claim 2 or 3, characterized in that: The step of changing the storage state of the virtual storage location corresponding to the erased invalid data in the virtual storage device from the invalid state to the idle state comprises: Based on the physical block address PBA corresponding to the virtual storage location corresponding to the erased failed data, the bitmap mark corresponding to the physical block address PBA in the existing second bitmap is updated from the second mark to the initial setting mark; The step of changing the storage state of the virtual storage location before migration corresponding to the migrated valid data from a to-be-occupied state to an idle state comprises: Based on the physical block address PBA corresponding to the virtual storage location before the migration corresponding to the migrated valid data, the bitmap mark corresponding to the physical block address PBA in the existing first bitmap is updated from the second mark to the initial setting mark; The step of changing the storage state of the migrated virtual storage location corresponding to the migrated valid data from an idle state to a to-be-occupied state comprises: Based on the physical block address PBA corresponding to the migrated virtual storage location corresponding to the migrated valid data, the bitmap mark corresponding to the physical block address PBA in the existing first bitmap is updated from the initial setting mark to the second mark.

5. The method according to claim 1, characterized in that The step of determining whether there is a virtual storage space in the virtual storage device that at least accommodates the data written by the current write request comprises: The storage status of each virtual storage location in the virtual storage device and the data written by the current write request are input into an external storage algorithm. If the storage algorithm outputs a virtual storage location that accommodates the data written by the current write request, it is determined that there is a virtual storage space that can accommodate at least the data written by the current write request. If the storage algorithm outputs a virtual storage location that accommodates the data written by the current write request, it is determined that there is no virtual storage space that can accommodate at least the data written by the current write request.

6. The method according to claim 1, characterized in that Any write request carries a logical block address LBA; After determining that there is a virtual storage space in the virtual storage device that at least accommodates the data written by the current write request, the method further includes: In the correspondence relationship between LBA and PBA recorded locally, find whether there is a record in which the same historical LBA as the LBA carried in the current write request exists, and the historical PBA corresponding to the historical LBA is different from the current PBA; If found, the corresponding relationship between the LBA and the historical PBA is changed to the corresponding relationship between the LBA and the current PBA; If the record is not found, the corresponding relationship between the LBA and the current PBA is recorded; When the record is found, it is determined that the data written by the current write request results in the existence of invalid data, and the data at the historical PBA is the invalid data.

7. A storage write amplification WA evaluation device, characterized in that: The device comprises: A parsing module, used for parsing a read and write I / O request file of a target business to obtain each I / O request in the I / O request file; the I / O request includes a read request and a write request; An evaluation module, used for obtaining a write request from each obtained I / O request, traversing the obtained write requests in order, and taking the currently traversed write request as the current write request; Replay the current write request, determine whether there is a virtual storage space in the virtual storage device that can accommodate at least the data written by the current write request, and if so, update the storage state of the virtual storage location corresponding to the virtual storage space in the virtual storage device to a to-be-occupied state, wherein the storage state is used to indicate to-be-occupied, and when it is determined that the data written by the current write request causes the existence of invalid data, update the storage state of the virtual storage location corresponding to the invalid data in the virtual storage device to a invalid state, wherein the invalid state is used to indicate to-be-erased; and traverse the next write request of the current write request from all other write requests that have not been traversed, and return to replay the current write request; if not, output an alarm indication, and end the replay; The determination module is used to determine the WA value based on the number of virtual storage locations currently in a to-be-occupied state and the number of logical storage locations corresponding to each write request in the I / O request file after the playback is finished.

8. The device according to claim 7, characterized in that The evaluation module is also used to: In the process of replaying each write request, when the number of virtual storage locations in a failed state meets the set erasure condition, the failed data in a failed state is erased and the valid data in the same virtual storage block as the failed data in a to-be-occupied state is migrated; Changing the storage state of the virtual storage location corresponding to the erased invalid data in the virtual storage device from the invalid state to the idle state; Changing the storage state of the virtual storage location before migration corresponding to the migrated valid data from a waiting-to-be-occupied state to an idle state, and changing the storage state of the virtual storage location after migration corresponding to the migrated valid data from an idle state to a waiting-to-be-occupied state; When the number of virtual storage locations in a failed state meets a set migration stop condition, the migration is stopped; The WA value is determined based on the number of virtual storage locations currently in a to-be-occupied state after the playback ends, the number of virtual storage locations occupied by the migrated valid data, and the number of logical storage locations corresponding to each I / O request; and / or, The updating of the storage state of the virtual storage location corresponding to the virtual storage space in the virtual storage device to a to-be-occupied state comprises: Based on a physical block address PBA in a virtual physical storage space corresponding to the virtual storage location, updating a bitmap mark corresponding to the physical block address PBA in an existing first bitmap from an initial setting mark to a first mark; The updating of the storage state of the virtual storage location corresponding to the invalid data in the virtual storage device to the invalid state comprises: Based on the physical block address PBA corresponding to the virtual storage location corresponding to the failed data, updating the bitmap mark corresponding to the physical block address PBA in the existing second bitmap from the initial setting mark to the second mark; The updating of the storage state of the virtual storage location corresponding to the invalid data in the virtual storage device to the invalid state comprises: Based on the physical block address PBA corresponding to the virtual storage location corresponding to the failed data, updating the bitmap mark corresponding to the physical block address PBA in the existing second bitmap from the initial setting mark to the second mark; and / or, The step of changing the storage state of the virtual storage location corresponding to the erased invalid data in the virtual storage device from the invalid state to the idle state comprises: Based on the physical block address PBA corresponding to the virtual storage location corresponding to the erased failed data, the bitmap mark corresponding to the physical block address PBA in the existing second bitmap is updated from the second mark to the initial setting mark; The step of changing the storage state of the virtual storage location before migration corresponding to the migrated valid data from a to-be-occupied state to an idle state comprises: Based on the physical block address PBA corresponding to the virtual storage location before the migration corresponding to the migrated valid data, the bitmap mark corresponding to the physical block address PBA in the existing first bitmap is updated from the second mark to the initial setting mark; The step of changing the storage state of the migrated virtual storage location corresponding to the migrated valid data from an idle state to a to-be-occupied state comprises: Based on the physical block address PBA corresponding to the migrated virtual storage location corresponding to the migrated valid data, the bitmap mark corresponding to the physical block address PBA in the existing first bitmap is updated from the initial setting mark to the second mark; and / or, The step of determining whether there is a virtual storage space in the virtual storage device that at least accommodates the data written by the current write request comprises: Inputting the storage status of each virtual storage location in the virtual storage device and the data written by the current write request into an external storage algorithm, if the storage algorithm outputs the virtual storage location accommodating the data written by the current write request, it is determined that there is a virtual storage space that can accommodate at least the data written by the current write request, and if the storage algorithm outputs the virtual storage location accommodating the data written by the current write request, it is determined that there is no virtual storage space that can accommodate at least the data written by the current write request; and / or, Any write request carries a logical block address LBA; the device also includes: The storage module is used to search for a record of whether there is a historical LBA that is the same as the LBA carried in the current write request in the correspondence relationship between the LBA and the PBA recorded locally, and the historical PBA corresponding to the historical LBA is different from the current PBA; If found, the corresponding relationship between the LBA and the historical PBA is changed to the corresponding relationship between the LBA and the current PBA; If the record is not found, the corresponding relationship between the LBA and the current PBA is recorded; When the record is found, it is determined that the data written by the current write request results in the existence of invalid data, and the data at the historical PBA is the invalid data.

9. An electronic device, characterized in that: The electronic device includes: Processor; and A computer-readable storage medium, wherein computer program instructions are stored in the computer-readable storage medium, and when the computer program instructions are executed by the processor, the processor executes the steps of the method according to any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, which, when executed by a processor, enable the processor to perform the steps of any one of the methods of claims 1 to 6.

Images