Memory write amplification wa evaluation method and apparatus, electronic device, and medium

Abstract

Embodiments of the present application provide a storage write amplification (WA) evaluation method and device, electronic equipment and medium. The method comprises: constructing a virtual storage space for a target service; simulating execution of each write request in the virtual storage space; and determining a WA value of the target service when the end of the write request is revisited, by simulating a change in a storage state of each virtual storage location corresponding to the virtual storage space when there is at least a virtual storage space to accommodate data written by the current write request. The method does not need to actually write data of the write request to a real physical storage medium to obtain the WA value, removes the dependence on hardware, saves hardware costs and running time of real writing of the physical storage medium, and quickly obtains the WA value.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, and particularly relates to a storage write amplification (WA) evaluation method and device, an electronic device and a medium. BACKGROUND

[0002] At present, in actual application, write amplification (WA) exists in a storage system. The write amplification refers to a case that a data amount written by a storage medium exceeds a data amount written by a business logic. The write amplification can cause the storage medium to wear out and affect a service life of the storage medium. A reasonable storage method can effectively reduce the write amplification of the storage system when storing target business data. Accurate and rapid evaluation of the write amplification is crucial for determining a most matched storage algorithm.

[0003] At present, the write amplification (WA) value is evaluated by deploying and running the storage algorithm in the storage system to store data of the target business. This WA evaluation method has problems of a long cycle and high hardware cost. SUMMARY

[0004] Therefore, the present application provides a storage write amplification evaluation method and device, an electronic device and a medium to accurately and rapidly evaluate the WA value.

[0005] The present application provides a storage write amplification evaluation method, which comprises the following steps.

[0006] Analyzing a read-write I / O request file of a target business to obtain each I / O request in the I / O request file; the I / O request comprises a read request and a write request;

[0007] Obtaining the write request from each I / O request, sequentially traversing each obtained write request, and taking the currently traversed write request as a current write request;

[0008] Replaying the current write request to determine whether there is a virtual storage space in a virtual storage device that can accommodate data written by the current write request, if yes, updating a storage state of a virtual storage location corresponding to the virtual storage space in the virtual storage device to a to-be-occupied state, the storage state is used to indicate to-be-occupied, and when it is determined that the data written by the current write request causes invalid data to exist, updating a storage state of a virtual storage location corresponding to the invalid data in the virtual storage device to an invalid state, the invalid state is used to indicate to-be-erased; and traversing a next write request of the current write request from all other write requests that have not been traversed, returning to replay the current write request; if no, outputting an alarm indication and ending the replay;

[0009] After ending the playback, the WA value is determined based on the number of virtual storage locations currently in the to-be-occupied state and based on the number of logical storage locations corresponding to each write request in the I / O request file.

[0010] The embodiment of the present application further provides a storage write amplification evaluation device, which comprises:

[0011] The analysis module is configured to analyze the I / O request file of the target service to obtain each I / O request in the I / O request file.

[0012] The evaluation module is configured to sequentially traverse each write request from the obtained I / O requests, and take the traversed write request as a current write request.

[0013] The playback of the current write request is performed, and it is determined whether there is virtual storage space in the virtual storage device that can accommodate at least the data written by the current write request, if yes, the storage state of the virtual storage location corresponding to the virtual storage space in the virtual storage device is updated to the to-be-occupied state, the storage state is used to indicate to-be-occupied, and when it is determined that the data written by the current write request causes the existence of invalid data, the storage state of the virtual storage location corresponding to the invalid data in the virtual storage device is updated to an invalid state, the invalid state is used to indicate to-be-erased; and the next write request of the current write request is traversed from all other write requests that have not been traversed, and the playback of the current write request is returned; if no, an alarm indication is output, and the playback is ended.

[0014] The determination module is configured to determine the WA value based on the number of virtual storage locations currently in the to-be-occupied state and based on the number of logical storage locations corresponding to each write request in the I / O request file after ending the playback.

[0015] The embodiment of the present application further provides an electronic device, which comprises a processor and a computer readable storage medium for storing computer program instructions, and the computer program instructions enable the processor to perform the steps of the above method when the computer program instructions are run by the computer readable storage medium.

[0016] The embodiment of the present application further provides a machine readable storage medium, which stores computer program instructions, and the computer program instructions can implement the steps of the above method when the computer program instructions are executed.

[0017] It can be seen from the above technical solution that in the embodiment, the I / O request file of the target service is parsed, each write request is traversed in sequence from the obtained I / O requests, the traversed write request is taken as a current write request, the current write request is played back, it is determined whether there is a virtual storage space in the virtual storage device that can accommodate at least the data written by the current write request, if yes, the storage state of the virtual storage position corresponding to the virtual storage space in the virtual storage device is updated to a to-be-occupied state, the storage state is used to indicate to-be-occupied, and when invalid data caused by the data written by the current write request is determined, the storage state of the virtual storage position corresponding to the invalid data in the virtual storage device is updated to an invalid state, the invalid state is used to indicate to-be-erased; the next write request of the current write request is traversed from all other write requests that have not been traversed, and the playback of the current write request is returned; if no, an alarm indication is output, and the playback is ended, and after the playback is ended, the WA value is determined based on the number of virtual storage positions in the to-be-occupied state and based on the number of logical storage positions corresponding to each write request in the I / O request file.

[0018] This way of constructing a virtual storage space for a target service, simulating the execution of each write request in the virtual storage space, changing the storage state of each virtual storage position corresponding to the virtual storage space when there is a virtual storage space that can accommodate at least the data written by the current write request, and determining the WA value of the target service when the playback is ended, does not need to actually write the data of the write request to a real physical storage medium to obtain the WA value, removes the dependence on hardware, saves hardware costs, and saves the running time of real writing of the physical storage medium, so that the WA value is quickly obtained. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 A flowchart of a method provided by the embodiment of the application is provided.

[0020] Figure 2 A schematic diagram of storage state information of each virtual storage position in a virtual storage device provided by the embodiment of the application is provided.

[0021] Figure 3 Another flowchart of a method provided by the embodiment of the application is provided.

[0022] Figure 4 A structural schematic diagram of an apparatus provided by the embodiment of the application is provided.

[0023] Figure 5 A structural schematic diagram of an electronic device provided by the embodiment of the application is provided. DETAILED DESCRIPTION

[0024] In order to better understand the technical solutions provided by the embodiments of the present application by the skilled in the art, and to make the above-mentioned purposes, features and advantages of the embodiments of the present application more obvious and easy to understand, the technical solutions in the embodiments of the present application are further described in detail below with reference to the drawings.

[0025] In order to facilitate the understanding of the present application, the technical problems existing in the prior art are described in detail before the present application is described.

[0026] In the storage system, the sequential write performance of the common physical storage medium such as the solid state disk (SSD) is better than the random write performance, and the redirect on write (ROW) is a key technology for realizing the conversion of random write to sequential write, therefore, the use of ROW can improve the storage performance of the storage system.

[0027] The SSD in the storage system is a traditional block device, which has a minimum storage unit, and for very small data, it may be necessary to fill up when writing, for example, the minimum storage data is 8K, but the writing data is only 2K, and it is necessary to fill 2K to 8K storage by zero padding, which will cause the storage space rate to decrease, in order to avoid the decrease of the storage space utilization rate, it is necessary to perform IO aggregation, for example, each IO writes 2k, and 4 IOs are aggregated together for writing. That is, the IO aggregation can improve the space utilization rate of the storage system.

[0028] As can be seen from the above description, in order to improve the storage performance and storage space utilization rate of the storage system, ROW and I / O aggregation are used. However, ROW and I / O aggregation will cause write amplification.

[0029] Specifically, the redirect on write directly modifies the original data block when data modification is needed, and writes the updated data to a new location and updates the relevant metadata to point to the new location. Limited by this writing method and the fact that the minimum writing unit of the storage medium SSD is a storage page, while the minimum erasing unit is a storage block, when a user wants to update a small part of data, the entire data in the old data block needs to be read out, the valid data in the old block is merged with the new data, and then the entire data is written back to a new blank block, and the data in the old block is erased to release space. This will cause write amplification. Here, the way of reading out the entire data in the old data block, merging the valid data in the old block with the new data, and then writing back the entire data to a new blank block, and erasing the data in the old block to release space is called garbage collection (GC).

[0030] For example, if you want to modify a 4KB file in the SSD, but the SSD controller cannot update only the 4KB, so it needs to read the entire 512KB block containing the 4KB first, write the original 512KB minus the 4KB changed valid data plus the new 4KB data to a new blank block, and finally erase the read 512KB block. In this way, in order to change 4KB of data, the SSD has to erase and write 512KB of data, so the write amplification is as high as 128 times.

[0031] I / O aggregation is used to aggregate multiple write instructions. When a write instruction needs to be modified (when an overwrite write is performed), the data of the overwrite write is small, but the entire data in the aggregated old data block also needs to be read out, and the valid data in the old block and the new data are merged and then written back to a new blank block, and the data in the old block is erased to release space, which will also cause write amplification.

[0032] In summary, write amplification exists in the storage system, which will not only cause wear and tear of the storage medium, but also affect the service life of the storage medium.

[0033] Different businesses and storage systems differ greatly, and ROW and GC can be configured according to industry data characteristics and access patterns (that is, the storage algorithm for writing business data to the storage medium can be selected according to industry data characteristics and access patterns). In this way, write amplification can be reduced as much as possible while performance is taken into account. That is, a reasonable storage algorithm can effectively reduce the write amplification of the storage system when storing target business data. Therefore, accurately and quickly evaluating the write amplification of the test program when storing target business data in the storage system is crucial to determining the most suitable storage algorithm.

[0034] Currently, the write method result is evaluated by deploying and running the storage algorithm in the storage system to store the data of the target business. However, this evaluation method needs to arrange the storage algorithm in the storage system, and run the storage algorithm to actually write the data to the SSD. The physical storage medium and server installation and deployment are complex, the evaluation period is very long, and the hardware cost is high. Therefore, this method cannot accurately and quickly evaluate the write amplification result of the storage algorithm for the target business.

[0035] Therefore, in order to solve the above problems, the embodiments of the present application provide a storage write amplification evaluation method and device, an electronic device and a medium, to accurately and quickly evaluate the write amplification result of the storage algorithm for the target business.

[0036] The method provided by the embodiments of the present application is described below:

[0037] Please refer to Figure 1 ,Figure 1 A method flowchart is provided for the embodiments of the present application. As shown in the flowchart, the flowchart can include the following steps: Figure 1

[0038] S101, parsing a read-write I / O request file of a target service to obtain each I / O request in the I / O request file; the I / O request includes a read request and a write request.

[0039] Parses an I / O request file of a target service to obtain each I / O request in the I / O request file.

[0040] In the embodiment, the I / O request file is an existing Block I / O Trace file, and the obtained Block I / O Trace file of the target service is parsed to obtain each I / O request recorded in the file; the Block I / O Trace file of any service is used to record each I / O request requested by the service in a specified historical time period.

[0041] In the embodiment, the Block I / O Trace file can only record each I / O request from the service level (i.e., the logical level), and cannot record the storage address in the actual storage medium corresponding to each I / O request.

[0042] As an embodiment, the Block I / O Trace file can be obtained from a trace storage module of a service end in another running storage system storing the target service.

[0043] S102, obtaining a write request from each I / O request obtained, traversing each write request obtained in sequence, and taking the write request currently traversed as a current write request.

[0044] In the embodiment, the traversal can be performed according to the sequence of each write request recorded in the Block I / O Trace file.

[0045] S103, playing back the current write request, determining whether there is a virtual storage space in the virtual storage device that can accommodate at least the data written by the current write request, if yes, updating the storage state of the virtual storage location corresponding to the virtual storage space in the virtual storage device to a to-be-occupied state, the storage state is used to indicate to-be-occupied, and when it is determined that the data written by the current write request causes the existence of invalid data, updating the storage state of the virtual storage location corresponding to the invalid data in the virtual storage device to an invalid state, the invalid state is used to indicate to-be-erased; and traversing the next write request of the current write request from all other write requests that have not been traversed, returning to playing back the current write request; if no, outputting an alarm indication, and ending the playback.

[0046] ​The specific implementation of step S103 will be described in detail hereinafter, and thus will not be repeated here.

[0047] S104, after the playback is ended, determining the WA value based on the number of virtual storage locations currently in the to-be-occupied state, and based on the number of logical storage locations corresponding to each write request in the I / O request file.

[0048] As an embodiment, the above method further comprises the following steps: during the playback of each write request, when the number of virtual storage locations in the invalid state meets the set erasure condition, erasing the invalid data in the invalid state and migrating the valid data in the to-be-occupied state in the same virtual storage block as the invalid data (this process is referred to as garbage collection GC).

[0049] The storage state of the virtual storage location corresponding to the erased invalid data in the virtual storage device is changed from the invalid state to the free state; the storage state of the virtual storage location before migration corresponding to the migrated valid data is changed from the to-be-occupied state to the free state, and the storage state of the virtual storage location after migration corresponding to the migrated valid data is changed from the free state to the to-be-occupied state.

[0050] When the number of virtual storage locations in the invalid state meets the set stop-migration condition, the migration is stopped.

[0051] Correspondingly, the implementation of step S104 is changed to be determined based on the number of virtual storage locations currently in the to-be-occupied state after the playback is ended, the number of virtual storage locations occupied by the migrated valid data, and the number of logical storage locations corresponding to each I / O request. This specific implementation of step S104 will be described in detail hereinafter in the form of a specific embodiment, and thus will not be repeated here.

[0052] At this point, the completion Figure 1 of the flowchart shown.

[0053] By Figure 1As shown in the flow, in the embodiment, the I / O request file of the target service is parsed, each write request in the obtained I / O requests is traversed in sequence, the traversed write request is taken as a current write request, the current write request is played back, it is determined whether there is a virtual storage space in the virtual storage device which at least contains the data written by the current write request, if yes, the storage state of the virtual storage location corresponding to the virtual storage space in the virtual storage device is updated to a to-be-occupied state, the storage state is used to indicate to-be-occupied, and when invalid data caused by the data written by the current write request is determined, the storage state of the virtual storage location corresponding to the invalid data in the virtual storage device is updated to an invalid state, the invalid state is used to indicate to-be-erased; and the next write request of the current write request is traversed from all other write requests which have not been traversed, and the playback of the current write request is returned; if no, an alarm indication is output, and the playback is ended, and after the playback is ended, the WA value is determined based on the number of virtual storage locations which are currently in the to-be-occupied state and based on the number of logical storage locations corresponding to each write request in the I / O request file.

[0054] This way of constructing a virtual storage space for a target service, simulating the execution of each write request in the virtual storage space, and determining the WA value of the target service when the virtual storage space corresponding to each virtual storage location changes in the case that there is a virtual storage space which at least contains the data written by the current write request, removes the dependence on hardware, saves hardware costs, and saves the running time of real writing of the physical storage medium, so as to quickly obtain the WA value.

[0055] The specific implementation of the above step S103 is described in detail as follows:

[0056] The determination of whether there is a virtual storage space in the virtual storage device which at least contains the data written by the current write request in the above step S103 has many implementation ways, for example, as an embodiment, the storage state of each virtual storage location in the virtual storage device and the data written by the current write request are input to an external storage algorithm, if the storage algorithm outputs a virtual storage space which contains the data written by the current write request, it is determined that there is a virtual storage space which at least contains the data written by the current write request, and if the storage algorithm outputs a virtual storage location which contains the data written by the current write request, it is determined that there is no virtual storage space which at least contains the data written by the current write request.

[0057] Specifically, the virtual storage space in the known constructed virtual storage device is composed of a plurality of virtual storage blocks and a plurality of storage pages on each block. Determining whether there is virtual storage space in the virtual storage device to accommodate the data written by the current write request is to find the page (i.e. the physical block address PBA) in the virtual storage space that accommodates the data written by the current write request. When playing back the first write request, the configuration information of the virtual storage space composed of a plurality of virtual storage blocks and a plurality of storage pages on each block, and the logical block address LBA carried by the current write request are sent to the external storage algorithm, and the external storage algorithm calculates whether the PBA corresponding to the LBA is calculated. If calculated, the next write request of the current write request is traversed, and if not calculated, an alarm indication is output and the playback is ended.

[0058] Correspondingly, the specific implementation of updating the storage state of the virtual storage location corresponding to the virtual storage space in the virtual storage device to the to-be-occupied state when it is determined in step S103 that there is virtual storage space in the virtual storage device to accommodate the data written by the current write request can be: based on the calculated PBA corresponding to the LBA, updating the bitmap corresponding to the physical block address PBA in the existing first bitmap from the initial set mark to the first mark, such as from 0 to 1.

[0059] Correspondingly, the specific implementation of determining whether the data written by the current write request causes invalid data can be: if it is found from the locally recorded correspondence between the historical LBA and the PBA that there is a historical LBA identical to the LBA carried by the current write request, and the historical PBA corresponding to the historical LBA is different from the current PBA, if the historical LBA identical to the LBA is found, and the historical PBA corresponding to the historical LBA is different from the current PBA, it is determined that there is invalid data, and the data at the historical PBA is determined as invalid data, and the correspondence between the LBA and the historical PBA is changed to the correspondence between the LBA and the current PBA. If not found, it is determined that there is no invalid data, and the correspondence between the LBA and the current PBA is directly recorded.

[0060] Based on the above description, the specific implementation of updating the storage state of the virtual storage location corresponding to the invalid data in the virtual storage device to the invalid state in step S103 can be: based on the PBA corresponding to the invalid data, updating the bitmap corresponding to the physical block address PBA in the existing second bitmap from the initial set mark to the second mark, such as from 0 to 1.

[0061] The specific implementation of the above garbage collection process steps is described in detail below.

[0062] The specific implementation of changing the storage state of the virtual storage location corresponding to the invalid data erased in the virtual storage device from the invalid state to the idle state can be: based on the physical block address PBA corresponding to the invalid data erased, updating the bit mark corresponding to the physical block address PBA in the existing second bitmap from the second mark to the initial setting mark, such as from 1 to 0.

[0063] The specific implementation of changing the storage state of the virtual storage location corresponding to the invalid data erased in the virtual storage device from the invalid state to the idle state can be: based on the physical block address PBA corresponding to the invalid data erased, updating the bit mark corresponding to the physical block address PBA in the existing second bitmap from the second mark to the initial setting mark, such as from 1 to 0.

[0064] The specific implementation of changing the storage state of the virtual storage location corresponding to the invalid data erased in the virtual storage device from the invalid state to the idle state can be: based on the physical block address PBA corresponding to the invalid data erased, updating the bit mark corresponding to the physical block address PBA in the existing second bitmap from the second mark to the initial setting mark, such as from 1 to 0.

[0065] The specific implementation of changing the storage state of the virtual storage location corresponding to the invalid data erased in the virtual storage device from the invalid state to the idle state can be: based on the physical block address PBA corresponding to the invalid data erased, updating the bit mark corresponding to the physical block address PBA in the existing second bitmap from the second mark to the initial setting mark, such as from 1 to 0.

[0066] The specific implementation of changing the storage state of the virtual storage location corresponding to the invalid data erased in the virtual storage device from the invalid state to the idle state can be: based on the physical block address PBA corresponding to the invalid data erased, updating the bit mark corresponding to the physical block address PBA in the existing second bitmap from the second mark to the initial setting mark, such as from 1 to 0.

[0067] The total number of pages in the invalid state in the virtual storage device can be calculated by the following method: after each write request playback, when it is determined that the page corresponding to the historical PBA corresponding to the LBA carried in the write request is invalid data, the number of pages in the invalid state in a block is counted, and the corresponding count is increased to update the total number of pages in the invalid state in the block. The total number of pages in the invalid state in all blocks is counted to update the total number of pages in the invalid state in the virtual storage device.

[0068] The number of invalid pages in the invalid state deleted in the garbage collection process of a block is counted, and the corresponding count is reduced to update the total number of pages in the invalid state in the block. The number of invalid pages deleted in all blocks is counted, and the corresponding count is reduced to update the total number of invalid pages in the invalid state in the virtual storage device.

[0069] The specific implementation of the above-mentioned erasing of the invalid data in the invalid state and migrating the valid data in the to-be-occupied state in the same virtual storage block as the invalid data can be: inputting the storage state of each virtual storage location in the virtual storage device into an external recycling algorithm to obtain a recycling strategy, the recycling strategy being used to indicate which block is selected. The invalid pages in the block are deleted and the valid pages are migrated.

[0070] The above-mentioned garbage collection process is described in detail.

[0071] The specific implementation of the above-mentioned step S104 is described in detail below.

[0072] The specific implementation of the above-mentioned determination based on the number of virtual storage locations in the to-be-occupied state after the end of the playback, the number of virtual storage locations occupied by the migrated valid data, and the number of logical storage locations corresponding to each I / O request can be: the ratio of the sum of the total number of pages in the to-be-occupied state after the end of the playback and the total number of migrated pages to the number of pages corresponding to each I / O request at the logical level is the above-mentioned WA value.

[0073] Exemplarily, the total number of pages in the to-be-occupied state can be calculated as follows: after playing back a write request, the above-mentioned first bitmap is updated once, and the garbage collection of a block is performed, and the above-mentioned first bitmap is also updated once. Each time the first bitmap is updated, the number of pages in the to-be-occupied state newly added in the block corresponding to the first bitmap or the number of pages in the to-be-occupied state reduced in the block due to the migration of valid data is counted by comparing the bitmap before and after the update, and the total number of pages in the to-be-occupied state in the block is updated by increasing or reducing the count.

[0074] The total number of migrated pages can be calculated as follows: after obtaining the migration strategy, the number of migrated valid pages in each block is counted, and the corresponding count is increased to update the total number of migrated pages in the virtual storage device.

[0075] The specific implementation of step S104 is described in detail above.

[0076] To elaborate the present solution in more detail, the following is combined with Figure 2 and Figure 3 The method of the present application is described in more detail as follows:

[0077] The flow is as follows:

[0078] 1. Analyze the input parameters of the device.

[0079] In this step, the above-mentioned parameters at least include a plurality of virtual storage blocks block of a virtual storage space, and a plurality of storage pages page on each block.

[0080] 2. Analyze the I / O request file Block I / O Trace file of the target service to obtain each I / O request therefrom.

[0081] 3. Traverse each write request in order from each I / O request obtained, and take the traversed write request as the current write request.

[0082] 4. Call an external storage algorithm (i.e., a storage algorithm to be tested) to determine whether the LBA carried in the current write request corresponds to the PBA, if found, execute step 5 below, and continue to issue the next write request, otherwise, end the playback.

[0083] 5. Update the storage state information of each virtual storage location in the virtual storage device based on the found PBA.

[0084] The specific storage state information of each virtual storage location in the virtual storage device is described in detail in Figure 2 , Figure 2 The specific updating method of each information is referred to the specific implementation of steps S103 and S104 described above, which will not be repeated here.

[0085] 6. After ending the playback, obtain the statistical information, and take the ratio of the sum of the total number of pages to be occupied and the total number of migrated pages in the statistical information to the number of pages corresponding to the logical level of each I / O request as the WA value.

[0086] The method provided by the embodiment of the present application is described above, and the device provided by the embodiment of the present application is described as follows:

[0087] Refer to Figure 4 , Figure 4 the device structure diagram provided by the embodiment of the present application. As shown in Figure 4 , the device comprises an analysis module 401, an evaluation module 402, and a determination module 403.

[0088] The parsing module 401 is configured to parse an I / O request file of a target service to obtain each I / O request in the I / O request file.

[0089] The evaluation module 402 is configured to sequentially traverse each write request from the obtained I / O requests, and take the traversed write request as a current write request.

[0090] The playback is performed on the current write request, and it is determined whether there is a virtual storage space in the virtual storage device that can accommodate at least the data written by the current write request. If yes, the storage state of the virtual storage location corresponding to the virtual storage space in the virtual storage device is updated to a to-be-occupied state, and the storage state is used to indicate to-be-occupied. When it is determined that the data written by the current write request causes invalid data to exist, the storage state of the virtual storage location corresponding to the invalid data in the virtual storage device is updated to an invalid state, and the invalid state is used to indicate to-be-erased. The next write request of the current write request is traversed from all other write requests that have not been traversed, and the playback on the current write request is returned. If no, an alarm indication is output, and the playback is ended.

[0091] The determination module 403 is configured to determine a WA value based on the number of virtual storage locations in the to-be-occupied state and the number of logical storage locations corresponding to each write request in the I / O request file after the playback is ended.

[0092] As an embodiment, the evaluation module is further configured to:

[0093] In the process of playing back each write request, when the number of virtual storage locations in the invalid state meets a set erasing condition, the invalid data in the invalid state is erased, and the valid data in the to-be-occupied state in the same virtual storage block as the invalid data is migrated.

[0094] The storage state of the virtual storage location corresponding to the erased invalid data in the virtual storage device is changed from the invalid state to an idle state. The storage state of the virtual storage location before migration corresponding to the migrated valid data is changed from the to-be-occupied state to the idle state, and the storage state of the virtual storage location after migration corresponding to the migrated valid data is changed from the idle state to the to-be-occupied state.

[0095] When the number of virtual storage locations in the invalid state meets a set stop-migration condition, the migration is stopped.

[0096] The WA value is determined based on the number of virtual storage locations in the to-be-occupied state, the number of virtual storage locations occupied by the migrated valid data, and the number of logical storage locations corresponding to each I / O request after the playback is ended.

[0097] As an embodiment, updating the storage state of the virtual storage location corresponding to the virtual storage space in the virtual storage device to the to-be-occupied state comprises:

[0098] updating, based on a physical block address PBA corresponding to the virtual storage location, a bit map mark corresponding to the physical block address PBA in the existing first bit map from an initial setting mark to a first mark;

[0099] updating the storage state of the virtual storage location corresponding to the invalid data in the virtual storage device to the invalid state comprises:

[0100] updating, based on a physical block address PBA corresponding to the virtual storage location corresponding to the invalid data, a bit map mark corresponding to the physical block address PBA in the existing second bit map from an initial setting mark to a second mark.

[0101] As an embodiment, changing the storage state of the virtual storage location corresponding to the erased invalid data in the virtual storage device from the invalid state to the idle state comprises:

[0102] updating, based on a physical block address PBA corresponding to the virtual storage location corresponding to the erased invalid data, a bit map mark corresponding to the physical block address PBA in the existing second bit map from the second mark to the initial setting mark;

[0103] changing the storage state of the virtual storage location corresponding to the migrated valid data before migration from the to-be-occupied state to the idle state comprises:

[0104] updating, based on a physical block address PBA corresponding to the virtual storage location corresponding to the migrated valid data before migration, a bit map mark corresponding to the physical block address PBA in the existing first bit map from the second mark to the initial setting mark;

[0105] changing the storage state of the virtual storage location corresponding to the migrated valid data after migration from the idle state to the to-be-occupied state comprises:

[0106] updating, based on a physical block address PBA corresponding to the virtual storage location corresponding to the migrated valid data after migration, a bit map mark corresponding to the physical block address PBA in the existing first bit map from the initial setting mark to the second mark.

[0107] As an embodiment, determining whether there is virtual storage space in the virtual storage device to accommodate at least the data written by the current write request comprises:

[0108] The storage state of each virtual storage location in the virtual storage device and the data written by the current write request are input to an external storage algorithm, and if the storage algorithm outputs a virtual storage location that can accommodate the data written by the current write request, it is determined that there is at least a virtual storage space that can accommodate the data written by the current write request, and if the storage algorithm does not output a virtual storage location that can accommodate the data written by the current write request, it is determined that there is no virtual storage space that can accommodate the data written by the current write request.

[0109] As an embodiment, any write request carries a logical block address (LBA); the device further comprises:

[0110] The storage module is configured to search, in the correspondence between the locally recorded LBAs and PBAs, whether there is a historical LBA identical to the LBA carried in the current write request and a record in which the historical PBA corresponding to the historical LBA is different from the current PBA.

[0111] If the record is found, the correspondence between the LBA and the historical PBA is changed to the correspondence between the LBA and the current PBA.

[0112] If the record is not found, the correspondence between the LBA and the current PBA is recorded.

[0113] In the case where the record is found, it is determined that the data written by the current write request causes the existence of invalid data, and the data at the historical PBA is invalid data.

[0114] Thus, the following is completed Figure 4 The structure of the device is described.

[0115] Referring to Figure 5 , Figure 5 The electronic device structure diagram provided by the embodiment of the present application is shown. As shown in the figure, the hardware structure can include a processor and a machine readable storage medium, the machine readable storage medium stores machine executable instructions that can be executed by the processor; the processor is configured to execute the machine executable instructions to implement the method disclosed in the above examples of the present application. Figure 5 Based on the same application concept as the above method, the embodiment of the present application further provides a machine readable storage medium, the machine readable storage medium stores a plurality of computer instructions, and the computer instructions are executed by the processor to implement the method disclosed in the above examples of the present application.

[0116]

[0117] ​Exemplarily, the machine-readable storage medium described above can be any electronic, magnetic, optical, or other physical storage apparatus, and can contain or store information such as executable instructions, data, and the like. For example, the machine-readable storage medium can be a RAM (Random Access Memory), a volatile memory, a non-volatile memory, a flash memory, a storage drive (such as a hard drive), a solid-state drive, any type of storage disc (such as a compact disc, a DVD, and the like), or similar storage media, or a combination thereof.

[0118] The above merely provides an example of the present application, and is not intended to limit the present application. The present application can have various modifications and changes for those skilled in the art. Any modification, equivalent replacement, improvement, and the like within the spirit and principle of the present application shall be included in the scope of claims of the present application.

Claims

1. A method for evaluating write amplification (WA) in memory, characterized in that, The method comprises: analyzing a read-write I / O request file of a target service to obtain each I / O request in the I / O request file; the I / O request comprises a read request and a write request; obtaining a write request from each I / O request obtained, sequentially traversing each write request obtained, and taking the write request currently traversed as a current write request; playing back the current write request to determine whether there is a virtual storage space in a virtual storage device that can accommodate at least data written by the current write request, if yes, updating a storage state of a virtual storage location corresponding to the virtual storage space in the virtual storage device to a to-be-occupied state, the storage state being used to indicate to-be-occupied, and when it is determined that the data written by the current write request causes invalid data to exist, updating a storage state of a virtual storage location corresponding to the invalid data in the virtual storage device to an invalid state, the invalid state being used to indicate to-be-erased; and traversing a next write request of the current write request from all other write requests that have not been traversed, and returning to playing back the current write request; if no, outputting an alarm indication, and ending the playback; after the playback is ended, determining a WA value based on a number of virtual storage locations currently in the to-be-occupied state and based on a number of logical storage locations corresponding to each write request in the I / O request file.

2. The method of claim 1, wherein, The method further comprises: in the process of playing back each write request, when a number of virtual storage locations in the invalid state meets a set erasing condition, erasing the invalid data in the invalid state and migrating valid data in the to-be-occupied state in a same virtual storage block as the invalid data; changing the storage state of the virtual storage location corresponding to the erased invalid data in the virtual storage device from the invalid state to an idle state; changing the storage state of the virtual storage location before migration corresponding to the migrated valid data from the to-be-occupied state to the idle state, and changing the storage state of the virtual storage location after migration corresponding to the migrated valid data from the idle state to the to-be-occupied state; when the number of virtual storage locations in the invalid state meets a set stopping migration condition, stopping the migration; wherein the WA value is determined based on the number of virtual storage locations currently in the to-be-occupied state after the playback is ended, a number of virtual storage locations occupied by the migrated valid data, and the number of logical storage locations corresponding to each I / O request.

3. The method of claim 1, wherein, The method further comprises: updating a bit map corresponding to a physical block address PBA in a virtual physical storage space corresponding to the virtual storage location from an initial set mark in an existing first bit map to a first mark; The method further comprises: updating a bit map corresponding to a physical block address PBA in a virtual physical storage space corresponding to the virtual storage location from an initial set mark in an existing second bit map to a second mark.

4. The method of claim 2, wherein, The method further comprises: The changing of the storage state of the virtual storage location corresponding to the invalid data in the virtual storage device from the invalid state to the idle state comprises: updating the bit mark corresponding to the PBA of the virtual storage location corresponding to the invalid data in the existing second bitmap from the second mark to the initial setting mark; The changing of the storage state of the virtual storage location corresponding to the migrated valid data before migration from the to-be-occupied state to the idle state comprises: updating the bit mark corresponding to the PBA of the virtual storage location corresponding to the migrated valid data before migration in the existing first bitmap from the initial setting mark to the second mark; The changing of the storage state of the virtual storage location corresponding to the migrated valid data after migration from the idle state to the to-be-occupied state comprises:

5. The method of claim 1, wherein, updating the bit mark corresponding to the PBA of the virtual storage location corresponding to the migrated valid data after migration in the existing first bitmap from the initial setting mark to the second mark. The determining of whether there is virtual storage space in the virtual storage device to accommodate the data written by the current write request comprises:

6. The method of claim 1, wherein, inputting the storage state of each virtual storage location in the virtual storage device and the data written by the current write request into an external storage algorithm, and if the storage algorithm outputs the virtual storage location to accommodate the data written by the current write request, it is determined that there is virtual storage space to accommodate the data written by the current write request, and if the storage algorithm does not output the virtual storage location to accommodate the data written by the current write request, it is determined that there is no virtual storage space to accommodate the data written by the current write request. Any write request carries a logical block address (LBA); After the determining of whether there is virtual storage space in the virtual storage device to accommodate the data written by the current write request, the method further comprises: in the locally recorded correspondence between the LBA and the PBA, searching for whether there is a record of a historical LBA identical to the LBA carried in the current write request, and the historical PBA corresponding to the historical LBA is different from the current PBA; if the record is found, changing the correspondence between the LBA and the historical PBA to the correspondence between the LBA and the current PBA; if the record is not found, recording the correspondence between the LBA and the current PBA; 7. A storage write amplification (WA) evaluation apparatus, comprising: wherein, when the record is found, it is determined that the data written by the current write request causes the existence of invalid data, and the data at the historical PBA is the invalid data. The apparatus comprises: a parsing module configured to parse a read-write I / O request file of a target service to obtain each I / O request in the I / O request file; the I / O request comprises a read request and a write request; an evaluation module configured to obtain a write request from each I / O request obtained, and traverse each write request obtained in sequence, and take the currently traversed write request as a current write request. playing back the current write request, determining whether there is a virtual storage space in the virtual storage device that can accommodate at least the data written by the current write request, if yes, updating a storage state of a virtual storage location corresponding to the virtual storage space in the virtual storage device to a to-be-occupied state, the storage state being used to indicate to-be-occupied, and when it is determined that the data written by the current write request causes invalid data to exist, updating a storage state of a virtual storage location corresponding to the invalid data in the virtual storage device to an invalid state, the invalid state being used to indicate to-be-erased; and traversing a next write request of the current write request from all other write requests that have not been traversed, returning to playing back the current write request; if no, outputting an alarm indication, and ending the playing back; determining a WA value based on a number of virtual storage locations in the to-be-occupied state and a number of logical storage locations corresponding to each write request in the I / O request file after the playing back ends.

8. The apparatus of claim 7, wherein, The evaluation module is further configured to: when the number of virtual storage locations in the invalid state meets a set erasing condition, erasing the invalid data in the invalid state and migrating valid data in the to-be-occupied state in a same virtual storage block as the invalid data during the playing back of each write request; changing the storage state of the virtual storage location corresponding to the erased invalid data in the virtual storage device from the invalid state to an idle state; changing the storage state of the virtual storage location corresponding to the migrated valid data before migration from the to-be-occupied state to the idle state, and changing the storage state of the virtual storage location corresponding to the migrated valid data after migration from the idle state to the to-be-occupied state; when the number of virtual storage locations in the invalid state meets a set stopping migration condition, stopping the migration; wherein the WA value is determined based on the number of virtual storage locations in the to-be-occupied state, the number of virtual storage locations occupied by the migrated valid data, and the number of logical storage locations corresponding to each I / O request after the playing back ends; and / or, the updating of the storage state of the virtual storage location corresponding to the virtual storage space in the virtual storage device to the to-be-occupied state comprises: updating a bit map corresponding to a physical block address PBA in a virtual physical storage space corresponding to the virtual storage location from an initial set mark to a first mark in an existing first bit map; the updating of the storage state of the virtual storage location corresponding to the invalid data in the virtual storage device to the invalid state comprises: updating a bit map corresponding to a physical block address PBA in a virtual physical storage space corresponding to the virtual storage location from an initial set mark to a second mark in an existing second bit map; the updating of the storage state of the virtual storage location corresponding to the invalid data in the virtual storage device to the invalid state comprises: updating a bit map corresponding to a physical block address PBA in a virtual physical storage space corresponding to the virtual storage location from an initial set mark to a second mark in an existing second bit map; and / or, the changing the storage state of the virtual storage location corresponding to the invalid data erased in the virtual storage device from the invalid state to the free state comprises: based on a physical block address PBA corresponding to the virtual storage location corresponding to the invalid data erased, updating a bit map mark corresponding to the physical block address PBA in the existing second bit map from the second mark to the initial setting mark; the changing the storage state of the virtual storage location corresponding to the invalid data erased in the virtual storage device from the invalid state to the free state comprises: based on a physical block address PBA corresponding to the virtual storage location corresponding to the invalid data erased, updating a bit map mark corresponding to the physical block address PBA in the existing second bit map from the second mark to the initial setting mark; the changing the storage state of the virtual storage location corresponding to the invalid data erased in the virtual storage device from the invalid state to the free state comprises: based on a physical block address PBA corresponding to the virtual storage location corresponding to the invalid data erased, updating a bit map mark corresponding to the physical block address PBA in the existing second bit map from the second mark to the initial setting mark; and / or, the determining whether there is virtual storage space in the virtual storage device to accommodate at least the data written by the current write request comprises: inputting the storage state of each virtual storage location in the virtual storage device and the data written by the current write request into an external storage algorithm, if the storage algorithm outputs a virtual storage location to accommodate the data written by the current write request, it is determined that there is virtual storage space to accommodate at least the data written by the current write request, and if the storage algorithm does not output a virtual storage location to accommodate the data written by the current write request, it is determined that there is no virtual storage space to accommodate at least the data written by the current write request; and / or, any write request carries a logical block address LBA; the apparatus further comprises: a storage module, configured to search, in a recorded correspondence between the LBA and the PBA, whether there is a historical LBA identical to the LBA carried in the current write request, and the historical PBA corresponding to the historical LBA is different from the current PBA; if the record is found, changing the correspondence between the LBA and the historical PBA to the correspondence between the LBA and the current PBA; if the record is not found, recording the correspondence between the LBA and the current PBA; wherein, when the record is found, it is determined that the data written by the current write request causes the existence of invalid data, and the data at the historical PBA is the invalid data.

9. An electronic device, comprising: The electronic device comprises: a processor; and a computer readable storage medium having stored therein computer program instructions, which, when executed by the processor, cause the processor to perform the steps of the method of any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium has stored thereon computer program instructions, which, when executed by a processor, cause the processor to perform the steps of the method of any one of claims 1 to 6.

Images