A data processing method and device for storage device and storage device

Abstract

This application provides a data processing method, apparatus, storage device, and computer-readable storage medium for a storage device; and relates to hard disk technology for cloud computing. The method comprises: obtaining a power-on time and a power-off time of a storage device; determining a power-off duration of the storage device based on the power-on time and the power-off time; determining a data migration frequency of the storage device based on the power-off duration; and performing a data migration operation on at least some data blocks in the storage device based on the data migration frequency. This application can improve the stability of data in a storage device.

Description

Technical Field

[0001] The present application relates to data storage technology and cloud computing technology, and in particular to a data processing method and apparatus for a storage device, a storage device, and a computer-readable storage medium. Background Art

[0002] Cloud computing is a computing model that distributes computing tasks across a resource pool consisting of a large number of computers, enabling various application systems to access computing power, storage space, and information services as needed. The network that provides these resources is called the "cloud." Cloud computing infrastructure providers establish a cloud computing resource pool (referred to as a cloud platform, often referred to as an Infrastructure as a Service (IaaS) platform) and deploy various types of virtual resources within the resource pool for external clients to choose from. This resource pool primarily includes computing devices (virtualized machines, including operating systems), storage devices, and network equipment.

[0003] For example, a solid-state drive (SSD) or mechanical hard drive uses computer flash memory (NAND) or magnetic disks as its storage medium. Due to the physical properties of NAND disks, the electrons and magnetism of the NAND disks gradually degrade over time, and when this threshold is exceeded, data errors may occur.

[0004] The related art lacks an effective solution for improving the stability of data in storage devices. Summary of the Invention

[0005] Embodiments of the present application provide a data processing method and apparatus for a storage device, a storage device, and a computer-readable storage medium, which can improve the stability of data in the storage device.

[0006] The technical solution of the embodiment of the present application is implemented as follows:

[0007] An embodiment of the present application provides a data processing method for a storage device, including:

[0008] Get the power-on time and power-off time of the storage device;

[0009] Determining a power-off duration of the storage device based on the power-on time and the power-off time;

[0010] Determining a data migration frequency of the storage device based on the power-off duration;

[0011] According to the data migration frequency, a data migration operation is performed on at least part of the data blocks in the storage device.

[0012] An embodiment of the present application provides a data processing device for a storage device, comprising:

[0013] An acquisition module is used to obtain the power-on time and power-off time of the storage device; a first determination module is used to determine the power-off duration of the storage device based on the power-on time and the power-off time; a second determination module is used to determine the data migration frequency of the storage device based on the power-off duration; and a migration module is used to perform data migration operations on at least part of the data blocks in the storage device according to the data migration frequency.

[0014] In the above solution, the acquisition module is also used to receive the power-on time sent by the host device when the storage device is powered on, and to search the power-off time recorded when the power is off from the storage device; wherein there is an electrical connection between the host device and the storage device.

[0015] In the above scheme, the second determination module is also used to determine the difference between the power-off duration and the maximum duration of stable operation of the storage device when the power-off duration is less than the maximum duration of stable operation of the storage device; and use the ratio of the total amount of data blocks to the difference as the frequency of moving at least some of the data blocks in the storage device; wherein the total amount of data blocks is the number of at least some of the data blocks in the storage device.

[0016] In the above solution, the migration module is also used to start data migration operations on at least part of the data blocks in the storage device immediately after the storage device is powered on when the power-off duration is greater than or equal to the maximum duration of stable operation of the storage device.

[0017] In the above scheme, the moving module is also used to perform error detection on the data in the data block moved in the storage device to obtain the number of erroneous bits of the data; by comparing the relationship between the number of bits and the first bit number threshold, the status of the storage device is determined, and the status is recorded in the storage device; wherein the status of the storage device includes: normal status, warning status, and error status.

[0018] In the above scheme, the migration module is also used to scan and process at least part of the data blocks in the storage device when the power-on time or power-off time of the storage device is not obtained; when the number of erroneous bits in any data block exceeds a second bit number threshold, perform a data migration operation on the any data block; wherein the order of the scanning and processing includes: the time order of the data blocks being written; the numbering order of the data blocks; and a random order.

[0019] In the above scheme, the moving module is further used to perform the following processing on the at least part of the data block: reading data from at least part of the data pages in the data block and performing data verification to determine the number of bits in which the data is erroneous; recording the number of bits in the detection file of the storage device; wherein the at least part of the data pages includes at least one of the following: data pages in the data block whose number of errors exceeds a third threshold; data pages randomly selected from the data block; data pages specified in the data block; data pages selected in the data block according to an incremental algorithm.

[0020] In the above solution, the moving module is further configured to perform the following processing on at least part of the data block: copying the data in the data block to a blank data block of the storage device, and when the copying is completed, erasing all the data in the data block.

[0021] In the above solution, the moving module is further used to copy the valid data in the data block to a blank data block of the storage device, and when the copying is completed, perform an erasing operation on the valid data and invalid data in the data block.

[0022] An embodiment of the present application provides a storage device, including:

[0023] Memory for storing data;

[0024] The processor stores executable instructions for implementing the data processing method of the storage device of the embodiment of the present application when executed by the processor.

[0025] An embodiment of the present application provides a computer-readable storage medium storing executable instructions for implementing the data processing method of the storage device provided in the embodiment of the present application when executed by a processor.

[0026] The embodiments of the present application have the following beneficial effects:

[0027] Obtain the power-on time and power-off time of the storage device, determine the power-off duration of the storage device, and be able to calculate the power-off duration of the storage device when the storage device is powered on again, and determine the corresponding data migration strategy based on the power-off duration, that is, determine the data migration frequency of the storage device based on the power-off duration; then perform data migration operations on some data blocks in the storage device based on the data migration frequency, correct the data instability caused by the long power-off period, and improve the stability of the data in the storage device. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 1 is a schematic diagram of the architecture of a data processing system 100 of a storage device provided in an embodiment of the present application;

[0029] Figure 23 is a schematic diagram of the structure of a storage device 300 provided in an embodiment of the present application;

[0030] Figure 3 is a schematic diagram of the logical storage structure of a solid-state drive provided in an embodiment of the present application;

[0031] Figure 4A This is a flow chart of a data processing method for a storage device provided in an embodiment of the present application;

[0032] Figure 4B This is a flow chart of a data processing method for a storage device provided in an embodiment of the present application;

[0033] Figure 4C This is a flow chart of a data processing method for a storage device provided in an embodiment of the present application;

[0034] Figure 5 This is a schematic diagram of an application scenario of performing a data migration operation on a data block provided by an embodiment of the present application;

[0035] Figure 6 This is a flow chart of a data processing method for a storage device provided in an embodiment of the present application;

[0036] Figure 7 This is a flow chart of a data processing method for a storage device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical solutions and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limiting this application. All other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0038] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0039] In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0040] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0041] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0042] 1) Storage devices are electronic devices that store data, such as hard drives, including solid-state drives and mechanical hard drives.

[0043] 2) The host is a computing device that is electrically connected to the storage device, such as a server or terminal device, and can perform read and write operations on the storage device.

[0044] 3) Data block, also called data unit or storage unit, is the unit for storing data in a storage device.

[0045] 4) Computer flash memory devices (NAND, NAND flash memory), non-volatile flash memory particles, can still retain data after power failure.

[0046] 5) NVMe storage system (NVMe, Non-Volatile Memory express) is an open collection of standards and information, a register interface and command set for high-speed serial computer bus (PCI-E, Peripheral Component Interconnect Express) attached storage, and industry-standard software for multiple operating systems.

[0047] 6) Self-Monitoring Analysis and Reporting Technology (SMART) is an automatic hard drive status detection and early warning system and specification. It uses detection instructions built into the hard drive hardware to monitor and record the operating conditions of the hard drive's hardware, such as the head, platter, motor, and circuitry, and compares them with the manufacturer's preset safety values. If the detected conditions are about to or have exceeded the preset safety range, the host's detection hardware or software will automatically warn the user and perform minor automatic repairs to ensure the safety of the hard drive data in advance.

[0048] Due to the characteristics of storage devices (for example, mechanical disks, solid-state drives, etc.), long-term storage will cause data instability. Taking the storage device as an SSD as an example, in order to ensure the stability of the data, the relevant technology regularly moves all the written data in the SSD. Because of the re-write operation, the SSD can maintain stability for a period of time. Assuming that the time to ensure stability is T1, and the SSD has a total of N data blocks, then the relevant technology will regularly perform data migration operations on each data block, and the time interval for the data migration operation is T1 / N. However, during the implementation of this application, it was found that SSD uses NAND as the storage medium. Due to the physical characteristics of NAND itself, as time goes by, the electrons in the storage unit will gradually lose, and when it exceeds a certain threshold, it will cause data errors. In other words, if the SSD is powered off and left for a period of time, the placement time of the erroneous data caused by the SSD during the power-off time exceeds the stable operation time, resulting in an increase in the data error rate and aggravated data instability.

[0049] The present invention provides a data processing method, apparatus, storage device, and computer-readable storage medium for a storage device, which can improve the stability of data in the storage device. The following describes an exemplary application of the data processing method for a storage device provided by the present invention. The data processing method for a storage device provided by the present invention can be implemented by various storage devices, such as mechanical hard disks, solid-state drives, and other types of storage devices. The following describes an exemplary application of the storage device.

[0050] See also Figure 1 , Figure 1 This is an architectural diagram of the data processing system 100 of the storage device provided in an embodiment of the present application. The data processing system of the storage device can be used to support various data storage scenarios. In the data processing system of the storage device, the storage device 300 is electrically connected to the host device 200. It should be noted that the storage device 300 can be installed inside the chassis of the host device 200, and the storage device 300 can also be connected to the host device 200 from the outside through an interface such as USB and Thunderbolt.

[0051] In some embodiments, the central processing unit (CPU) in the host device 200 receives a read / write instruction sent by the user to the CPU through an input device, and sends the read / write instruction to the storage device 300. Then, the processor in the storage device 300 caches the data required for the read / write instruction in the memory, where the memory is used for data exchange between the storage device 300 and the host device 200; the host device 200 can then perform corresponding read / write operations on the data cached in the memory.

[0052] In some embodiments, host device 200 can be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. Storage device 300 can serve as the storage medium of the cloud server and be mounted to a storage directory, in which the cloud server stores data.

[0053] Next, the structure of the storage device for implementing the data processing method of the storage device provided in the embodiment of the present application is described. As mentioned above, the storage device provided in the embodiment of the present application can be Figure 1 Storage device 300 in. Figure 2 , Figure 2 is a structural diagram of a storage device 300 provided in an embodiment of the present application, Figure 2 The storage device 300 shown includes a processor 310 , a cache 320 , and a memory 330 .

[0054] Processor 310 is the main controller of storage device 300. It can be an embedded microchip implemented using one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), or field-programmable gate arrays (FPGAs), in which various executable instructions for the main control algorithm are stored in a programmable manner. Processor 310 functions as a command center, issuing all operation requests for the storage device, from actually reading and writing data to executing garbage collection and wear-leveling algorithms to ensure the speed and cleanliness of the storage device. It can be said that the processor is the brain of the storage device.

[0055] The buffer 320 is used to buffer read / write data. For example, the buffer 320 may be a DRAM buffer.

[0056] Memory 330 is the storage medium of the storage device, which is used to store data of the storage device. Memory 330 not only determines the service life of the storage device, but also has a great impact on the performance of the storage device. For example, memory 330 can be NAND flash memory or magnetic media.

[0057] In some embodiments, the processor 310 of the storage device 300 sets a task amount for the cache 320 (for example, DR AM). After the storage device 300 receives data sent by the host device 200, it first stores the data in the cache 320 for corresponding read and write operations. When the data in the cache 320 exceeds the set task amount, the processor 310 stores the data in the cache 320 in the memory 330 to reduce the burden on the processor and improve the efficiency of the storage device in processing data.

[0058] In some embodiments, the data processing apparatus for a storage device provided in embodiments of the present application may include the following modules: an acquisition module 3111, a first determination module 3112, a second determination module 3113, and a migration module 3114. These modules are logical and can be arbitrarily combined or further separated based on the functions they implement. The functions of each module are described below.

[0059] Taking a solid-state hard disk as an example, the logical storage structure of the solid-state hard disk provided in the embodiment of the present application is described. Figure 3 , Figure 3 This is a schematic diagram of the logical storage structure of the solid-state drive provided by the embodiment of the present application. Among them, the data page Page is the smallest read and write unit, and a Page includes multiple bytes; the data block Block is the smallest erase unit, and a Block includes multiple Pages; the data plane Plane is the smallest unit that the NAND can operate according to commands such as read, write, and erase. A Plane is a storage matrix, and a Plane includes multiple Blocks; the die is the smallest independent unit in the flash memory that can execute commands and report its own status. Multiple Dies can be packaged in a chip, and a Die includes two Planes. The solid-state drive is composed of a floating gate, which is a logic circuit surrounded by silicon dioxide. After being charged to a certain potential, it represents 0, and after the potential drops to a certain threshold, it represents 1; due to the physical characteristics of the solid-state drive, after a long period of power failure, the electrons in the floating gate will overflow, resulting in a change in potential, which will cause data loss.

[0060] Below, by Figure 1 The storage device 300 in the embodiment of the present application performs the data processing method of the storage device as an example. Figure 4A , Figure 4A This is a flow chart of a data processing method for a storage device provided in an embodiment of the present application, which will be combined with Figure 4A Steps 101 to 104 are shown for explanation.

[0061] In step 101, the storage device obtains the power-on time and power-off time of the storage device.

[0062] In some embodiments, when the storage device is powered on, the power-on time sent by the host device is received, and the power-off time recorded when the power is off is searched from the storage device.

[0063] For example, if the storage device is a solid-state drive (SSD), when it is powered on, it synchronizes the system time with the host computer. The SSD's processor maintains and records the system time. If a power outage occurs, the last recorded time is recorded on the SSD as the power-out time. Therefore, the last recorded system time is the storage device's power-out time. When the SSD is powered back on, it synchronizes the host computer's system time again. This synchronized system time is the storage device's power-on time.

[0064] For example, the power-on process can be initiated by the host device, or by inserting a removable hard drive into a port on the host device. Alternatively, the power-on process can be initiated by drawing power from an external power supply independent of the host device or the storage device. For example, if the storage device is a solid-state drive, the electrical connection between the host device and the storage device can be achieved by installing the solid-state drive inside the chassis of the host device, or by connecting the solid-state drive to the host device externally via a USB, Thunderbolt, or other interface. For example, a removable hard drive can be connected to the host device via a USB port.

[0065] In step 102 , the power-off duration of the storage device is determined based on the power-on time and the power-off time of the storage device.

[0066] In some embodiments, the difference between the power-on time and the power-off time of the storage device is the time period during which the storage device is powered off / on, i.e., the power-off duration of the storage device. For example, a solid-state drive that supports the NVMe protocol can record the system time of the host device. That is, the power-off time and power-on time of the storage device can be obtained from the recorded system time, and the power-off duration of the storage device can be determined. The corresponding data migration strategy is then determined based on the power-off duration, which can improve the stability of the data in the storage device.

[0067] In step 103, the data migration frequency of the storage device is determined based on the power-off duration of the storage device.

[0068] In some embodiments, see Figure 4B , Figure 4B This is a flow chart of a data processing method for a storage device provided by an embodiment of the present application, showing Figure 4A Step 103 in the embodiment can also be implemented by executing steps 1031 and 1032. The following description will be given in conjunction with each step.

[0069] In step 1031 , when the power-off duration is less than the maximum duration for stable operation of the storage device, the difference between the power-off duration and the maximum duration for stable operation of the storage device is determined.

[0070] In step 1032, the ratio of the total number of data blocks to the difference is used as the migration frequency of at least some of the data blocks in the storage device, wherein the total number of data blocks is the number of at least some of the data blocks in the storage device.

[0071] For example, if 10 data blocks need to be moved without power failure, and the maximum stable operation duration of the storage device is 10 hours, the data block movement frequency in the storage device can be determined to be 1 time / hour, that is, one data block is moved every hour. If the storage device loses power for 5 hours, the difference between the power failure duration and the maximum stable operation duration of the storage device is determined to be 5 hours, that is, the remaining migration time period is 5 hours. Therefore, the data block movement frequency of each data block in the storage device can be determined to be 2 times / hour, that is, one data block is moved every half hour. This ensures that the data blocks that need to be moved are moved within the maximum stable operation duration of the storage device, thereby ensuring the stability of the stored data.

[0072] It should be noted that the maximum duration of stable operation of the SSD (i.e., the maximum limit duration) can be a pre-set empirical value; it can also be a dynamic value calculated based on the data in the SMART file of the memory; or a reasonable dynamic value can be predicted by training a neural network model.

[0073] In some examples, the maximum duration of stable operation of the SSD is obtained by training with a neural network model, which can be achieved in the following way: using the data in the SMART file as training samples, and marking the data as the maximum duration of actual observations; inputting the training samples into each layer of the neural network model for forward propagation to obtain the predicted maximum duration; initializing the loss function including each training sample and the predicted maximum duration of the corresponding training sample; determining the error between the predicted maximum duration of each training sample and the actual observed maximum duration, and backpropagating the error in the neural network model according to the loss function to determine the change value of the neural network model when the loss function reaches the minimum value, and updating the parameters of the neural network model according to the change value; thereby, the maximum duration of stable operation of the SSD is obtained based on the trained neural network model.

[0074] It should be noted that the neural network model here can be various types of neural network models, such as a recurrent neural network, a recursive neural network, etc.; the loss function can be a cross entropy loss function, a square loss function, etc.

[0075] In some examples, dynamic values ​​calculated based on data in a SMART file of a storage device's memory may include remaining read and write capacity (designed read and write capacity during the life cycle - cumulative number of reads and writes by the host), power-on cycles (one cycle is considered turning on and off), power-on duration (how many hours), unsafe shutdown count, and number of errors.

[0076] In an embodiment of the present application, the data migration frequency of the storage device is adjusted based on the power-off duration, and based on the data migration frequency, a data migration operation is performed on the data blocks that need to be moved in the storage device, so as to correct the data instability problem caused by the power-off time within the maximum duration of stable operation of the storage device, thereby improving the stability of the data in the storage device.

[0077] In step 104, a data migration operation is performed on at least part of the data blocks in the storage device according to the data migration frequency.

[0078] In some embodiments, see Figure 4C , Figure 4C This is a flow chart of a data processing method for a storage device provided by an embodiment of the present application, showing Figure 4A Step 104 in the embodiment can also be implemented by executing steps 1041 and 1042, which will be described in conjunction with each step.

[0079] In step 1041 , the following processing is performed on at least a portion of the data blocks: data in the data blocks are copied to blank data blocks of a storage device.

[0080] In step 1042 , when the copying is completed, an erase operation is performed on all data in the data block.

[0081] In some examples, data in a data block is copied to a blank data block of a storage device, and when the copying is completed, an erase operation is performed on all data in the data block, including: copying valid data in the data block to a blank data block of a storage device, and when the copying is completed, an erase operation is performed on the valid data and invalid data in the data block.

[0082] For example, see Figure 5 , Figure 5 Schematic diagram of an application scenario of performing a data migration operation on a data block provided by an embodiment of the present application. 503 is data block 1 and blank data block 2 before the migration operation; 504 is data block 1 and blank data block 2 after the migration operation. Data block 1 includes valid data and invalid data, 501 is valid data, and 502 is invalid data. Performing a data migration operation on data block 1, that is, copying the valid data in data block 1 to blank data block 2, and marking the valid data in data block 1 as invalid data, can obtain data block 1 and blank data block 2 after the migration operation, as shown in FIG. Figure 5 As shown in 504 , at this time, the processor of the storage device marks the data page in the data block as “unused”, but the data is still stored in the data page.

[0083] In some examples, the data block move operation may move valid data in at least some of the data blocks to blank data blocks whose number is less than that of at least some of the data blocks, so as to reduce the addressing burden and reserve more blank data blocks.

[0084] For example, a data block has 256 data pages, data block 1 includes 156 valid data pages and 100 invalid data pages, and data block 2 includes 50 valid data pages and 206 invalid data pages. The data blocks are moved in sequence, and data block 1 is moved to the blank data block 3. When the data block 2 is moved, data block 2 is first moved to the data block 3 that is not fully filled. Only one data block 3 is needed to store all the valid data pages in data block 1 and data block 2.

[0085] It should be noted that when performing a data migration operation, the erasing operation on a data block can be to clear the data in the data block. Since frequent erasing operations on data blocks will affect the life of the solid-state drive, the erasing operation on the data block in the embodiment of the present application is to mark the corresponding data page in the data block as "unused" (i.e., invalid data page), allowing data to be stored on the data page. When all blank data blocks are used up, new data can be written to the data blocks marked as invalid data pages, that is, the invalid data pages in the data block are first completely erased and replaced with new data.

[0086] It should be noted that the at least some data blocks in the data migration operation performed on at least some data blocks in the storage device may be all data blocks or part of the data blocks.

[0087] In some embodiments, when the power-off duration is greater than or equal to the maximum duration of stable operation of the storage device, a data migration operation is immediately started on at least part of the data blocks in the storage device after the storage device is powered on.

[0088] In some examples, because the storage device performs data migration and read / write operations simultaneously, the data migration operation on at least some of the data blocks in the storage device begins immediately after the storage device is powered on, and the specific time for performing the data migration operation can be determined based on the amount of data in the read / write operations currently being performed by the storage device. If, after power-on, the storage device receives a read / write operation instruction from a host with a data volume greater than a data volume threshold, the data migration operation on at least some of the data blocks is delayed and paused, and the storage device first responds to the read / write operation instruction. After completing the read / write operation instruction, the storage device continues to perform the data migration operation on at least some of the data blocks. If, after power-on, the storage device receives a read / write operation instruction from a host with a data volume less than or equal to the data volume threshold, the storage device immediately begins to perform the data migration operation on at least some of the data blocks in the storage device.

[0089] In some embodiments, during the data migration operation, an error correction function is called to perform error detection on the data in the data block being moved in the storage device to obtain the number of erroneous bits of the data; the status of the storage device is determined by comparing the relationship between the number of bits and a first bit number threshold, and the status is recorded in the storage device; wherein the status of the storage device includes: normal status, warning status, and error status.

[0090] In some examples, when the difference between the number of data error bits and the first bit number threshold exceeds a fourth bit number threshold, the storage device is determined to be in a fault state; when the difference between the number of data error bits and the first bit number threshold is equal to or less than the fourth bit number threshold, the storage device is determined to be in a warning state; when the difference between the number of data error bits and the first bit number threshold is less than the fourth bit number threshold, the storage device is determined to be in a normal state; and the storage device status is recorded in the storage device. When the storage device is connected to a host device, this information can be read and displayed by the host device to alert a user that the storage device is at risk.

[0091] It should be noted that the first bit number threshold is a threshold value representing the reliability of the storage device. If the number of data error bits approaches the first bit number threshold, it means that the storage device will become unreliable, which may lead to data loss or storage device failure. The fourth bit number threshold represents the fluctuation value between the number of data error bits and the first bit number threshold. The fourth bit number threshold is less than the first bit number threshold, that is, when the number of data error bits exceeds the first bit number threshold and the fluctuation amplitude exceeds the fourth bit number threshold, it is a fault or error state; when the number of data error bits exceeds the first bit number threshold and the fluctuation amplitude does not exceed the fourth bit number threshold, it is a warning state; when the number of data error bits is less than the first bit number threshold, it is a normal state.

[0092] The processor of the storage device has data error correction capabilities, that is, it implements data error correction functions through algorithms for error detection and correction (ECC, Error Checking and Correction or Error Correction Coding). For example, the BCH and LDPC algorithms are used to detect errors in the data blocks moved in the storage device. When data is written to the solid-state drive, in addition to the original data, data for error correction is also written. Therefore, when performing data movement operations, errors in the data blocks are discovered and corrected using the error correction data. In this way, even if data errors occur during the data movement operation, they can be discovered and corrected, so that the correct data is written to the target data block to be moved.

[0093] In an embodiment of the present application, a data migration strategy for a storage device is determined based on the duration of the storage device's power failure. When the storage device's power failure duration exceeds a maximum limit (i.e., the maximum duration during which the storage device can operate stably), the data migration operation begins immediately, performs error detection on the migrated data, and records the storage device's status. This allows the user to be alerted to the risks of the storage device when a storage device failure occurs. When migrating data from a storage device, the amount of data being read / written is considered, ensuring both read / write efficiency and correcting data errors in the storage device as quickly as possible through data migration.

[0094] In some embodiments, the storage device cannot support recording the system time of the host device, that is, when the storage device is powered on again, the power-on time and / or power-off time cannot be obtained, and thus the power-off duration of the storage device cannot be determined. The data errors caused by the power-off of the storage device can be corrected by performing the following processing: when the power-on time or power-off time of the storage device is not obtained, at least part of the data blocks in the storage device are scanned and processed; when the number of erroneous bits of data in any data block exceeds a second bit number threshold, data migration operation is performed on this any data block; wherein, the order of scanning and processing includes: the time order of data block writing; the numbering order of data blocks; and the random order.

[0095] In some examples, scanning and processing of at least a portion of data blocks in a storage device can be implemented in the following manner: performing the following processing on at least a portion of the data blocks: reading data from at least a portion of the data pages in the data blocks and performing data verification to determine the number of bits in which the data has errors; recording the number of bits in a monitoring file of the storage device; wherein at least a portion of the data pages include at least one of the following: a data page in which the number of errors in the data block exceeds a third number threshold; a data page randomly selected from the data block; a data page specified in the data block; a data page selected in the data block according to an incremental algorithm.

[0096] In some examples, a solid-state drive is scanned and processed according to the time sequence of data block writing, and data pages in each data block with the number of errors exceeding the third number threshold are selected as an example. When the solid-state drive is powered on, the scanning order of at least some data blocks in the solid-state drive is determined according to the time sequence of data block writing; data is read and verified for the data page in the currently selected data block with the number of errors exceeding the third number threshold, and the number of bits of data errors in this data block is obtained. This bit number is recorded in the detection file of the solid-state drive. When the number of bits in the detection file exceeds the second bit number threshold, a data migration operation is performed on the data block corresponding to this bit number. The second bit number threshold here represents the reliability of the data block. When the data errors in the data block exceed the second bit number threshold, it indicates that the data block is faulty and a data migration operation is required. The third number threshold represents the error rate of the data page. The data page with the number of errors exceeding the third number threshold represents the data page in the data block that is more prone to errors.

[0097] In an embodiment of the present application, when reading data pages in a data block, a method of selecting some data pages in the data block can be used to improve the efficiency of the scanning process while ensuring the detection of data errors.

[0098] The following describes an exemplary application of the present invention in a practical application scenario. Figure 6 , Figure 6 This is a flow chart of a data processing method for a storage device provided in an embodiment of the present application.

[0099] In step 601, the host device synchronizes the system time to the SSD. When the SSD is powered on, the SSD synchronizes the system time of the host device.

[0100] In step 602, the SSD is responsible for updating the maintenance time.

[0101] For example, the SSD's processor may record system time updates into the SSD's memory.

[0102] In step 603, when the SSD is powered off, the power off time is recorded in the SSD.

[0103] In step 604 , when the SSD is powered on again, the host device synchronizes the system time to the SSD.

[0104] When the SSD is powered on next time, it will receive the newly synchronized system time from the host device. This system time is the power-on time.

[0105] For example, when the SSD is powered on, the host device and the SSD synchronize the time to 9 o'clock. The processor in the SSD maintains and records this time. At 10 o'clock, the SSD loses power. At this time, when the SSD is powered on again, it synchronizes the host device time to 11 o'clock, which is the power-on time. At the same time, the SSD obtains the time recorded at the time of power failure from the processor, which is 10 o'clock, and this time is the power-off time.

[0106] In step 605, the SSD calculates the power-off duration based on the time difference and modifies the data migration frequency based on the power-off duration.

[0107] Steps 601 to 605 are a process of obtaining the duration of SSD power failure.

[0108] In some examples, the difference between the two times (power-off time and power-on time) is the period of time during which the SSD loses or loses power, i.e., the power-off duration. When the power-off duration is less than the maximum stable operation duration of the SSD, the difference between the power-off duration and the maximum stable operation duration of the SSD is determined; the ratio of the total number of data blocks to the difference is used as the migration frequency for each data block in the SSD. The total number of data blocks refers to the total number of data blocks in the SSD.

[0109] For example, there are N data blocks in the SSD (here, all data blocks are moved. In some other embodiments, it can be N data blocks that need to be moved). If it is known that in order to maintain data stability, the maximum operating time of the SSD in stable operation is T1, and the power-off time is T2. When T2 >= T1, it means that the power-off placement time of the SSD (i.e., the power-off time) has exceeded the maximum limit time (i.e., the maximum operating time to maintain the stable operation of the SSD), and the SSD needs to immediately start the data migration operation for all data blocks. It should be noted that if the power-off placement time is extremely long, the data may have already been incorrect. Therefore, it is necessary to immediately perform the data migration operation on the data blocks, so as to detect errors in advance, mark the status of the SSD in the SMART (SMART is used to evaluate the current status of the hard disk: normal, warning, error, and can sound an alarm when starting up, allowing the user to have enough time to transfer important data to other storage devices), and record it in the SSD. When the SSD is connected to the host device, this SMART information can be read and displayed by the host device to prompt the user of the risks of this SSD. When T2 < T1, the power-off placement time of the SSD does not exceed the maximum operating time to maintain the stable operation of the SSD, which means that there is still enough time for migration, but the data migration frequency needs to be increased to make up for the previous power-off time. The data migration frequency F = N / (T1 - T2), and the new time interval T for each data block migration can also be calculated as T = (T1 - T2) / N. Performing the data migration operation on the data blocks in the SSD according to the new time interval can correct the incorrect data caused during the power-off time within the maximum operating time of the SSD in stable operation.

[0110] For example, NVMe is currently a protocol widely supported by SSD manufacturers, which can provide faster data reading and writing, and the NVMe protocol itself enables the SSD to have the ability to record the host time. Therefore, the above scheme of the embodiments of the present application can be applied to more and more SSDs.

[0111] In some embodiments, if the host or the SSD does not support recording the host time, the SSD can quickly scan the data in the disk. If, after reading the data, it is found that the number of incorrect bits in any data block reaches a certain threshold, then the SSD will move that any data block. Refer to Figure 7 , Figure 7 is a flowchart of the data processing method of the storage device provided by the embodiments of the present application. It shows that obtaining the power-off duration of the SSD can be achieved by executing steps 701 to 705. It will be described in combination with each step.

[0112] In step 701, the SSD completes power-on.

[0113] In step 702, the SSD scans the data in the disk according to the scanning order. After the SSD is powered on, the SSD will quickly start a round of scanning to prevent data instability caused by the previous power-off period.

[0114] In step 703 , the SSD performs a read operation on the data pages in the data blocks determined by the scanning order.

[0115] In step 704, the SSD determines whether the number of error bits in the data block exceeds a second bit count threshold. If the number of error bits in the data block exceeds the second bit count threshold, step 705 is executed. If the number of error bits in the data block is less than or equal to the second bit count threshold, the next data block to be scanned is determined according to the scanning order, and step 703 is executed.

[0116] In step 705 , the SSD performs a data migration operation on the data block.

[0117] It's important to note that 1) the SSD determines the order in which data blocks are scanned. This can be done by using the chronological order of data writes, with the first-written block scanned first; or by scanning sequentially according to the block number; or randomly. 2) When performing data stability checks on a data block (i.e., scanning processing), it's not necessary to read all the data. Reading can be performed on only one or more data pages. The specific data pages selected can be selected from the data block's more error-prone pages; random selection can be performed; specific data pages can be selected; or incremental selection can be performed using a specific algorithm, such as a greedy algorithm. 3) The read operation performed during scanning processing checks the read data. The read command used here differs from a standard data read. SSDs have built-in data error correction capabilities (such as the LDPC algorithm). While typical read commands only return corrected data, scanning reads crucially return the number of erroneous bits. When this bit count exceeds a certain threshold (i.e., including the number of erroneous bits in the data that have been corrected by the internal error correction function after scanning), a data migration operation is initiated.

[0118] The embodiments of this application cover all application scenarios of SSD power failures. The data processing method for storage devices provided by the embodiments of this application effectively enhances the stability of data in SSDs. Data stability and accuracy are of paramount importance for storage services, reducing the possibility of data loss and improving the reliability of data services.

[0119] The following continues to describe the exemplary structure of the data processing device 311 of the storage device provided by the embodiment of the present application. In some embodiments, such as Figure 2 As shown, the software modules stored in the data processing device 311 of the storage device of the processor 310 may include:

[0120] An acquisition module 3111 is used to obtain the power-on time and power-off time of the storage device; a first determination module 3112 is used to determine the power-off duration of the storage device based on the power-on time and the power-off time; a second determination module 3113 is used to determine the data migration frequency of the storage device based on the power-off duration; and a migration module 3114 is used to perform data migration operations on at least part of the data blocks in the storage device according to the data migration frequency.

[0121] In the above solution, the acquisition module 3111 is also used to receive the power-on time sent by the host device when the storage device is powered on, and to search the power-off time recorded when the power is off from the storage device; wherein there is an electrical connection between the host device and the storage device.

[0122] In the above scheme, the second determination module 3113 is also used to determine the difference between the power-off duration and the maximum duration of stable operation of the storage device when the power-off duration is less than the maximum duration of stable operation of the storage device; and use the ratio of the total amount of data blocks to the difference as the frequency of moving at least some of the data blocks in the storage device; wherein the total amount of data blocks is the number of at least some of the data blocks in the storage device.

[0123] In the above solution, the migration module 3114 is also used to start data migration operations on at least part of the data blocks in the storage device immediately after the storage device is powered on when the power-off duration is greater than or equal to the maximum duration of stable operation of the storage device.

[0124] In the above scheme, the moving module 3114 is also used to perform error detection on the data in the data block moved in the storage device to obtain the number of erroneous bits of the data; by comparing the relationship between the number of bits and the first bit number threshold, the status of the storage device is determined, and the status is recorded in the storage device; wherein the status of the storage device includes: normal status, warning status, and error status.

[0125] In the above scheme, the migration module 3114 is also used to scan and process at least part of the data blocks in the storage device when the power-on time or power-off time of the storage device is not obtained; when the number of erroneous bits in any data block exceeds a second bit number threshold, perform a data migration operation on any data block; wherein the order of the scanning and processing includes: the time order of the data blocks being written; the numbering order of the data blocks; and a random order.

[0126] In the above scheme, the moving module 3114 is also used to perform the following processing on the at least part of the data block: reading data from at least part of the data pages in the data block and performing data verification to determine the number of bits in which the data is erroneous; recording the number of bits in the detection file of the storage device; wherein the at least part of the data pages includes at least one of the following: data pages in the data block whose number of errors exceeds a third threshold; data pages randomly selected from the data block; data pages specified in the data block; data pages selected in the data block according to an incremental algorithm.

[0127] In the above solution, the moving module 3114 is further used to perform the following processing on at least part of the data block: copying the data in the data block to a blank data block of the storage device, and when the copying is completed, erasing all the data in the data block.

[0128] In the above solution, the migration module 3114 is further configured to copy the valid data in the data block to a blank data block of the storage device, and when the copying is completed, perform an erasing operation on the valid data and invalid data in the data block.

[0129] The present invention provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the data processing method of the storage device described in the present invention.

[0130] The embodiment of the present application provides a computer-readable storage medium storing executable instructions, wherein the executable instructions are stored. When the executable instructions are executed by a processor, the processor will cause the processor to execute the data processing method of the storage device provided by the embodiment of the present application, for example, Figure 4A 、 4B 4C. In some embodiments, the computer-readable storage medium may be a storage device such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface storage device, optical disk, or CD-ROM; or may be any device including one or any combination of the above storage devices.

[0131] In summary, the difference between the power-on time and the power-off time of the storage device in the embodiment of the present application is the time period when the storage device is powered off / off, that is, the power-off duration of the storage device. For example, the new version of the NVMe protocol supports the solid-state drive to record the system time of the host device, that is, the power-off time and power-on time of the storage device can be obtained by recording the system time, and then the power-off duration of the storage device can be determined. Thus, the corresponding data migration strategy is determined according to the power-off duration, which can improve the stability of the data in the storage device; the data migration frequency of the storage device is adjusted based on the power-off duration, and based on the data migration frequency, the data blocks that need to be moved in the storage device are moved to correct the data instability caused by the power-off time within the stable operation time of the storage device, thereby improving the stability of the data in the storage device; the data migration operation strategy for the storage device is determined according to the power-off time of the storage device. When the power-off time of the storage device exceeds the maximum limit (that is, the maximum time that the storage device can operate stably), the data migration operation is immediately started, and the moved data is error detected, the status of the storage device is recorded, and when the storage device fails, the user can be prompted to the risk of the storage device. When moving data between storage devices, the data volume of the read / write operation is considered to ensure the efficiency of the read / write operation and to correct data errors in the storage device as soon as possible through the data movement operation. When reading and processing data pages in a data block, partial data pages in the data block can be selected to improve the efficiency of scanning processing while ensuring the detection of data errors.

[0132] The above description is merely an embodiment of the present application and is not intended to limit the scope of protection of the present application. Any modifications, equivalent replacements, and improvements made within the spirit and scope of the present application are included in the scope of protection of the present application.

Claims

1. A data processing method for a storage device, characterized in that: include: Get the power-on time and power-off time of the storage device; Determining a power-off duration of the storage device based on the power-on time and the power-off time; When the power-off duration is less than the maximum duration of stable operation of the storage device, determining a difference between the power-off duration and the maximum duration of stable operation of the storage device; using a ratio of the total amount of data blocks to the difference as a data migration frequency of at least part of the data blocks in the storage device; The total amount of data blocks is the number of at least some of the data blocks in the storage device; performing a data migration operation on at least part of the data blocks in the storage device according to the data migration frequency; When the power-off duration is greater than or equal to the maximum duration of stable operation of the storage device, a data migration operation is immediately started for at least part of the data blocks in the storage device after the storage device is powered on.

2. The method according to claim 1, characterized in that Obtaining the power-on time and power-off time of the storage device includes: When the storage device is powered on, it receives the power-on time sent by the host device, and Searching the power-off time recorded when the power is off from the storage device; Wherein, there is an electrical connection between the host device and the storage device.

3. The method according to claim 1, characterized in that During the data migration operation, the method further includes: Performing error detection on data in the data block moved in the storage device to obtain the number of erroneous bits of the data; Determine the state of the storage device by comparing the relationship between the number of bits and a first bit number threshold, and record the state in the storage device; The status of the storage device includes: normal status, warning status, and error status.

4. The method according to claim 1, wherein The method further comprises: When the power-on time or the power-off time of the storage device is not obtained, scanning and processing at least part of the data blocks in the storage device; When the number of erroneous bits in any data block exceeds a second bit number threshold, performing a data migration operation on the any data block; The scanning process sequence includes: the time sequence of writing the data blocks; the numbering sequence of the data blocks; and a random sequence.

5. The method according to claim 4, characterized in that The scanning and processing of at least part of the data blocks in the storage device includes: The following processing is performed on at least part of the data block: Reading data from at least part of the data pages in the data block and performing data verification to determine the number of erroneous bits of the data; recording the bit number in a detection file of the storage device; The at least part of the data pages includes at least one of the following: a data page in the data block whose number of errors exceeds a third threshold; a data page randomly selected from the data block; a data page specified in the data block; a data page selected in the data block according to an incremental algorithm.

6. The method according to claim 1, characterized in that The performing a data migration operation on at least part of the data blocks in the storage device includes: The following processing is performed on at least part of the data block: The data in the data block is copied to a blank data block of the storage device, and when the copying is completed, an erasing operation is performed on all the data in the data block.

7. A data processing device for a storage device, characterized in that: include: An acquisition module is used to obtain the power-on time and power-off time of the storage device; A first determining module, configured to determine a power-off duration of the storage device based on the power-on time and the power-off time; a second determining module configured to, when the power-off duration is less than the maximum duration of stable operation of the storage device, determine a difference between the power-off duration and the maximum duration of stable operation of the storage device; and use a ratio of a total number of data blocks to the difference as a data migration frequency of at least some of the data blocks in the storage device; wherein the total number of data blocks is the number of at least some of the data blocks in the storage device; A migration module is used to perform data migration operations on at least part of the data blocks in the storage device according to the data migration frequency; when the power-off duration is greater than or equal to the maximum duration of stable operation of the storage device, immediately start the data migration operation on at least part of the data blocks in the storage device after the storage device is powered on.

8. A storage device, characterized in that: include: Memory for storing data; A processor storing executable instructions for implementing the data processing method of the storage device according to any one of claims 1 to 6 when executed by the processor.

9. A computer-readable storage medium storing executable instructions, characterized in that: When the executable instructions are executed by a processor, the data processing method of the storage device according to any one of claims 1 to 6 is implemented.

10. A computer program product comprising computer instructions, characterized in that When the computer instructions are executed by a processor, the data processing method of the storage device according to any one of claims 1 to 6 is implemented.

Images